S2144 Chatbots and Ethical Medical Practice: A Detailed Analysis of Chat GPT and Google GEMINI in Gastroenterology

Abstract

Introduction: The integration of chatbots in gastroenterology presents a promising frontier for enhancing patient care through artificial intelligence. This study delves into their ability to navigate nuanced medical and ethical scenarios, crucial for sensitive healthcare domains. By rigorously evaluating chatbots like ChatGPT and Google GEMINI, we aim to determine their effectiveness in interpreting complex cases, ensuring they meet the high standards required for ethical decision-making and accurate medical advisement in real-world clinical settings. Methods: The study assessed the medical and ethical reliability of 2 gastroenterology chatbots, ChatGPT, and Google GEMINI, through a questionnaire evaluated by 5 independent reviewers. Responses were rated on a 1-10 Likert scale. We analyzed both chatbots' accuracy, reliability, and user interface using a dataset from various questionnaire categories, calculating the mean and standard deviation. Independent t-tests evaluated statistical differences. Missing data were imputed using column means. Visualizations such as bar charts and scatter plots highlighted metric comparisons. Results: The study evaluated the performance of ChatGPT 4o and GOOGLE GEMINI across 3 criteria: accuracy, reliability, and user interface, based on data from various questionnaire categories. Descriptive statistics showed that ChatGPT 4o had higher mean scores in accuracy (7.80 vs 3.90), reliability (7.76 vs 3.67), and user interface (7.99 vs 3.51). T-tests confirmed these differences as statistically significant with p-values less than 0.05 for all criteria (accuracy: =2.53×10−32P =2.53×10−32, reliability: p=9.38×10−28P =9.38×10−28, and user interface: p=2.00×10−48P =2.00×10−48). These results suggest that ChatGPT 4o is more accurate, reliable, and user-friendly than GOOGLE GEMINI, indicating its potential suitability for supporting gastroenterologists. Conclusion: The study's results underscore the superior performance of ChatGPT 4o over GOOGLE GEMINI in all evaluated criteria—accuracy, reliability, and user interface. The statistical significance of these outcomes highlights the potential of advanced chatbots to support gastroenterologists effectively. However, while ChatGPT shows promise in understanding complex medical and ethical scenarios, continuous monitoring and iterative updates will be crucial to address any evolving challenges and ensure the technology remains a reliable adjunct in clinical decision-making (see Figure 1, Table 1).Figure 1.: ChatGPT vs Google GEMINI on ethical and medical practice in the field of Gastroenterology. Table 1. - Comparative Analysis of ChatGPT 4o and GOOGLE GEMINI on Accuracy, Reliability, and User Interface Criterion Mean ChatGPT 4o Std Dev ChatGPT 4o Mean GOOGLE GEMINI Std Dev GOOGLE GEMINI p-value Accuracy 7.80 0.67 3.90 1.17 < 0.01 Reliability 7.76 0.63 3.67 1.42 < 0.01 User Interface 7.99 0.71 3.51 0.69 < 0.01

Ähnliche Arbeiten