Ability for Artificial Intelligence to Solve United States Medical Licensing Exam Ethics Questions: Can AI replace social science-based thinking?

Abstract

Introduction: Artificial intelligence (AI) emergence has changed the medical education landscape. United States’ medical students must learn empathetic thinking for United States Medical Licensing Exams (USMLE). This study evaluates three AI platforms in answering such questions. Methods: Ethics questions from UWorld and Amboss were accumulated for Step 1 and 2. Multiple-choice questions were entered into ChatGPT, Gemini, and Perplexity, yielding a correct/incorrect response. Percentages of correct responses were compared between platforms and to student averages. ANOVA and t-tests conducted determined statistical significance, the upper threshold being P=0.05. Results: 1 09 UWorld and 63 Amboss questions were available for Step 1. 189 UWorld and 185 Amboss questions were available for Step 2. Google Gemini had highest accuracy for Step 1 (86.6%) while ChatGPT had the highest accuracy for Step 2 (83.6%). All platforms outperformed the student average for Step 1 (68.5%) and Step 2 (68.9%); ChatGPT and Gemini doing so significantly for Step 1 (p < 0.01), and ChatGPT doing so significantly for Step 2 (p < 0.01). Conclusions: It is critical to understand whether empathetic thinking can be replicated by technology or prepare students. This study highlights the ability for AI to solve ethical dilemmas that students may struggle with.

Ähnliche Arbeiten