From algorithms to empathy: can large language models effectively answer patients’ questions in restorative dentistry?

Abstract

The aim of this study was to compare the responses given by the large language models (LLMs) ChatGPT-5, Gemini 2.5 Pro, DeepSeek-V3.2, and Claude Sonnet-4.5 in terms of accuracy, empathy, and readability, to frequently asked patient questions about restorative dentistry. Twenty-five open-ended questions were posed in English to ChatGPT-5, Gemini 2.5 Pro, DeepSeek-V3.2-Exp, and Claude Sonnet-4.5 models in individual, independent sessions. Accuracy was scored using a 5-point Likert-type scale and empathy a 3-point Likert-type scale by two experienced evaluators, and inter-rater reliability was calculated using the Intraclass Correlation Coefficient (ICC). Readability was evaluated through an online platform using the FRES, FKGL, GFI, SMOG, and CLI indexes. Group comparisons were performed using ANOVA/Kruskall-Wallis and post-hoc Dunn-Bonferroni tests according to the normality distribution of the data. The inter-rater reliability was found to be high score for accuracy and showed variability according to the model for empathy. Statistically significant differences were determined between the models in terms of accuracy (p < 0.001), with the highest mean value obtained by DeepSeek-V3.2 (4.8 ± 0.5), followed by Claude Sonnet-4.5 (4.28 ± 0.61) and ChatGPT-5 (4.12 ± 0.67), and the lowest accuracy was determined for Gemini 2.5 Pro (3.6 ± 0.5). Statistically significant differences were determined between the models in terms of empathy (p < 0.001), with the highest mean value obtained by DeepSeek-V3.2 (1.96 ± 0.2). The readability measurements showed that overall, DeepSeek-V3.2 produced more readable texts with higher FRES values (60.85 ± 8.2) (p < 0.001). Within the scope of this study, DeepSeek-V3.2 exhibited a comparatively more balanced performance profile across accuracy, empathy, and readability measures. While Claude Sonnet-4.5 and ChatGPT-5 showed high accuracy, the results for empathy and readability criteria were variable. No model can replace a clinical specialist, and these systems should be evaluated as decision-making tool under specialist supervision, especially at the stages of patient education and first providing information.

Ähnliche Arbeiten