Large Language Models Support Delivery of Telemedicine in Pharmacy: A Cross-sectional Study

Abstract

Telemedicine has emerged as a critical modality in healthcare delivery, especially for elderly patients with complex medication regimens. Pharmacists play a central role in optimizing pharmacotherapy, and the integration of artificial intelligence tools, particularly large language models (LLMs), may enhance telepharmacy services by providing timely, accurate, and empathetic responses. Despite growing theoretical applications, limited evidence exists on how LLMs perform in pharmacy-relevant teleconsultations. However, little is known about whether LLMs can reliably address patient-specific pharmacotherapy concerns during teleconsultations. This study specifically investigated how different LLMs perform when applied to simulated, pharmacy-relevant scenarios in elderly care. This observational, cross-sectional study was conducted between May and June 2025. Ten simulated patient case scenarios representing common pharmacotherapeutic concerns in elderly populations were used as the dataset. Three LLMs (ChatGPT-4.0, DeepSeek-V3, and Gemini 2.5 Flash) were tested. Each LLM was prompted with identical patient scenarios, and responses were independently evaluated by two authors. A structured rubric assessed eight primary parameters: clarity, tone, empathy, patient-centeredness, accuracy, actionability, risk mitigation and safety accuracy. Scores (0–3 per domain, max 21) with negative score for safety accuracy and errors were estimated. Readability assessments were carried out using Flesch-Kincaid (FK) grade level and Flesch reading ease (FRE) scores along with public reach (%). Descriptive statistics and Kruskal-Wallis H test with Bonferroni corrections were used for comparative analysis. All three LLMs successfully generated responses for all ten case scenarios. Their outputs demonstrated high levels of empathy, patient-specific guidance, clinical appropriateness; and excelled in emotional tone and structural clarity. Overall, Gemini scores (20+0.7) were statistically significantly higher compared to ChatGPT (18.6+1.1; P=0.002), and DeepSeek (19+0.8; P=0.031). Regarding the reading assessment, FK grade levels were significantly higher with Gemini (8.9+0.6) compared to DeepSeek (7.5+1.1; P=0.002) as like public reach ((79.3+3.7) vs. (84+1.9); P=0.001). All their responses were graded excellent except DeepSeek for case 1 and ChatGPT for case 7 where their responses were good. DeepSeek stood out for tiered, instructional formatting and inclusion of safety tips such as pill organizers and warning signs; while Gemini delivered detailed pharmacologic explanations and emphasized urgent care needs when appropriate. Variability was noted in verbosity and response structuring, with Gemini being more narrative and ChatGPT was more conversational. All LLMs displayed accurate drug-related knowledge and patient-centered communication aligned with best practices in telepharmacy. LLMs consistently generated accurate, empathetic, and actionable responses to simulated telepharmacy scenarios. Their use may serve as a supportive tool in delivering high-quality virtual pharmacy consultations. Rigorous real-world validation and comparative trials against pharmacist-delivered care are required before LLMs can be responsibly integrated into clinical workflows.

Ähnliche Arbeiten