Efficacy of ChatGPT in personalized glucose-lowering strategy development: a clinician-based comparative study

Abstract

Background: The increasing incidence of diabetes poses a significant burden on healthcare systems. Limited research exists on tools to assist providers in developing personalized glucose-lowering strategies, which could alleviate this pressure and enhance patient outcomes. Objective: This study aims to evaluate the capability of ChatGPT-4o in developing personalized glucose-lowering strategies for individuals with diabetes. Methods: First, an evaluation of ChatGPT-4o's performance on China's qualification examination for attending physicians in endocrinology. Second, a cross-sectional study was conducted, involving the comparison of glucose-lowering strategies formulated by ChatGPT-4o, general practitioners (GPs), and attending physicians (APs) in endocrinology for a set of 30 real-world diabetes cases. Three clinical experts scored blindly the reasonableness of each strategy on a scale, with stratification of cases into three complexity levels (A, B, and C) and evaluation of mean scores for each level. Results: 86.35 ± 4.142; p = .0467). Performance declined with increasing case complexity, with mean scores dropping from 89.90 ± 2.936 for simple cases (A-level) to 76.12 ± 11.93 for complex cases (C-level) (p <.0020). Conclusions: ChatGPT-4o performs reliably in generating glucose-lowering strategies for simpler diabetes cases, highlighting its potential to assist community health workers. However, its accuracy in complex cases, especially concerning medication contraindications, requires improvement.

Ähnliche Arbeiten