Assessing the accuracy, safety, and clinical utility of AI-generated rehabilitation guidance for Bankart lesions: a blinded comparative evaluation with expert protocols

Abstract

Aims: Artificial Intelligence (AI)–based language models are increasingly used to generate medical information and patient education materials. However, the reliability and safety of AI-generated rehabilitation guidance remain uncertain. This study aimed to evaluate the accuracy, safety, clinical utility, and readability of rehabilitation recommendations generated by ChatGPT-5 for Bankart lesions and to compare these outputs with expert-developed rehabilitation protocols.Methods: A blinded, cross-sectional comparative quality assessment was conducted. Standardized prompts regarding nonoperative and postoperative Bankart rehabilitation were used to generate responses from ChatGPT-5. AI-generated texts were compared with protocols prepared by a panel of orthopedic shoulder surgeons and an experienced physiotherapist. All texts were anonymized and independently evaluated by three blinded expert raters using a structured 5-point Likert scale assessing clinical accuracy, safety, actionability, comprehensiveness, and overall quality. Major clinical errors were recorded separately. Readability was assessed using Flesch Reading Ease and Flesch–Kincaid Grade Level scores. Inter-rater reliability was analyzed using intraclass correlation coefficients (ICC). Results: A total of 20 rehabilitation texts (10 AI-generated and 10 expert-developed) were evaluated. Expert protocols demonstrated significantly higher scores in clinical accuracy (4.6±0.4 vs 3.4±0.7, p

Ähnliche Arbeiten