Quality of ChatGPT-Generated Responses to Common Patient Questions About Peripheral Nerve Stimulation: A Cross-Sectional Study

Abstract

Background: Peripheral nerve stimulation (PNS) is increasingly used in selected patients with neuropathic pain, and many individuals seek supplemental online information to clarify procedural expectations and postoperative care. Large language models such as ChatGPT may provide scalable patient education; however, their performance for PNS-related questions has not been evaluated. This study assessed the reliability, accuracy, and comprehensibility of ChatGPT-5.0 responses to common PNS patient questions. Methods: We conducted a cross-sectional evaluation of ChatGPT-5.0 responses to 21 standardized questions derived through expert consensus, spanning pre-implantation, implantation, and post-implantation domains. Sixteen board-certified interventional pain specialists and a nurse educator independently rated each response using validated scales for reliability (1–6), accuracy (1–3), and comprehensibility (1–3). Descriptive statistics were calculated, and domain-level patterns were examined. Results: Clinician ratings demonstrated generally strong performance across all domains. Mean reliability was 4.7 ± 1.4, mean accuracy 2.6 ± 0.6, and mean comprehensibility 2.8 ± 0.5. Foundational questions addressing mechanisms, expectations, and postoperative care received the highest ratings. Lower ratings were observed for implantation-focused items requiring procedural nuance. No response fell below predefined acceptability thresholds, and sensitivity analyses confirmed that including one partial evaluator did not alter the observed trends. Conclusions: ChatGPT-5.0 generated responses to PNS-related patient questions that clinicians rated as generally reliable, accurate, and understandable, particularly for foundational and postoperative topics. Performance was more variable for procedural questions, underscoring the need for clinician oversight and verification. These findings provide a benchmark of current LLM capabilities and highlight the importance of ongoing evaluation as models evolve and as patients access versions with differing functionalities.

Ähnliche Arbeiten