Optimizing ChatGPT

Abstract

INTRODUCTION: Patients rely on online searches for patient education materials (PEMs). PEMs are recommended to be written at or below a sixth-grade reading level but are regularly written at a college reading level. Using prompt engineering, we assess the information, misinformation, and readability of ChatGPT responses to urologic oncology questions. METHODS: Forty-five questions relating to prostate, bladder, and kidney cancer were presented to ChatGPT (version 4o, OpenAI). Quality of health information was assessed using DISCERN (1 [low] to 5 [high]). Understandability and actionability were assessed using PEMAT-P (0 [low] - 100% [high]). Misinformation was scored from 1 [no misinformation] to 5 [high misinformation]. Grade and reading level were calculated using the Flesch-Kincaid scale [5 (easy) to 16 (difficult), and 100-90 (5th grade level) to 10-0 (professional level), respectively]. Prompt engineering was then applied to responses and evaluated. RESULTS: grade], p<0.0001), all significantly improved. Misinformation did not change significantly. CONCLUSIONS: Using prompt engineering, ChatGPT provides highly accurate and understandable PEMs at a patient appropriate reading level and provides concrete resources for patient action. Urologists should understand prompt engineering and be involved in the development of artificial chatbots to optimize results.

Ähnliche Arbeiten