<scp>ChatGPT</scp> Across Domains: A Systematic Review of Applications, Evaluation Approaches, and Open Challenges

Abstract

ABSTRACT In recent years, there has been a rise in the use of ChatGPT for education, healthcare, smart cities and emerging technologies. However, no available studies or reviews have provided a consistent and reliable depiction of the situation regarding its usage and evaluation. The reporting of datasets, evaluation indicators, factors influencing performance and conditions of deployment has also varied from study to study. This fragmented state of affairs seriously inhibits attempts to assess ChatGPT's capabilities and limitations and thus improve the design of future versions. Earlier reviews were often conducted in a way pertaining to a single area or were mainly descriptive, with less emphasis on methodological evaluation and issues in deployment and ethics. To fill this void, we undertook a systematic review, according to PRISMA guidelines, limiting our searches to English‐language journal articles published during 2021–2025 by reputable publishers. Such studies focused directly on GPT models, provided assessment conditions and were cited extensively as preprints, while the exclusion criteria encompassed poorly linked studies, those not in English and studies employing ChatGPT as an adjunct. An analysis of these studies revealed that the vast majority of research has taken place in the area of education (32%) and health (28%). The review revealed significant variation in assessment accuracy across domains, frequent challenges with doubtful sensitivity, unpredictable and rapid changes and risks associated with specific domains that impact reliability and safety. This study's primary contribution is an effort to develop an integrated analytical framework that puts together these interdisciplinary results in a streamlined manner for interpreting the capabilities and limitations of ChatGPT. Because of the methodological heterogeneity of existing studies, the results can be viewed as qualitative trends instead of standard quantitative evidence. The results thereby accentuate the need for consistent criteria, domain‐informed evaluation practices and stronger methodological reporting to underpin a more reliable deployment of ChatGPT‐based systems.

Ähnliche Arbeiten