Enabling Episode-Level Transparency in Value-Based Care Through Large Language Model-Driven Provider Directories

Abstract

Conventional provider directories, as a cornerstone interface, remain a critical yet structurally fragile component of the United States healthcare system, limiting transparency and constraining the effectiveness of value-based care (VBC). Conventional directory interfaces lack episode-level cost and risk context, rely on rigid search paradigms, and can contain inaccurate or incomplete information. These deficiencies hinder informed provider selection and weaken the operational impact of episode-based payment models. This study evaluates a large language model (LLM)-driven provider directory chatbot designed to support episode-based care navigation using strictly structured, synthetic cost and performance datasets. Four widely used LLMs, i.e., GPT-3.5-turbo, GPT-4o-mini, GPT-4o, and GPT-5.1, were assessed under identical deterministic conditions. Using 87 natural language test scenarios, we examined structured output validity, episode and risk-band identification, provider ranking accuracy, numeric fidelity, hallucination risk, abstention behavior, and operational performance. We further introduce a revised formulation of ranking correctness that explicitly treats accurate episode identification as a prerequisite for meaningful transparency. All models demonstrated consistently high episode identification accuracy, approaching 91%, though substantial variability was observed in downstream provider ranking reliability and numeric precision. Collectively, these preliminary findings suggest that LLM-enabled provider directories can meaningfully enhance transparency and the user experience within VBC settings, while highlighting specific performance dimensions that require optimization before large-scale deployment.

Ähnliche Arbeiten