Tumor Board–Inspired Multiagent Artificial Intelligence System for Interpreting Oncology Guidelines

Abstract

PURPOSE Clinical guidelines are essential for evidence-based oncology care but are often long, complex, and difficult to navigate. We developed a multiagent artificial intelligence (AI) system to accurately retrieve and interpret guideline content in response to guideline-based clinical questions. METHODS We included 34 ASCO guidelines published between January 2021 and December 2024. Using a multiagent framework, we assigned distinct roles to AI agents: a Coordinator Agent selected the relevant guideline, specialized Tumor Board Agents extracted information from text, tables, and figures, and a Reviewer Agent synthesized a final answer. A total of 100 open-ended questions were created on the basis of the guideline content. The system's performance was compared with GPT-4o, Claude 3.7, Gemini 2.5 flash, DeepSeek-R1, and the ASCO Guidelines Assistant. RESULTS The multi-agent system achieved (94% [95% CI, 89.3 to 98.7]) accuracy in selecting the correct guidelines and (90% [95% CI, 84.1 to 95.9]) accuracy in answering questions. This significantly outperformed GPT-4o (48%), Claude 3.7 (49%), Gemini 2.5 (50%), DeepSeek-R1 (58%), and the ASCO Guidelines Assistant (67%, all P < .01, McNemar's test). Most errors were due to incorrect guideline selection or misinterpretation; no hallucinated answers were observed. Removing the Coordinator Agent reduced accuracy to 40%, and excluding tables and figures reduced accuracy to 51%. CONCLUSION By assigning specialized tasks to AI agents and incorporating visual elements from clinical guidelines, our system outperformed existing tools in accurately answering oncology questions. This pilot study, limited to ASCO guidelines, may improve access to guideline-based care.

Ähnliche Arbeiten