Abstract P2-03-05: AI-Powered Breast Cancer Survivorship Support: A Comparative Analysis of ChatGPT and Gemini in Providing Evidence-Based Lifestyle Guidance

Abstract

Abstract Background: Breast cancer survivors are a growing population in the United States, with an increasing number of women facing challenges in navigating the complexities of post-treatment care. However, the transition to survivorship is often challenging, as many women struggle to adhere to recommended lifestyle guidelines, which are crucial for long-term health and well-being. As highlighted in a recent survey study of cancer survivorship programs accredited by the American College of Surgeons Commission on Cancer (CoC), there are significant gaps in the provision of certain services, particularly those related to sexual health and fertility1. Additionally, low patient awareness and lack of referrals remain barriers to accessing available resources. Integrating innovative solutions like large language models (LLMs) into breast cancer care could address these challenges and empower survivors to actively participate in their health and wellness journeys. This study aims to evaluate the potential of AI-powered chatbots, specifically ChatGPT and Gemini, to provide personalized, evidence-based guidance on exercise, diet, and weight management, consequently improving long-term health outcomes for breast cancer survivors. Methods: This study utilized the Exercise, Diet, and Weight Management During Cancer Treatment guidelines from the American Society of Clinical Oncology (ASCO) and the National Comprehensive Cancer Network (NCCN) to formulate 22 questions focused on preventive health, physical activity, nutrition, and weight management for cancer survivors. These questions were posed to ChatGPT-4 and Gemini. Responses were evaluated independently by 2 physicians who graded each response on 5 criteria: factual accuracy, relevance, completeness, clarity, and coherence, using a scale from 1 (poor) to 5 (excellent). Grading was analyzed and compared to determine their alignment with ASCO and NCCN guidelines. Results: ChatGPT demonstrated high performance in factual accuracy, with an average score of 4.52/5. Gemini exhibited a lower average score of 4.38 but achieved 75% of responses rated 4 or higher. In terms of relevance, ChatGPT maintained an average score of 4.43/5. Gemini performed well in relevance, with an average score of 4.29. For completeness, ChatGPTachieved an average score of 4.38/5. Gemini showed slightly higher performance in this criterion, with an average score of 4.48/5. Both models excelled in clarity, each attaining an average score of 4.57/5, with high ratings and minimal ambiguity in their responses. For coherence, both ChatGPT and Gemini demonstrated logical structuring, with average scores of 4.33/5. Conclusion: The findings highlight the potential of integrating LLMs into oncology for survivorship care. Both models demonstrated robust performance across all criteria. ChatGPT excelled in factual accuracy and relevance, while Gemini showed slightly better completeness. Both models achieved high clarity and coherence scores, indicating their ability to provide clear, comprehensive, and logically structured responses. This integration can significantly enhance adherence to survivorship guidelines, offering personalized, real-time support that improves patient education, risk communication, behavior modification, and systematic follow-up. Continued development and refinement of these models, with a focus on addressing specific needs and concerns of breast cancer survivors, could revolutionize survivorship care, leading to improved adherence to guidelines, better quality of life, and, improved long-term outcomes. Reference: 1 - Stal J, Miller KA, Mullett TW, et al. Cancer Survivorship Care in the United States at Facilities Accredited by the Commission on Cancer. JAMA Netw Open. 2024;7(7):e2418736. doi:10.1001/jamanetworkopen.2024.18736 Citation Format: Jasmin Hundal, Asfand Yar Cheema, Amna Zaheer, Mishaal Munir, Baidehi Maiti. AI-Powered Breast Cancer Survivorship Support: A Comparative Analysis of ChatGPT and Gemini in Providing Evidence-Based Lifestyle Guidance [abstract]. In: Proceedings of the San Antonio Breast Cancer Symposium 2024; 2024 Dec 10-13; San Antonio, TX. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(12 Suppl):Abstract nr P2-03-05.

Ähnliche Arbeiten