Trustworthy Multimodal AI Agents for Early Breast Cancer Detection and Clinical Decision Support

Abstract

Timely and precise identification of breast cancer is crucial for enhancing clinical outcomes; however, current AI systems frequently exhibit deficiencies in transparency, trustworthiness, and the capacity to assimilate varied data modalities. We introduce a reliable, multi-agent, multimodal AI system for individualised early breast cancer diagnosis, created on the CBIS-DDSM dataset. The system consists of four specialised agents that cooperatively analyse diverse data. An Imaging Agent employs convolutional and transformer-based models to analyse mammograms for lesion classification and localisation; a Clinical Agent extracts structured features including breast density (ACR), view type (CC/MLO), laterality, mass shape, margin, calcification type and distribution, BI-RADS score, pathology status, and subtlety rating utilising optimised tabular learning models; a Risk Assessment Agent integrates outputs from the imaging and clinical agents to produce personalised malignancy predictions; and an Explainability Agent provides role-specific interpretations through Grad-CAM for imaging, SHAP for clinical features, and natural language explanations customised for radiologists, general practitioners, and patients. Predictive dependability is assessed by Expected Calibration Error (ECE) and Brier Score. The framework employs a modular design with a Streamlit interface, facilitating both comprehensive deployment and interactive demonstration. This paradigm enhances the creation of reliable AI systems for clinical decision assistance in oncology by the integration of strong interpretability, personalised risk assessment, and smooth multimodal integration.

Ähnliche Arbeiten