MedXAI-MM: A Unified Multi-Modality Explainable Artificial Intelligence Framework for Clinical Medical Imaging

Abstract

Explainable Artificial Intelligence (XAI) is now crucial for ensuring the safe use of deep learning models in hospitals and clinics. This holds especially true for medical imaging, where reliably informed diagnoses are difficult to make when the model’s workings are unknown. Although existing interpretability methods, such as saliency maps, attribution techniques, and prototype-based reasoning, give some insights into how models work, they typically address only one type of inputs at a time. In real clinical work, though, doctors often deal with multimodal imaging like MRI, CT, and retinal scans. Each one brings its own resolution, contrast, and anatomical details. Therefore, this paper presents MedXAIMM taht is a unified framework for multi-modality explainability. The framework pulls together three main parts. First, there is Grad-CAM++ for spotting spatial saliency. Second, DeepSHAP handles pixel-level feature attribution. Third, Case-Based Retrieval, or CBR, adds clinical context through evidence from similar cases. The setup uses a hybrid backbone of ResNet-50 and Swin Transformer. This allows for pulling out both local and global features in a complementary way. Then, attention-based fusion helps with learning representations that account for different modalities. We also bring in a new metric called the Cross-Modality Fidelity Score, or CMFS. It measures how consistent explanations stay across various imaging types that differ a lot. Tests on datasets like BraTS-MRI, ChestX-ray14, and DRIVE show strong results. MedXAI-MM reaches higher faithfulness, (up to $+18 \%$) up to eighteen percent better. It also improves localization IoU by twelve percent $(+12 \%)$. Plus, clinicians rate its interpretability higher than top baselines. Overall, the findings point to how unified multimodal interpretability can bridge the gap between accurate diagnostics and clear transparency in medicine. This pushes AI closer to being ready for everyday clinical use.

Ähnliche Arbeiten