MULTIMODAL CANCER DETECTION USING MACHINE LEARNING

Abstract

Early and accurate cancer diagnosis remains one of the most critical challenges in modern healthcare. Traditional unimodal diagnostic approaches—such as radiology, histopathology, or genomics—offer valuable but fragmented insights, often resulting in incomplete clinical interpretations. This research proposes an interpretable cross-modal attention-based deep learning framework for multimodal cancer detection, designed to integrate heterogeneous data sources including radiological images, histopathology slides, genomic profiles, and structured clinical records. Each modality is processed through a dedicated deep encoder—3D CNNs for imaging, Vision Transformers (ViTs) for histopathology, transformer-based sequence models for genomics, and MLPs for clinical data—to extract high-level feature representations. These are fused using a cross-modal attention mechanism that dynamically learns intermodality relationships and gracefully handles missing data. The attention weights further serve as an intrinsic interpretability feature, revealing the contribution of each modality to the final diagnostic decision. Experimental validation on benchmark multimodal datasets demonstrates that the proposed framework achieves superior accuracy, robustness, and generalization compared to unimodal and existing fusion-based models. Moreover, a Clinical Decision Support Dashboard provides transparent visual explanations through saliency maps and modality importance scores, fostering clinician trust and practical usability. The results highlight the potential of interpretable multimodal AI to transform diagnostic precision, reduce uncertainty, and advance personalized cancer care.