Multimodal Outer Arithmetic Block Dual Fusion of Whole Slide Images and Omics Data for Precision Oncology
Posted:
Our Contributions
- We introduce a novel dual fusion network that seamlessly incorporates both early and late fusion approaches, enabling detailed integration of molecular and imaging data at patch and slide levels.
- Our approach develops a unique Multimodal Outer Arithmetic Block (MOAB) fusion strategy that enhances cross-modal feature interaction and improves the model’s ability to capture complex tumor subtype features.
- MOAD-FNet, is the first imaging-omics pipeline to leverage the NHNN BRAIN UK dataset, demonstrating exceptional performance in brain tumor subtyping. Extensive ablation studies on TCGA datasets validate its robustness, achieving state-of-the-art survival prediction on the BLCA dataset and on par results on the BRCA dataset, showcasing its versatility across multimodal oncology tasks.