{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T21:53:40Z","timestamp":1781733220026,"version":"3.54.5"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Qatar Research Development and Innovation Council","award":["ARG01-0513-230141"],"award-info":[{"award-number":["ARG01-0513-230141"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Accurate breast cancer survival prediction using multi-modal data is vital for enhancing clinical decisions. This study evaluates deep learning based fusion strategies, early, intermediate, late, and a hybrid approach, to integrate histopathology images and genomic data for one year survival prediction. We developed a robust evaluation framework, employing tailored deep learning architectures and metrics including accuracy, precision, recall, F1 score, and AUC. Model performance was validated using Kaplan\u2013Meier curves and log-rank tests, with SHAP-based feature importance analysis enhancing interpretability. Results highlight the strengths and limitations of each fusion strategy, offering insights into optimal multi-modal learning approaches for breast cancer prognosis. Our findings underscore the importance of selecting task specific fusion methods, providing a reproducible, interpretable framework to advance survival prediction. All code and configurations are publicly available.<\/jats:p>","DOI":"10.1007\/s40747-025-02133-y","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T03:33:47Z","timestamp":1764646427000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Integrating histopathology and genomic data: a comparative study of fusion methods for breast cancer survival prediction"],"prefix":"10.1007","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7175-4326","authenticated-orcid":false,"given":"Younes","family":"Akbari","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Faseela","family":"Abdullakutty","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Somaya","family":"Al Maadeed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed","family":"Bouridane","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rifat","family":"Hamoudi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"2133_CR1","doi-asserted-by":"crossref","unstructured":"Abdullakutty F, Akbari Y, Al-Maadeed S, Bouridane A, Hamoudi R (2024a) Transforming tabular data for multi-modality: enhancing breast cancer metastasis prediction through data conversion. In: 2024 IEEE international conference on image processing challenges and workshops (ICIPCW). IEEE, pp 4149\u20134155","DOI":"10.1109\/ICIPCW64161.2024.10769174"},{"key":"2133_CR2","doi-asserted-by":"publisher","first-page":"1450103","DOI":"10.3389\/fmed.2024.1450103","volume":"11","author":"F Abdullakutty","year":"2024","unstructured":"Abdullakutty F, Akbari Y, Al-Maadeed S, Bouridane A, Talaat IM, Hamoudi R (2024) Histopathology in focus: a review on explainable multi-modal approaches for breast cancer diagnosis. Front Med 11:1450103","journal-title":"Front Med"},{"key":"2133_CR3","volume":"1","author":"F Abdullakutty","year":"2024","unstructured":"Abdullakutty F, Akbari Y, Al-Maadeed S, Bouridane A, Talaat IM, Hamoudi R (2024) Towards improved breast cancer detection via multi-modal fusion and dimensionality adjustment. Computat Struct Biotechnol Rep 1:100019","journal-title":"Computat Struct Biotechnol Rep"},{"key":"2133_CR4","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","volume":"57","author":"Y Benjamini","year":"1995","unstructured":"Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B (Methodol) 57:289\u2013300","journal-title":"J R Stat Soc Ser B (Methodol)"},{"key":"2133_CR5","doi-asserted-by":"crossref","unstructured":"Cao W, Li C, Yang L, Yin M, Chen C, Kobsiriphat W, Utakapan T, Yang Y, Yu H, Wu X (2024) A fusion network with stacked denoise autoencoder and meta learning for lateral walking gait phase recognition and multi-step-ahead prediction. IEEE J Biomed Health Inform","DOI":"10.1109\/JBHI.2024.3380099"},{"key":"2133_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/gb-2006-7-10-r100","volume":"7","author":"AE Carpenter","year":"2006","unstructured":"Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, Guertin DA, Chang JH, Lindquist RA, Moffat J et al (2006) Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol 7:1\u201311","journal-title":"Genome Biol"},{"key":"2133_CR7","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","volume":"41","author":"RJ Chen","year":"2020","unstructured":"Chen RJ, Lu MY, Wang J, Williamson DF, Rodig SJ, Lindeman NI, Mahmood F (2020) Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis. IEEE Trans Med Imaging 41:757\u2013770","journal-title":"IEEE Trans Med Imaging"},{"key":"2133_CR8","doi-asserted-by":"crossref","unstructured":"Chen RJ, Lu MY, Weng W-H, Chen TY, Williamson DF, Manz T, Shady M, Mahmood F (2021) Multimodal co-attention transformer for survival prediction in gigapixel whole slide images. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp 4015\u20134025","DOI":"10.1109\/ICCV48922.2021.00398"},{"key":"2133_CR9","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1016\/j.ccell.2022.07.004","volume":"40","author":"RJ Chen","year":"2022","unstructured":"Chen RJ, Lu MY, Williamson DF, Chen TY, Lipkova J, Noor Z, Shaban M, Shady M, Williams M, Joo B et al (2022) Pan-cancer integrative histology-genomic analysis via multimodal deep learning. Cancer Cell 40:865\u2013878","journal-title":"Cancer Cell"},{"key":"2133_CR10","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1093\/bioinformatics\/btx723","volume":"34","author":"J Cheng","year":"2018","unstructured":"Cheng J, Mo X, Wang X, Parwani A, Feng Q, Huang K (2018) Identification of topological features in renal tumor microenvironment associated with patient survival. Bioinformatics 34:1024\u20131030","journal-title":"Bioinformatics"},{"key":"2133_CR11","doi-asserted-by":"publisher","first-page":"e91","DOI":"10.1158\/0008-5472.CAN-17-0313","volume":"77","author":"J Cheng","year":"2017","unstructured":"Cheng J, Zhang J, Han Y, Wang X, Ye X, Meng Y, Parwani A, Han Z, Feng Q, Huang K (2017) Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis. Cancer Res 77:e91\u2013e100","journal-title":"Cancer Res"},{"key":"2133_CR12","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1006076","volume":"14","author":"T Ching","year":"2018","unstructured":"Ching T, Zhu X, Garmire LX (2018) Cox-nnet: an artificial neural network method for prognosis prediction of high-throughput omics data. PLoS Comput Biol 14:e1006076","journal-title":"PLoS Comput Biol"},{"key":"2133_CR13","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1038\/nature10983","volume":"486","author":"C Curtis","year":"2012","unstructured":"Curtis C, Shah SP, Chin S-F, Turashvili G, Rueda OM, Dunning MJ, Speed D, Lynch AG, Samarajiwa S, Yuan Y et al (2012) The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486:346\u2013352","journal-title":"Nature"},{"key":"2133_CR14","doi-asserted-by":"crossref","unstructured":"Ding S, Li J, Wang J, Ying S, Shi J (2024) Multimodal co-attention fusion network with online data augmentation for cancer subtype classification. IEEE Transact Med Imag","DOI":"10.1109\/TMI.2024.3405535"},{"key":"2133_CR15","doi-asserted-by":"crossref","unstructured":"Duan J, Xiong J, Li Y, Ding W (2024) Deep learning based multimodal biomedical data fusion: an overview and comparative review. In: Information fusion, p 102536","DOI":"10.1016\/j.inffus.2024.102536"},{"key":"2133_CR16","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1080\/01621459.1961.10482090","volume":"56","author":"OJ Dunn","year":"1961","unstructured":"Dunn OJ (1961) Multiple comparisons among means. J Am Stat Assoc 56:52\u201364","journal-title":"J Am Stat Assoc"},{"key":"2133_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2022.104111","volume":"79","author":"A Elaanba","year":"2023","unstructured":"Elaanba A, Ridouani M, Hassouni L (2023) A stacked generalization chest-X-ray-based framework for mispositioned medical tubes and catheters detection. Biomed Signal Process Control 79:104111","journal-title":"Biomed Signal Process Control"},{"key":"2133_CR18","doi-asserted-by":"crossref","unstructured":"Farooq A, Mishra D, Chaudhury S (2025) Survival prediction in lung cancer through multi-modal representation learning. In: 2025 IEEE\/CVF winter conference on applications of computer vision (WACV). IEEE, pp 3907\u20133915","DOI":"10.1109\/WACV61041.2025.00384"},{"key":"2133_CR19","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.102488","volume":"66","author":"J Fu","year":"2021","unstructured":"Fu J, Li W, Du J, Huang Y (2021) A multiscale residual pyramid attention network for medical image fusion. Biomed Signal Process Control 66:102488","journal-title":"Biomed Signal Process Control"},{"key":"2133_CR20","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1109\/TCBB.2021.3083566","volume":"19","author":"J Gao","year":"2021","unstructured":"Gao J, Lyu T, Xiong F, Wang J, Ke W, Li Z (2021) Predicting the survival of cancer patients with multimodal graph neural network. IEEE\/ACM Trans Comput Biol Bioinf 19:699\u2013709","journal-title":"IEEE\/ACM Trans Comput Biol Bioinf"},{"key":"2133_CR21","doi-asserted-by":"publisher","first-page":"63373","DOI":"10.1109\/ACCESS.2019.2916887","volume":"7","author":"W Guo","year":"2019","unstructured":"Guo W, Wang J, Wang S (2019) Deep multimodal representation learning: a survey. IEEE Access 7:63373\u201363394","journal-title":"IEEE Access"},{"key":"2133_CR22","doi-asserted-by":"crossref","unstructured":"Hou L, Agarwal A, Samaras D, Kurc TM, Gupta RR, Saltz JH (2019) Robust histopathology image analysis: to label or to synthesize? In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 8533\u20138542","DOI":"10.1109\/CVPR.2019.00873"},{"key":"2133_CR23","doi-asserted-by":"publisher","first-page":"166","DOI":"10.3389\/fgene.2019.00166","volume":"10","author":"Z Huang","year":"2019","unstructured":"Huang Z, Zhan X, Xiang S, Johnson TS, Helm B, Yu CY, Zhang J, Salama P, Rizkalla M, Han Z et al (2019) Salmon: survival analysis learning with multi-omics neural networks on breast cancer. Front Genet 10:166","journal-title":"Front Genet"},{"key":"2133_CR24","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.1007\/s40747-023-01204-2","volume":"10","author":"L Jia","year":"2024","unstructured":"Jia L, Ren X, Wu W, Zhao J, Qiang Y, Yang Q (2024) DCCAFN: deep convolution cascade attention fusion network based on imaging genomics for prediction survival analysis of lung cancer. Complex Intell Syst 10:1115\u20131130","journal-title":"Complex Intell Syst"},{"key":"2133_CR25","doi-asserted-by":"crossref","unstructured":"Kunhoth S, Maadeed SA, Akbari Y, Saady RA (2024) Computational methods for breast cancer molecular profiling through routine histopathology: a review. arXiv preprint arXiv:2412.10392","DOI":"10.1007\/s11831-025-10374-w"},{"key":"2133_CR26","doi-asserted-by":"publisher","first-page":"4679","DOI":"10.1038\/s41598-020-61588-w","volume":"10","author":"Y-H Lai","year":"2020","unstructured":"Lai Y-H, Chen W-N, Hsu T-C, Lin C, Tsao Y, Wu S (2020) Overall survival prediction of non-small cell lung cancer by integrating microarray and clinical data with deep learning. Sci Rep 10:4679","journal-title":"Sci Rep"},{"key":"2133_CR27","doi-asserted-by":"publisher","first-page":"2587","DOI":"10.1093\/bioinformatics\/btac113","volume":"38","author":"R Li","year":"2022","unstructured":"Li R, Wu X, Li A, Wang M (2022) HFBSurv: hierarchical multimodal fusion with factorized bilinear models for cancer survival prediction. Bioinformatics 38:2587\u20132594","journal-title":"Bioinformatics"},{"key":"2133_CR28","doi-asserted-by":"crossref","unstructured":"Li T, Zhou X, Xue J, Zeng L, Zhu Q, Wang R, Yu H, Xia J (2025) Cross-modal alignment and contrastive learning for enhanced cancer survival prediction. In: Computer methods and programs in biomedicine, p 108633","DOI":"10.1016\/j.cmpb.2025.108633"},{"key":"2133_CR29","doi-asserted-by":"publisher","first-page":"1095","DOI":"10.1016\/j.ccell.2022.09.012","volume":"40","author":"J Lipkova","year":"2022","unstructured":"Lipkova J, Chen RJ, Chen B, Lu MY, Barbieri M, Shao D, Vaidya AJ, Chen C, Zhuang L, Williamson DF et al (2022) Artificial intelligence for multimodal data integration in oncology. Cancer Cell 40:1095\u20131110","journal-title":"Cancer Cell"},{"key":"2133_CR30","doi-asserted-by":"crossref","unstructured":"Liu M, Liu Y, Cui H, Li C, Ma J (2023) MGCT: mutual-guided cross-modality transformer for survival outcome prediction using integrative histopathology-genomic features. In: 2023 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp 1306\u20131312","DOI":"10.1109\/BIBM58861.2023.10385897"},{"key":"2133_CR31","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.irbm.2020.12.002","volume":"43","author":"T Liu","year":"2022","unstructured":"Liu T, Huang J, Liao T, Pu R, Liu S, Peng Y (2022) A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data. IRBM 43:62\u201374","journal-title":"IRBM"},{"key":"2133_CR32","unstructured":"Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: Advances in neural information processing systems, vol 30"},{"key":"2133_CR33","doi-asserted-by":"crossref","unstructured":"Nie D, Zhang H, Adeli E, Liu L, Shen D (2016) 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In: Medical image computing and computer-assisted intervention\u2013MICCAI 2016: 19th international conference, Athens, Greece, October 17\u201321, 2016, Proceedings, Part II 19. Springer, pp 212\u2013220","DOI":"10.1007\/978-3-319-46723-8_25"},{"key":"2133_CR34","doi-asserted-by":"publisher","first-page":"2888","DOI":"10.1093\/bioinformatics\/btaa056","volume":"36","author":"Z Ning","year":"2020","unstructured":"Ning Z, Pan W, Chen Y, Xiao Q, Zhang X, Luo J, Wang J, Zhang Y (2020) Integrative analysis of cross-modal features for the prognosis prediction of clear cell renal cell carcinoma. Bioinformatics 36:2888\u20132895","journal-title":"Bioinformatics"},{"key":"2133_CR35","doi-asserted-by":"publisher","DOI":"10.1002\/2056-4538.70008","volume":"10","author":"N Stathonikos","year":"2024","unstructured":"Stathonikos N, Aubreville M, de Vries S, Wilm F, Bertram CA, Veta M, van Diest PJ (2024) Breast cancer survival prediction using an automated mitosis detection pipeline. J Pathol Clin Res 10:e70008","journal-title":"J Pathol Clin Res"},{"key":"2133_CR36","doi-asserted-by":"crossref","unstructured":"Subramanian V, Syeda-Mahmood T, Do MN (2021) Multimodal fusion using sparse CCA for breast cancer survival prediction. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI). IEEE, pp 1429\u20131432","DOI":"10.1109\/ISBI48211.2021.9434033"},{"key":"2133_CR37","doi-asserted-by":"crossref","unstructured":"Subramanian V, Syeda-Mahmood T, Do MN (2024) Modelling-based joint embedding of histology and genomics using canonical correlation analysis for breast cancer survival prediction. Artif Intell Med 149:102787","DOI":"10.1016\/j.artmed.2024.102787"},{"key":"2133_CR38","doi-asserted-by":"crossref","unstructured":"Tan W, Shen Y (2023) Multimodal learning of noncoding variant effects using genome sequence and chromatin structure. Bioinformatics 39:btad541","DOI":"10.1093\/bioinformatics\/btad541"},{"key":"2133_CR39","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3","volume":"16","author":"R Tibshirani","year":"1997","unstructured":"Tibshirani R (1997) The lasso method for variable selection in the Cox model. Stat Med 16:385\u2013395","journal-title":"Stat Med"},{"key":"2133_CR40","doi-asserted-by":"crossref","unstructured":"Tsai Y-HH, Bai S, Liang PP, Kolter JZ, Morency L-P, Salakhutdinov R (2019) Multimodal transformer for unaligned multimodal language sequences. In: Proceedings of the conference. Association for Computational Linguistics. Meeting, p 6558. NIH Public Access volume 2019","DOI":"10.18653\/v1\/P19-1656"},{"key":"2133_CR41","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, vol 30"},{"key":"2133_CR42","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0233678","volume":"15","author":"E Wulczyn","year":"2020","unstructured":"Wulczyn E, Steiner DF, Xu Z, Sadhwani A, Wang H, Flament-Auvigne I, Mermel CH, Chen P-HC, Liu Y, Stumpe MC (2020) Deep learning-based survival prediction for multiple cancer types using histopathology images. PLoS ONE 15:e0233678","journal-title":"PLoS ONE"},{"key":"2133_CR43","doi-asserted-by":"publisher","first-page":"1515","DOI":"10.1007\/s10586-018-2026-1","volume":"22","author":"K-J Xia","year":"2019","unstructured":"Xia K-J, Yin H-S, Wang J-Q (2019) A novel improved deep convolutional neural network model for medical image fusion. Clust Comput 22:1515\u20131527","journal-title":"Clust Comput"},{"key":"2133_CR44","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107050","volume":"162","author":"J Zhang","year":"2023","unstructured":"Zhang J, He X, Liu Y, Cai Q, Chen H, Qing L (2023) Multi-modal cross-attention network for Alzheimer\u2019s disease diagnosis with multi-modality data. Comput Biol Med 162:107050","journal-title":"Comput Biol Med"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-025-02133-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-025-02133-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-025-02133-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T11:49:05Z","timestamp":1769773745000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-025-02133-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":44,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["2133"],"URL":"https:\/\/doi.org\/10.1007\/s40747-025-02133-y","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"assertion":[{"value":"7 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 October 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 December 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study utilized histological images from pre-existing datasets and did not involve direct human or animal subjects. As such, it did not require formal ethics approval. The raw data for all datasets are publicly available, as you can see in the references.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}}],"article-number":"30"}}