{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T20:39:54Z","timestamp":1775075994819,"version":"3.50.1"},"reference-count":94,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T00:00:00Z","timestamp":1736121600000},"content-version":"vor","delay-in-days":45,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Municipal Government of Quzhou","award":["2023D036"],"award-info":[{"award-number":["2023D036"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LD24F020004"],"award-info":[{"award-number":["LD24F020004"]}]},{"name":"Guangdong Medical Science and Technology Research Foundation","award":["A2023002"],"award-info":[{"award-number":["A2023002"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62131004"],"award-info":[{"award-number":["62131004"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62402207"],"award-info":[{"award-number":["62402207"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"publisher","award":["2022ZD0117700"],"award-info":[{"award-number":["2022ZD0117700"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,11,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The burgeoning accumulation of large-scale biomedical data in oncology, alongside significant strides in deep learning (DL) technologies, has established multimodal DL (MDL) as a cornerstone of precision oncology. This review provides an overview of MDL applications in this field, based on an extensive literature survey. In total, 651 articles published before September 2024 are included. We first outline publicly available multimodal datasets that support cancer research. Then, we discuss key DL training methods, data representation techniques, and fusion strategies for integrating multimodal data. The review also examines MDL applications in tumor segmentation, detection, diagnosis, prognosis, treatment selection, and therapy response monitoring. Finally, we critically assess the limitations of current approaches and propose directions for future research. By synthesizing current progress and identifying challenges, this review aims to guide future efforts in leveraging MDL to advance precision oncology.<\/jats:p>","DOI":"10.1093\/bib\/bbae699","type":"journal-article","created":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T04:36:23Z","timestamp":1736138183000},"source":"Crossref","is-referenced-by-count":57,"title":["Multimodal deep learning approaches for precision oncology: a comprehensive review"],"prefix":"10.1093","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8197-1041","authenticated-orcid":false,"given":"Huan","family":"Yang","sequence":"first","affiliation":[{"name":"Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China , Chengdian Road, Kecheng District, Quzhou 324000, Zhejiang ,","place":["China"]}]},{"given":"Minglei","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Pathology, The First Affiliated Hospital of Zhengzhou University , Jianshe Dong Road, Erqi District, Zhengzhou 450052, Henan ,","place":["China"]}]},{"given":"Jiani","family":"Chen","sequence":"additional","affiliation":[{"name":"Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China , Chengdian Road, Kecheng District, Quzhou 324000, Zhejiang ,","place":["China"]},{"name":"School of Opto-electronic and Communication Engineering, Xiamen University of Technology , Ligong Road, Jimei District, Xiamen 361024, Fujian ,","place":["China"]}]},{"given":"Guocong","family":"Yao","sequence":"additional","affiliation":[{"name":"Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China , Chengdian Road, Kecheng District, Quzhou 324000, Zhejiang ,","place":["China"]},{"name":"School of Computer and Information Engineering, Henan University , Jinming Avenue, Longting District, Kaifeng 475001, Henan ,","place":["China"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6406-1142","authenticated-orcid":false,"given":"Quan","family":"Zou","sequence":"additional","affiliation":[{"name":"Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China , Chengdian Road, Kecheng District, Quzhou 324000, Zhejiang ,","place":["China"]},{"name":"Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China , Section2, North Jianshe Road, Chenghua District, Chengdu 610054, Sichuan ,","place":["China"]}]},{"given":"Linpei","family":"Jia","sequence":"additional","affiliation":[{"name":"Department of Nephrology, Xuanwu Hospital, Capital Medical University , Changchun Street, Xicheng District, Beijing 100053 ,","place":["China"]}]}],"member":"286","published-online":{"date-parts":[[2025,1,5]]},"reference":[{"key":"2025010601283342200_ref1","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA Cancer J Clin"},{"key":"2025010601283342200_ref2","doi-asserted-by":"publisher","first-page":"2006","DOI":"10.1158\/1055-9965.EPI-11-0650","article-title":"Economic burden of cancer in the United States: estimates, projections, and future research","volume":"20","author":"Yabroff","year":"2011","journal-title":"Cancer Epidemiol Biomarkers Prev"},{"key":"2025010601283342200_ref3","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1002\/cac2.12178","article-title":"Advancing to the era of cancer immunotherapy","volume":"41","author":"Wang","year":"2021","journal-title":"Cancer Commun (Lond)"},{"key":"2025010601283342200_ref4","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1200\/JCO.21.01626","article-title":"Targeted therapies for lung cancer patients with oncogenic driver molecular alterations","volume":"40","author":"Tan","year":"2022","journal-title":"J Clin Oncol"},{"key":"2025010601283342200_ref5","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1148\/radiol.212877","article-title":"Deep learning to optimize candidate selection for lung cancer CT screening: advancing the 2021 USPSTF recommendations","volume":"305","author":"Lee","year":"2022","journal-title":"Radiology"},{"key":"2025010601283342200_ref6","doi-asserted-by":"publisher","first-page":"e30157","DOI":"10.2196\/30157","article-title":"COVID-19 mortality prediction from deep learning in a large multistate electronic health record and laboratory information system data set: algorithm development and validation","volume":"23","author":"Sankaranarayanan","year":"2021","journal-title":"J Med Internet Res"},{"key":"2025010601283342200_ref7","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-024-4171-9","article-title":"SBSM-pro: support bio-sequence machine for proteins","volume":"67","author":"Wang","journal-title":"Sci China Inform Sci"},{"key":"2025010601283342200_ref8","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","article-title":"Data-efficient and weakly supervised computational pathology on whole-slide images","volume":"5","author":"Lu","year":"2021","journal-title":"Nat Biomed Eng"},{"key":"2025010601283342200_ref9","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1001\/jamadermatol.2018.4378","article-title":"Expert-level diagnosis of nonpigmented skin cancer by combined convolutional neural networks","volume":"155","author":"Tschandl","year":"2019","journal-title":"JAMA Dermatol"},{"key":"2025010601283342200_ref10","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1016\/S0140-6736(19)32998-8","article-title":"Deep learning for prediction of colorectal cancer outcome: a discovery and validation study","volume":"395","author":"Skrede","year":"2020","journal-title":"Lancet"},{"key":"2025010601283342200_ref11","doi-asserted-by":"publisher","first-page":"101348","DOI":"10.1016\/j.eclinm.2022.101348","article-title":"A CT-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: a multicenter cohort study","volume":"46","author":"Cui","year":"2022","journal-title":"EClinicalMedicine"},{"key":"2025010601283342200_ref12","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1016\/j.neunet.2023.06.013","article-title":"A multi-modal deep neural network for multi-class liver cancer diagnosis","volume":"165","author":"Khan","year":"2023","journal-title":"Neural Netw"},{"key":"2025010601283342200_ref13","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1109\/TMI.2020.3035789","article-title":"Combined spiral transformation and model-driven multi-modal deep learning scheme for automatic prediction of TP53 mutation in pancreatic cancer","volume":"40","author":"Chen","year":"2021","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref14","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","article-title":"Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"41","author":"Chen","year":"2022","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref15","doi-asserted-by":"publisher","first-page":"108485","DOI":"10.1016\/j.compbiomed.2024.108485","article-title":"ISMI-VAE: a deep learning model for classifying disease cells using gene expression and SNV data","volume":"175","author":"Li","year":"2024","journal-title":"Comput Biol Med"},{"key":"2025010601283342200_ref16","doi-asserted-by":"publisher","first-page":"bbac448","DOI":"10.1093\/bib\/bbac448","article-title":"ICSDA: a multi-modal deep learning model to predict breast cancer recurrence and metastasis risk by integrating pathological, clinical and gene expression data","volume":"23","author":"Yao","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025010601283342200_ref17","doi-asserted-by":"publisher","first-page":"847","DOI":"10.2967\/jnumed.121.262398","article-title":"Head-to-head comparison of (68)Ga-PSMA-11 PET\/CT and mpMRI with a histopathology gold standard in the detection, intraprostatic localization, and determination of local extension of primary prostate cancer: results from a prospective single-center imaging trial","volume":"63","author":"Sonni","year":"2022","journal-title":"J Nucl Med"},{"key":"2025010601283342200_ref18","doi-asserted-by":"publisher","first-page":"e230806","DOI":"10.1148\/radiol.230806","article-title":"Multimodal deep learning for integrating chest radiographs and clinical parameters: a case for transformers","volume":"309","author":"Khader","year":"2023","journal-title":"Radiology"},{"key":"2025010601283342200_ref19","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1186\/s13073-021-00968-x","article-title":"Deep learning in cancer diagnosis, prognosis and treatment selection","volume":"13","author":"Tran","year":"2021","journal-title":"Genome Med"},{"key":"2025010601283342200_ref20","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1038\/s41568-020-00327-9","article-title":"Designing deep learning studies in cancer diagnostics","volume":"21","author":"Kleppe","year":"2021","journal-title":"Nat Rev Cancer"},{"key":"2025010601283342200_ref21","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1002\/cac2.12012","article-title":"Emerging role of deep learning-based artificial intelligence in tumor pathology","volume":"40","author":"Jiang","year":"2020","journal-title":"Cancer Commun (Lond)"},{"key":"2025010601283342200_ref22","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1186\/s13073-024-01315-6","article-title":"Deep learning in cancer genomics and histopathology","volume":"16","author":"Unger","year":"2024","journal-title":"Genome Med"},{"key":"2025010601283342200_ref23","doi-asserted-by":"publisher","first-page":"bbad313","DOI":"10.1093\/bib\/bbad313","article-title":"Multimodal deep learning approaches for single-cell multi-omics data integration","volume":"24","author":"Athaya","year":"2023","journal-title":"Brief Bioinform"},{"key":"2025010601283342200_ref24","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1038\/s41698-024-00575-0","article-title":"Artificial intelligence in neuro-oncology: advances and challenges in brain tumor diagnosis, prognosis, and precision treatment","volume":"8","author":"Khalighi","year":"2024","journal-title":"NPJ Precis Oncol"},{"key":"2025010601283342200_ref25","doi-asserted-by":"publisher","first-page":"bbab569","DOI":"10.1093\/bib\/bbab569","article-title":"Multimodal deep learning for biomedical data fusion: a review","volume":"23","author":"Stahlschmidt","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025010601283342200_ref26","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1038\/s42256-023-00633-5","article-title":"Multimodal data fusion for cancer biomarker discovery with deep learning","volume":"5","author":"Steyaert","year":"2023","journal-title":"Nat Mach Intell"},{"key":"2025010601283342200_ref27","doi-asserted-by":"publisher","first-page":"102176","DOI":"10.1016\/j.compmedimag.2022.102176","article-title":"Hierarchical multimodal fusion framework based on noisy label learning and attention mechanism for cancer classification with pathology and genomic features","volume":"104","author":"Qiu","year":"2023","journal-title":"Comput Med Imaging Graph"},{"key":"2025010601283342200_ref28","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1186\/s12859-021-04376-1","article-title":"Non-small-cell lung cancer classification via RNA-Seq and histology imaging probability fusion","volume":"22","author":"Carrillo-Perez","year":"2021","journal-title":"BMC Bioinformatics"},{"key":"2025010601283342200_ref29","doi-asserted-by":"publisher","first-page":"2092","DOI":"10.1038\/s41591-024-03126-z","article-title":"Prediction of recurrence risk in endometrial cancer with multimodal deep learning","volume":"30","author":"Volinsky-Fremond","year":"2024","journal-title":"Nat Med"},{"key":"2025010601283342200_ref30","doi-asserted-by":"publisher","first-page":"5108","DOI":"10.1093\/bioinformatics\/btac641","article-title":"Predicting colorectal cancer tumor mutational burden from histopathological images and clinical information using multi-modal deep learning","volume":"38","author":"Huang","year":"2022","journal-title":"Bioinformatics"},{"key":"2025010601283342200_ref31","doi-asserted-by":"publisher","first-page":"0","DOI":"10.3390\/bioengineering11010013","article-title":"Multi-modal ensemble deep learning in head and neck cancer HPV sub-typing","volume":"11","author":"Saikia","year":"2023","journal-title":"Bioengineering (Basel)"},{"key":"2025010601283342200_ref32","doi-asserted-by":"publisher","first-page":"106948","DOI":"10.1016\/j.cmpb.2022.106948","article-title":"Deep learning based time-to-event analysis with PET, CT and joint PET\/CT for head and neck cancer prognosis","volume":"222","author":"Wang","year":"2022","journal-title":"Comput Methods Programs Biomed"},{"key":"2025010601283342200_ref33","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1038\/s41586-020-2140-0","article-title":"Integrating genomic features for non-invasive early lung cancer detection","volume":"580","author":"Chabon","year":"2020","journal-title":"Nature"},{"key":"2025010601283342200_ref34","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1016\/j.chest.2023.04.033","article-title":"Cell-free DNA fragmentomes in the diagnostic evaluation of patients with symptoms suggestive of lung cancer","volume":"164","author":"Leal","year":"2023","journal-title":"Chest"},{"key":"2025010601283342200_ref35","doi-asserted-by":"publisher","first-page":"3099","DOI":"10.1093\/bioinformatics\/btab236","article-title":"Early cancer detection from genome-wide cell-free DNA fragmentation via shuffled frog leaping algorithm and support vector machine","volume":"37","author":"Liu","year":"2021","journal-title":"Bioinformatics"},{"key":"2025010601283342200_ref36","doi-asserted-by":"publisher","first-page":"180161","DOI":"10.1038\/sdata.2018.161","article-title":"The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions","volume":"5","author":"Tschandl","year":"2018","journal-title":"Scientific Data"},{"key":"2025010601283342200_ref37","doi-asserted-by":"publisher","first-page":"643","DOI":"10.2967\/jnumed.123.267048","article-title":"Deep semisupervised transfer learning for fully automated whole-body tumor quantification and prognosis of cancer on PET\/CT","volume":"65","author":"Leung","year":"2024","journal-title":"J Nucl Med"},{"key":"2025010601283342200_ref38","doi-asserted-by":"publisher","first-page":"106775","DOI":"10.1016\/j.knosys.2021.106775","article-title":"A survey on federated learning","volume":"216","author":"Zhang","year":"2021","journal-title":"Knowl-Based Syst"},{"key":"2025010601283342200_ref39","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1186\/s13045-022-01225-3","article-title":"Development of a novel combined nomogram model integrating deep learning-pathomics, radiomics and immunoscore to predict postoperative outcome of colorectal cancer lung metastasis patients","volume":"15","author":"Wang","year":"2022","journal-title":"J Hematol Oncol"},{"key":"2025010601283342200_ref40","doi-asserted-by":"publisher","first-page":"e232057","DOI":"10.1148\/radiol.232057","article-title":"Predicting invasiveness of lung adenocarcinoma at chest CT with deep learning ternary classification models","volume":"311","author":"Pan","year":"2024","journal-title":"Radiology"},{"key":"2025010601283342200_ref41","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1016\/S0004-3702(96)00034-3","article-title":"Solving the multiple instance problem with axis-parallel rectangles","volume":"89","author":"Dietterich","year":"1997","journal-title":"Artif Intell"},{"key":"2025010601283342200_ref42","doi-asserted-by":"publisher","first-page":"1346","DOI":"10.1038\/s41551-022-00914-1","article-title":"Self-supervised learning in medicine and healthcare","volume":"6","author":"Krishnan","year":"2022","journal-title":"Nat Biomed Eng"},{"key":"2025010601283342200_ref43","doi-asserted-by":"publisher","first-page":"eabh1275","DOI":"10.1126\/sciadv.abh1275","article-title":"Predicting and characterizing a cancer dependency map of tumors with deep learning","volume":"7","author":"Chiu","year":"2021","journal-title":"Sci Adv"},{"key":"2025010601283342200_ref44","doi-asserted-by":"publisher","first-page":"2780","DOI":"10.1007\/s00259-019-04468-4","article-title":"PET image denoising using unsupervised deep learning","volume":"46","author":"Cui","year":"2019","journal-title":"Eur J Nucl Med Mol Imaging"},{"key":"2025010601283342200_ref45","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1002\/cjp2.302","article-title":"Optimization of deep learning models for the prediction of gene mutations using unsupervised clustering","volume":"9","author":"Chen","year":"2023","journal-title":"J Pathol Clin Res"},{"key":"2025010601283342200_ref46","volume-title":"Canonical Correlation Analysis: Uses and Interpretation","year":"1984"},{"key":"2025010601283342200_ref47","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1109\/TCYB.2015.2392052","article-title":"Spectral multimodal hashing and its application to multimedia retrieval","volume":"46","author":"Zhen","year":"2016","journal-title":"IEEE Trans Cybern"},{"key":"2025010601283342200_ref48","doi-asserted-by":"publisher","first-page":"i446","DOI":"10.1093\/bioinformatics\/btz342","article-title":"Deep learning with multimodal representation for pancancer prognosis prediction","volume":"35","author":"Cheerla","year":"2019","journal-title":"Bioinformatics"},{"key":"2025010601283342200_ref49","doi-asserted-by":"publisher","first-page":"3859","DOI":"10.1109\/TIP.2020.2963950","article-title":"Open-ended video question answering via multi-modal conditional adversarial networks","volume":"29","author":"Zhao","year":"2020","journal-title":"IEEE Trans Image Process"},{"key":"2025010601283342200_ref50","doi-asserted-by":"publisher","first-page":"e10480","DOI":"10.1002\/btm2.10480","article-title":"Deep learning-based multimodal fusion network for segmentation and classification of breast cancers using B-mode and elastography ultrasound images","volume":"8","author":"Misra","year":"2023","journal-title":"Bioeng Transl Med"},{"key":"2025010601283342200_ref51","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1038\/s41587-022-01520-x","article-title":"Discovery of drug-omics associations in type 2 diabetes with generative deep-learning models","volume":"41","author":"Alles\u00f8e","year":"2023","journal-title":"Nat Biotechnol"},{"key":"2025010601283342200_ref52","doi-asserted-by":"publisher","first-page":"7779","DOI":"10.1002\/mp.16566","article-title":"Deep learning-based combination of [18F]-FDG PET and CT images for producing pulmonary perfusion image","volume":"50","author":"Gu","year":"2023","journal-title":"Med Phys"},{"key":"2025010601283342200_ref53","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s00330-022-09031-8","article-title":"Multimodal deep learning model on interim [(18)F]FDG PET\/CT for predicting primary treatment failure in diffuse large B-cell lymphoma","volume":"33","author":"Yuan","year":"2023","journal-title":"Eur Radiol"},{"key":"2025010601283342200_ref54","doi-asserted-by":"publisher","first-page":"1216","DOI":"10.1109\/JBHI.2022.3233387","article-title":"A multi-modal heterogeneous graph forest to predict lymph node metastasis of non-small cell lung cancer","volume":"27","author":"Hu","year":"2023","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025010601283342200_ref55","doi-asserted-by":"publisher","first-page":"1215","DOI":"10.1007\/s11548-020-02172-5","article-title":"Improving detection of prostate cancer foci via information fusion of MRI and temporal enhanced ultrasound","volume":"15","author":"Sedghi","year":"2020","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2025010601283342200_ref56","doi-asserted-by":"publisher","first-page":"102612","DOI":"10.1016\/j.media.2022.102612","article-title":"Cross-modal attention for multi-modal image registration","volume":"82","author":"Song","year":"2022","journal-title":"Med Image Anal"},{"key":"2025010601283342200_ref57","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1021\/jasms.2c00254","article-title":"Deep learning on multimodal chemical and whole slide imaging data for predicting prostate cancer directly from tissue images","volume":"34","author":"Haque","year":"2023","journal-title":"J Am Soc Mass Spectrom"},{"key":"2025010601283342200_ref58","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1016\/j.media.2017.10.002","article-title":"A deep learning model integrating FCNNs and CRFs for brain tumor segmentation","volume":"43","author":"Zhao","year":"2018","journal-title":"Med Image Anal"},{"key":"2025010601283342200_ref59","doi-asserted-by":"publisher","first-page":"1103","DOI":"10.1038\/s41592-024-02233-6","article-title":"The multimodality cell segmentation challenge: toward universal solutions","volume":"21","author":"Ma","year":"2024","journal-title":"Nat Methods"},{"key":"2025010601283342200_ref60","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering11010013","article-title":"MEDIAR: harmony of data-centric and model-centric for multi-modality microscopy","author":"Lee","year":"2022","journal-title":"arXiv preprint arXiv:221203465"},{"key":"2025010601283342200_ref61","doi-asserted-by":"publisher","first-page":"104812","DOI":"10.1016\/j.compbiomed.2021.104812","article-title":"Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection","volume":"137","author":"Wang","year":"2021","journal-title":"Comput Biol Med"},{"key":"2025010601283342200_ref62","doi-asserted-by":"publisher","first-page":"110183","DOI":"10.1016\/j.isci.2024.110183","article-title":"AutoCancer as an automated multimodal framework for early cancer detection","volume":"27","author":"Liu","year":"2024","journal-title":"iScience"},{"key":"2025010601283342200_ref63","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1186\/s13045-021-01167-2","article-title":"Deep learning for differential diagnosis of malignant hepatic tumors based on multi-phase contrast-enhanced CT and clinical data","volume":"14","author":"Gao","year":"2021","journal-title":"J Hematol Oncol"},{"key":"2025010601283342200_ref64","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1097\/RLU.0000000000003661","article-title":"Performance evaluation of a deep learning system for differential diagnosis of lung cancer with conventional CT and FDG PET\/CT using transfer learning and metadata","volume":"46","author":"Park","year":"2021","journal-title":"Clin Nucl Med"},{"key":"2025010601283342200_ref65","doi-asserted-by":"publisher","first-page":"108039","DOI":"10.1016\/j.cmpb.2024.108039","article-title":"Sonography-based multimodal information platform for identifying the surgical pathology of ductal carcinoma in situ","volume":"245","author":"Wu","year":"2024","journal-title":"Comput Methods Programs Biomed"},{"key":"2025010601283342200_ref66","doi-asserted-by":"publisher","first-page":"109403","DOI":"10.1016\/j.isci.2024.109403","article-title":"Multi-modality deep learning model reaches high prediction accuracy in the diagnosis of ovarian cancer","volume":"27","author":"Wang","year":"2024","journal-title":"iScience"},{"key":"2025010601283342200_ref67","doi-asserted-by":"publisher","first-page":"2681","DOI":"10.1038\/s41467-024-46700-2","article-title":"Development and validation of an interpretable model integrating multimodal information for improving ovarian cancer diagnosis","volume":"15","author":"Xiang","year":"2024","journal-title":"Nat Commun"},{"key":"2025010601283342200_ref68","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/s10120-022-01358-x","article-title":"A deep-learning based system using multi-modal data for diagnosing gastric neoplasms in real-time (with video)","volume":"26","author":"Du","year":"2023","journal-title":"Gastric Cancer"},{"key":"2025010601283342200_ref69","doi-asserted-by":"publisher","first-page":"12623","DOI":"10.1109\/TCYB.2021.3069920","article-title":"Interpretability-based multimodal convolutional neural networks for skin lesion diagnosis","volume":"52","author":"Wang","year":"2022","journal-title":"IEEE Trans Cybern"},{"key":"2025010601283342200_ref70","doi-asserted-by":"publisher","first-page":"7513","DOI":"10.1038\/s41467-023-42811-4","article-title":"PET\/CT based cross-modal deep learning signature to predict occult nodal metastasis in lung cancer","volume":"14","author":"Zhong","year":"2023","journal-title":"Nat Commun"},{"key":"2025010601283342200_ref71","doi-asserted-by":"publisher","first-page":"106715","DOI":"10.1016\/j.compbiomed.2023.106715","article-title":"DeAF: a multimodal deep learning framework for disease prediction","volume":"156","author":"Li","year":"2023","journal-title":"Comput Biol Med"},{"key":"2025010601283342200_ref72","doi-asserted-by":"publisher","first-page":"bbac432","DOI":"10.1093\/bib\/bbac432","article-title":"Deep learning radiomics under multimodality explore association between muscle\/fat and metastasis and survival in breast cancer patients","volume":"23","author":"Miao","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025010601283342200_ref73","doi-asserted-by":"publisher","first-page":"106576","DOI":"10.1016\/j.compbiomed.2023.106576","article-title":"Deep multimodal graph-based network for survival prediction from highly multiplexed images and patient variables","volume":"154","author":"Fu","year":"2023","journal-title":"Comput Biol Med"},{"key":"2025010601283342200_ref74","doi-asserted-by":"publisher","first-page":"881","DOI":"10.1016\/j.clnu.2024.02.005","article-title":"Predicting malnutrition in gastric cancer patients using computed tomography(CT) deep learning features and clinical data","volume":"43","author":"Huang","year":"2024","journal-title":"Clin Nutr"},{"key":"2025010601283342200_ref75","doi-asserted-by":"publisher","first-page":"108104","DOI":"10.1016\/j.cmpb.2024.108104","article-title":"A comparison between centralized and asynchronous federated learning approaches for survival outcome prediction using clinical and PET data from non-small cell lung cancer patients","volume":"248","author":"Vo","year":"2024","journal-title":"Comput Methods Programs Biomed"},{"key":"2025010601283342200_ref76","doi-asserted-by":"publisher","first-page":"2678","DOI":"10.1109\/TMI.2023.3263010","article-title":"Survival prediction via hierarchical multimodal co-attention transformer: a computational histology-radiology solution","volume":"42","author":"Li","year":"2023","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref77","doi-asserted-by":"publisher","first-page":"1845","DOI":"10.1007\/s11548-022-02698-w","article-title":"Development and validation of a meta-learning-based multi-modal deep learning algorithm for detection of peritoneal metastasis","volume":"17","author":"Zhang","year":"2022","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2025010601283342200_ref78","doi-asserted-by":"publisher","first-page":"104127","DOI":"10.1016\/j.ebiom.2022.104127","article-title":"Prediction of lung malignancy progression and survival with machine learning based on pre-treatment FDG-PET\/CT","volume":"82","author":"Huang","year":"2022","journal-title":"EBioMedicine"},{"key":"2025010601283342200_ref79","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1186\/s12916-023-02898-4","article-title":"Multimodal-based machine learning strategy for accurate and non-invasive prediction of intramedullary glioma grade and mutation status of molecular markers: a retrospective study","volume":"21","author":"Ma","year":"2023","journal-title":"BMC Med"},{"key":"2025010601283342200_ref80","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1038\/s41746-022-00613-w","article-title":"Prostate cancer therapy personalization via multi-modal deep learning on randomized phase III clinical trials","volume":"5","author":"Esteva","year":"2022","journal-title":"NPJ Digit Med"},{"key":"2025010601283342200_ref81","doi-asserted-by":"publisher","first-page":"107054","DOI":"10.1016\/j.compbiomed.2023.107054","article-title":"Multimodality MRI synchronous construction based deep learning framework for MRI-guided radiotherapy synthetic CT generation","volume":"162","author":"Zhou","year":"2023","journal-title":"Comput Biol Med"},{"key":"2025010601283342200_ref82","doi-asserted-by":"publisher","first-page":"18800","DOI":"10.1038\/s41598-021-98408-8","article-title":"Multimodal deep learning models for the prediction of pathologic response to neoadjuvant chemotherapy in breast cancer","volume":"11","author":"Joo","year":"2021","journal-title":"Sci Rep"},{"key":"2025010601283342200_ref83","doi-asserted-by":"publisher","first-page":"102271","DOI":"10.1016\/j.eclinm.2023.102271","article-title":"Deep radiomics-based fusion model for prediction of bevacizumab treatment response and outcome in patients with colorectal cancer liver metastases: a multicentre cohort study","volume":"65","author":"Zhou","year":"2023","journal-title":"EClinicalMedicine"},{"key":"2025010601283342200_ref84","doi-asserted-by":"publisher","first-page":"e187","DOI":"10.1093\/oncolo\/oyad227","article-title":"Deep learning of multimodal ultrasound: stratifying the response to neoadjuvant chemotherapy in breast cancer before treatment","volume":"29","author":"Gu","year":"2024","journal-title":"Oncologist"},{"key":"2025010601283342200_ref85","doi-asserted-by":"publisher","first-page":"3848","DOI":"10.3390\/cancers14163848","article-title":"Multimodal prediction of five-year breast cancer recurrence in women who receive neoadjuvant chemotherapy","volume":"14","author":"Rabinovici-Cohen","year":"2022","journal-title":"Cancers (Basel)"},{"key":"2025010601283342200_ref86","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1038\/s41591-023-02715-8","article-title":"A federated learning system for precision oncology in Europe: DigiONE","volume":"30","author":"Mahon","year":"2024","journal-title":"Nat Med"},{"key":"2025010601283342200_ref87","doi-asserted-by":"publisher","first-page":"3566","DOI":"10.1109\/TMI.2023.3295489","article-title":"Disentangle first, then distill: a unified framework for missing modality imputation and Alzheimer\u2019s disease diagnosis","volume":"42","author":"Chen","year":"2023","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref88","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1109\/TMI.2020.3036584","article-title":"Disentangle, align and fuse for multimodal and semi-supervised image segmentation","volume":"40","author":"Chartsias","year":"2021","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref89","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1186\/s13059-023-02989-8","article-title":"CMOT: cross-modality optimal transport for multimodal inference","volume":"24","author":"Alatkar","year":"2023","journal-title":"Genome Biol"},{"key":"2025010601283342200_ref90","doi-asserted-by":"publisher","first-page":"6112","DOI":"10.1109\/JBHI.2023.3315073","article-title":"Fusion-based deep learning architecture for detecting drug-target binding affinity using target and drug sequence and structure","volume":"27","author":"Wang","year":"2023","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025010601283342200_ref91","doi-asserted-by":"publisher","first-page":"10461","DOI":"10.1109\/TNNLS.2023.3242049","article-title":"Residual sketch learning for a feature-importance-based and linguistically interpretable ensemble classifier","volume":"35","author":"Bian","year":"2024","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"2025010601283342200_ref92","doi-asserted-by":"publisher","volume-title":"Int J Comput Vis","author":"Selvaraju","DOI":"10.1007\/s11263-019-01228-7"},{"key":"2025010601283342200_ref93","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1109\/TMI.2022.3215017","article-title":"Graph node based interpretability guided sample selection for active learning","volume":"42","author":"Mahapatra","year":"2023","journal-title":"IEEE Trans Med Imaging"},{"key":"2025010601283342200_ref94","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1906.10263","article-title":"DLIME: a deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems","author":"Zafar","year":"2019.","journal-title":"arXiv preprint arXiv:190610263"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbae699\/61330897\/bbae699.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbae699\/61330897\/bbae699.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,6]],"date-time":"2025-01-06T04:36:35Z","timestamp":1736138195000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae699\/7942793"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,22]]},"references-count":94,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,11,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae699","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,11,22]]},"article-number":"bbae699"}}