{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T11:45:05Z","timestamp":1781264705611,"version":"3.54.1"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2025,3,21]],"date-time":"2025-03-21T00:00:00Z","timestamp":1742515200000},"content-version":"vor","delay-in-days":20,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100021171","name":"Guangdong Basic and Applied Basic Research Foundation","doi-asserted-by":"publisher","award":["2024A1515011769"],"award-info":[{"award-number":["2024A1515011769"]}],"id":[{"id":"10.13039\/501100021171","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["82370106"],"award-info":[{"award-number":["82370106"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key Research and Development Project of China","award":["2022YFA0806303"],"award-info":[{"award-number":["2022YFA0806303"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Accurate cancer prognosis is essential for personalized clinical management, guiding treatment strategies and predicting patient survival. Conventional methods, which depend on the subjective evaluation of histopathological features, exhibit significant inter-observer variability and limited predictive power. To overcome these limitations, we developed cross-attention transformer-based multimodal fusion network (CATfusion), a deep learning framework that integrates multimodal histology-genomic data for comprehensive cancer survival prediction. By employing self-supervised learning strategy with TabAE for feature extraction and utilizing cross-attention mechanisms to fuse diverse data types, including mRNA-seq, miRNA-seq, copy number variation, DNA methylation variation, mutation data, and histopathological images. By successfully integrating this multi-tiered patient information, CATfusion has become an advanced survival prediction model to utilize the most diverse data types across various cancer types. CATfusion\u2019s architecture, which includes a bidirectional multimodal attention mechanism and self-attention block, is adept at synchronizing the learning and integration of representations from various modalities. CATfusion achieves superior predictive performance over traditional and unimodal models, as demonstrated by enhanced C-index and survival area under the curve scores. The model\u2019s high accuracy in stratifying patients into distinct risk groups is a boon for personalized medicine, enabling tailored treatment plans. Moreover, CATfusion\u2019s interpretability, enabled by attention-based visualization, offers insights into the biological underpinnings of cancer prognosis, underscoring its potential as a transformative tool in oncology.<\/jats:p>","DOI":"10.1093\/bib\/bbaf121","type":"journal-article","created":{"date-parts":[[2025,3,23]],"date-time":"2025-03-23T08:52:24Z","timestamp":1742719944000},"source":"Crossref","is-referenced-by-count":20,"title":["Deep learning-driven survival prediction in pan-cancer studies by integrating multimodal histology-genomic data"],"prefix":"10.1093","volume":"26","author":[{"given":"Yongfei","family":"Hu","sequence":"first","affiliation":[{"name":"Dermatology Hospital, Southern Medical University , No. 2, Lujing Road, Yuexiu District, Guangzhou 510091 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyu","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Bioinformatics, School of Basic Medical Sciences, Guangdong Province Key Laboratory of Molecular Tumor Pathology, Southern Medical University , 1023 Shatai South Road, Baiyun District, Guangzhou 510515 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Yi","sequence":"additional","affiliation":[{"name":"Dermatology Hospital, Southern Medical University , No. 2, Lujing Road, Yuexiu District, Guangzhou 510091 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yan","family":"Huang","sequence":"additional","affiliation":[{"name":"Cancer Research Institute, School of Basic Medical Sciences, Southern Medical University , 1023 Shatai South Road, Baiyun District, Guangzhou 510515 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangyu","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Gastrointestinal Medical Oncology, Harbin Medical University Cancer Hospital , No. 150 Haping Road, Nangang District, Harbin 150000 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6860-6864","authenticated-orcid":false,"given":"Dong","family":"Wang","sequence":"additional","affiliation":[{"name":"Dermatology Hospital, Southern Medical University , No. 2, Lujing Road, Yuexiu District, Guangzhou 510091 ,","place":["China"]},{"name":"Department of Bioinformatics, School of Basic Medical Sciences, Guangdong Province Key Laboratory of Molecular Tumor Pathology, Southern Medical University , 1023 Shatai South Road, Baiyun District, Guangzhou 510515 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,3,21]]},"reference":[{"key":"2025032119061271100_ref1","doi-asserted-by":"publisher","first-page":"229","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","year":"2024","journal-title":"CA Cancer J Clin"},{"key":"2025032119061271100_ref2","doi-asserted-by":"publisher","first-page":"93","DOI":"10.3322\/caac.21388","article-title":"The eighth edition AJCC cancer staging manual: continuing to build a bridge from a population-based to a more \u201cpersonalized\u201d approach to cancer staging","volume":"67","author":"Amin","year":"2017","journal-title":"CA Cancer J Clin"},{"key":"2025032119061271100_ref3","doi-asserted-by":"publisher","first-page":"1755","DOI":"10.1016\/j.cell.2023.02.015","article-title":"Accelerating the understanding of cancer biology through the lens of genomics","volume":"186","author":"Wang","year":"2023","journal-title":"Cell"},{"key":"2025032119061271100_ref4","doi-asserted-by":"publisher","first-page":"3068","DOI":"10.1002\/cncr.32164","article-title":"Experiences with cancer survey in China","volume":"125","author":"Yao","year":"2019","journal-title":"Cancer"},{"key":"2025032119061271100_ref5","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1007\/s00428-021-03141-2","article-title":"Grading of invasive breast carcinoma: the way forward","volume":"480","author":"Dooijeweert","year":"2022","journal-title":"Virchows Arch"},{"key":"2025032119061271100_ref6","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1097\/PAP.0b013e318202645a","article-title":"Histologic grading and prognostic biomarkers in salivary gland carcinomas","volume":"18","author":"Seethala","year":"2011","journal-title":"Adv Anat Pathol"},{"key":"2025032119061271100_ref7","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.pbiomolbio.2022.07.004","article-title":"A systematic review on machine learning and deep learning techniques in cancer survival prediction","volume":"174","author":"Deepa","year":"2022","journal-title":"Prog Biophys Mol Biol"},{"key":"2025032119061271100_ref8","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1038\/nrc3261","article-title":"Intra-tumour heterogeneity: a looking glass for cancer?","volume":"12","author":"Marusyk","year":"2012","journal-title":"Nat Rev Cancer"},{"key":"2025032119061271100_ref9","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.1109\/TIP.2021.3139229","article-title":"Big-hypergraph factorization neural network for survival prediction from whole slide image","volume":"31","author":"Di","year":"2022","journal-title":"IEEE Trans Image Process"},{"key":"2025032119061271100_ref10","doi-asserted-by":"publisher","first-page":"101789","DOI":"10.1016\/j.media.2020.101789","article-title":"Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks","volume":"65","author":"Yao","year":"2020","journal-title":"Med Image Anal"},{"key":"2025032119061271100_ref11","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.1007\/s10278-024-01049-2","article-title":"From pixels to prognosis: a survey on AI-driven cancer patient survival prediction using digital histology images","volume":"37","author":"Parvaiz","year":"2024","journal-title":"J Imaging Inform Med"},{"key":"2025032119061271100_ref12","doi-asserted-by":"publisher","first-page":"2000","DOI":"10.1002\/hep.31207","article-title":"Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides","volume":"72","author":"Saillard","year":"2020","journal-title":"Hepatology"},{"key":"2025032119061271100_ref13","doi-asserted-by":"publisher","first-page":"103252","DOI":"10.1016\/j.media.2024.103252","article-title":"Dual-stream multi-dependency graph neural network enables precise cancer survival analysis","volume":"97","author":"Wang","year":"2024","journal-title":"Med Image Anal"},{"key":"2025032119061271100_ref14","doi-asserted-by":"publisher","first-page":"1177932219899051","DOI":"10.1177\/1177932219899051","article-title":"Multi-omics data integration, interpretation, and its application","volume":"14","author":"Subramanian","year":"2020","journal-title":"Bioinform Biol Insights"},{"key":"2025032119061271100_ref15","doi-asserted-by":"publisher","first-page":"3739","DOI":"10.1109\/TMI.2021.3097319","article-title":"Weakly supervised deep ordinal Cox model for survival prediction from whole-slide pathological images","volume":"40","author":"Shao","year":"2021","journal-title":"IEEE Trans Med Imaging"},{"key":"2025032119061271100_ref16","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1109\/TCBB.2021.3083566","article-title":"Predicting the survival of cancer patients with multimodal graph neural network","volume":"19","author":"Gao","year":"2022","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2025032119061271100_ref17","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1109\/TCBB.2018.2806438","article-title":"A multimodal deep neural network for human breast cancer prognosis prediction by integrating multi-dimensional data","volume":"16","author":"Sun","year":"2018","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2025032119061271100_ref18","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1186\/s13073-021-00968-x","article-title":"Deep learning in cancer diagnosis, prognosis and treatment selection","volume":"13","author":"Tran","year":"2021","journal-title":"Genome Med"},{"key":"2025032119061271100_ref19","doi-asserted-by":"publisher","first-page":"1248","DOI":"10.1158\/1078-0432.CCR-17-0853","article-title":"Deep learning-based multi-omics integration robustly predicts survival in liver cancer","volume":"24","author":"Chaudhary","year":"2018","journal-title":"Clin Cancer Res"},{"key":"2025032119061271100_ref20","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab569","article-title":"Multimodal deep learning for biomedical data fusion: A review","volume":"23","author":"Stahlschmidt","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025032119061271100_ref21","doi-asserted-by":"publisher","first-page":"690","DOI":"10.1111\/cas.15592","article-title":"Survival prediction of stomach cancer using expression data and deep learning models with histopathological images","volume":"114","author":"Wei","year":"2023","journal-title":"Cancer Sci"},{"key":"2025032119061271100_ref22","doi-asserted-by":"publisher","first-page":"104482","DOI":"10.1016\/j.jbi.2023.104482","article-title":"A scoping review on multimodal deep learning in biomedical images and texts","volume":"146","author":"Sun","year":"2023","journal-title":"J Biomed Inform"},{"key":"2025032119061271100_ref23","doi-asserted-by":"publisher","first-page":"102424","DOI":"10.1016\/j.artmed.2022.102424","article-title":"SurvivalCNN: a deep learning-based method for gastric cancer survival prediction using radiological imaging data and clinicopathological variables","volume":"134","author":"Hao","year":"2022","journal-title":"Artif Intell Med"},{"key":"2025032119061271100_ref24","doi-asserted-by":"publisher","first-page":"2587","DOI":"10.1093\/bioinformatics\/btac113","article-title":"HFBSurv: hierarchical multimodal fusion with factorized bilinear models for cancer survival prediction","volume":"38","author":"Li","year":"2022","journal-title":"Bioinformatics"},{"key":"2025032119061271100_ref25","doi-asserted-by":"publisher","first-page":"1762","DOI":"10.1007\/s10620-022-07782-4","article-title":"A gallbladder cancer survival prediction model based on multimodal fusion analysis","volume":"68","author":"Yin","year":"2023","journal-title":"Dig Dis Sci"},{"key":"2025032119061271100_ref26","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","article-title":"Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"41","author":"Chen","year":"2022","journal-title":"IEEE Trans Med Imaging"},{"key":"2025032119061271100_ref27","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btad025","article-title":"CAMR: cross-aligned multimodal representation learning for cancer survival prediction","volume":"39","author":"Wu","year":"2023  13","journal-title":"Bioinformatics"},{"key":"2025032119061271100_ref28","doi-asserted-by":"publisher","first-page":"2963","DOI":"10.1093\/bioinformatics\/btab185","article-title":"GPDBN: deep bilinear network integrating both genomic data and pathological images for breast cancer prognosis prediction","volume":"37","author":"Wang","year":"2021","journal-title":"Bioinformatics"},{"key":"2025032119061271100_ref29","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1016\/j.cell.2018.03.059","article-title":"SnapShot: TCGA-analyzed Tumors","volume":"173","author":"Blum","year":"2018","journal-title":"Cell"},{"key":"2025032119061271100_ref30","doi-asserted-by":"publisher","first-page":"1420","DOI":"10.1038\/s41551-022-00929-8","article-title":"Fast and scalable search of whole-slide images via self-supervised deep learning","volume":"6","author":"Chen","year":"2022","journal-title":"Nat Biomed Eng"},{"key":"2025032119061271100_ref31","doi-asserted-by":"publisher","first-page":"1430","DOI":"10.1038\/s41592-024-02353-z","article-title":"Transformers in single-cell omics: a review and new perspectives","volume":"21","author":"Szalata","year":"2024","journal-title":"Nat Methods"},{"key":"2025032119061271100_ref32","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1038\/s41586-023-06139-9","article-title":"Transfer learning enables predictions in network biology","volume":"618","author":"Theodoris","year":"2023","journal-title":"Nature"},{"key":"2025032119061271100_ref33","doi-asserted-by":"publisher","first-page":"11399","DOI":"10.1038\/s41598-019-47765-6","article-title":"DeepInsight: a methodology to transform a non-image data to an image for convolution neural network architecture","volume":"9","author":"Sharma","year":"2019","journal-title":"Sci Rep"},{"key":"2025032119061271100_ref34","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1038\/s41586-024-07441-w","article-title":"A whole-slide foundation model for digital pathology from real-world data","volume":"630","author":"Xu","year":"2024","journal-title":"Nature"},{"key":"2025032119061271100_ref35","first-page":"2406.05074","article-title":"Hibou: a family of foundational vision transformers for pathology","volume":"5","author":"Nechaev","year":"2024","journal-title":"arXiv"},{"key":"2025032119061271100_ref36","first-page":"2404.15217","article-title":"Towards large-scale training of pathology foundation models","volume":"2","author":"Ai","year":"2024","journal-title":"arXiv"},{"key":"2025032119061271100_ref37","article-title":"Phikon-v2, a large and public feature extractor for biomarker prediction","volume":"3","author":"Filiot","year":"2024","journal-title":"arXiv"},{"key":"2025032119061271100_ref38","first-page":"2303.00915","article-title":"BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs","volume":"4","author":"Zhang","year":"2023","journal-title":"arXiv"},{"key":"2025032119061271100_ref39","doi-asserted-by":"publisher","first-page":"2307","DOI":"10.1038\/s41591-023-02504-3","article-title":"A visual-language foundation model for pathology image analysis using medical twitter","volume":"29","author":"Huang","year":"2023","journal-title":"Nat Med"},{"key":"2025032119061271100_ref40","doi-asserted-by":"publisher","first-page":"102559","DOI":"10.1016\/j.media.2022.102559","article-title":"Transformer-based unsupervised contrastive learning for histopathological image classification","volume":"81","author":"Wang","year":"2022","journal-title":"Med Image Anal"},{"key":"2025032119061271100_ref41","doi-asserted-by":"publisher","first-page":"5906","DOI":"10.1038\/s41467-024-50317-w","article-title":"Self-supervised learning for characterising histomorphological diversity and spatial RNA expression prediction across 23 human tissue types","volume":"15","author":"Cisternino","year":"2024","journal-title":"Nat Commun"},{"key":"2025032119061271100_ref42","doi-asserted-by":"publisher","first-page":"3877","DOI":"10.1038\/s41467-020-17678-4","article-title":"A deep learning model to predict RNA-seq expression of tumours from whole slide images","volume":"11","author":"Schmauch","year":"2020","journal-title":"Nat Commun"},{"key":"2025032119061271100_ref43","doi-asserted-by":"publisher","first-page":"e866","DOI":"10.1016\/j.ccell.2022.07.004","article-title":"Pan-cancer integrative histology-genomic analysis via multimodal deep learning","volume":"40","author":"Chen","year":"2022","journal-title":"Cancer Cell"},{"key":"2025032119061271100_ref44","doi-asserted-by":"publisher","first-page":"101563","DOI":"10.1016\/j.media.2019.101563","article-title":"Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images","volume":"58","author":"Graham","year":"2019","journal-title":"Med Image Anal"},{"key":"2025032119061271100_ref45","doi-asserted-by":"publisher","first-page":"102474","DOI":"10.1016\/j.media.2022.102474","article-title":"Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology","volume":"79","author":"Ghaffari Laleh","year":"2022","journal-title":"Med Image Anal"},{"key":"2025032119061271100_ref46","doi-asserted-by":"publisher","first-page":"e84483","DOI":"10.1371\/journal.pone.0084483","article-title":"Boosting the concordance index for survival data\u2014a unified framework to derive and evaluate biomarker combinations","volume":"9","author":"Mayr","year":"2014","journal-title":"PLoS One"},{"key":"2025032119061271100_ref47","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1177\/02537176231176986","article-title":"Survival analysis, Kaplan-Meier curves, and Cox regression: basic concepts","volume":"45","author":"Andrade","year":"2023","journal-title":"Indian J Psychol Med"},{"key":"2025032119061271100_ref48","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1007\/s12561-023-09362-0","article-title":"Evaluating dynamic discrimination performance of risk prediction models for survival outcomes","volume":"15","author":"Zhang","year":"2023","journal-title":"Stat Biosci"},{"key":"2025032119061271100_ref49","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1093\/biostatistics\/kxu010","article-title":"Random survival forests for competing risks","volume":"15","author":"Ishwaran","year":"2014","journal-title":"Biostatistics"},{"key":"2025032119061271100_ref50","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4&lt;385::AID-SIM380&gt;3.0.CO;2-3","article-title":"The lasso method for variable selection in the Cox model","volume":"16","author":"Tibshirani","year":"1997","journal-title":"Stat Med"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/2\/bbaf121\/62519504\/bbaf121.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/2\/bbaf121\/62519504\/bbaf121.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,23]],"date-time":"2025-03-23T08:52:41Z","timestamp":1742719961000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaf121\/8089949"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,3,4]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaf121","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,3]]},"published":{"date-parts":[[2025,3]]},"article-number":"bbaf121"}}