{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T04:04:21Z","timestamp":1783483461542,"version":"3.55.0"},"reference-count":51,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2023,1,13]],"date-time":"2023-01-13T00:00:00Z","timestamp":1673568000000},"content-version":"vor","delay-in-days":12,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61971393"],"award-info":[{"award-number":["61971393"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61871361"],"award-info":[{"award-number":["61871361"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61571414"],"award-info":[{"award-number":["61571414"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61471331"],"award-info":[{"award-number":["61471331"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Accurately predicting cancer survival is crucial for helping clinicians to plan appropriate treatments, which largely improves the life quality of cancer patients and spares the related medical costs. Recent advances in survival prediction methods suggest that integrating complementary information from different modalities, e.g. histopathological images and genomic data, plays a key role in enhancing predictive performance. Despite promising results obtained by existing multimodal methods, the disparate and heterogeneous characteristics of multimodal data cause the so-called modality gap problem, which brings in dramatically diverse modality representations in feature space. Consequently, detrimental modality gaps make it difficult for comprehensive integration of multimodal information via representation learning and therefore pose a great challenge to further improvements of cancer survival prediction.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>To solve the above problems, we propose a novel method called cross-aligned multimodal representation learning (CAMR), which generates both modality-invariant and -specific representations for more accurate cancer survival prediction. Specifically, a cross-modality representation alignment learning network is introduced to reduce modality gaps by effectively learning modality-invariant representations in a common subspace, which is achieved by aligning the distributions of different modality representations through adversarial training. Besides, we adopt a cross-modality fusion module to fuse modality-invariant representations into a unified cross-modality representation for each patient. Meanwhile, CAMR learns modality-specific representations which complement modality-invariant representations and therefore provides a holistic view of the multimodal data for cancer survival prediction. Comprehensive experiment results demonstrate that CAMR can successfully narrow modality gaps and consistently yields better performance than other survival prediction methods using multimodal data.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>CAMR is freely available at https:\/\/github.com\/wxq-ustc\/CAMR.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btad025","type":"journal-article","created":{"date-parts":[[2023,1,12]],"date-time":"2023-01-12T13:12:55Z","timestamp":1673529175000},"source":"Crossref","is-referenced-by-count":44,"title":["CAMR: cross-aligned multimodal representation learning for cancer survival prediction"],"prefix":"10.1093","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5628-7088","authenticated-orcid":false,"given":"Xingqi","family":"Wu","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7909-6831","authenticated-orcid":false,"given":"Yi","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minghui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ao","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,1,13]]},"reference":[{"key":"2023012019093141800_btad025-B1","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal machine learning: a survey and taxonomy","volume":"41","author":"Baltrusaitis","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"2023012019093141800_btad025-B71388295","doi-asserted-by":"publisher","first-page":"e1001794","DOI":"10.1371\/journal.pmed.1001794","article-title":"Open access to large scale datasets is needed to translate knowledge of cancer heterogeneity into better patient outcomes","volume":"12","author":"Beck","year":"2015","journal-title":"PLoS Med."},{"key":"2023012019093141800_btad025-B2","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1038\/s43018-022-00388-9","article-title":"Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer","volume":"3","author":"Boehm","year":"2022","journal-title":"Nat. Cancer"},{"key":"2023012019093141800_btad025-B3","doi-asserted-by":"crossref","first-page":"R100","DOI":"10.1186\/gb-2006-7-10-r100","article-title":"CellProfiler: image analysis software for identifying and quantifying cell phenotypes","volume":"7","author":"Carpenter","year":"2006","journal-title":"Genome Biol"},{"key":"2023012019093141800_btad025-B4","doi-asserted-by":"crossref","first-page":"i446","DOI":"10.1093\/bioinformatics\/btz342","article-title":"Deep learning with multimodal representation for pancancer prognosis prediction","volume":"35","author":"Cheerla","year":"2019","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B6","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","article-title":"Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"41","author":"Chen","year":"2022","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2023012019093141800_btad025-B7","doi-asserted-by":"crossref","first-page":"e91","DOI":"10.1158\/0008-5472.CAN-17-0313","article-title":"Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis","volume":"77","author":"Cheng","year":"2017","journal-title":"Cancer Res"},{"key":"2023012019093141800_btad025-B8","doi-asserted-by":"crossref","first-page":"3185","DOI":"10.1109\/JBHI.2021.3064696","article-title":"Deep semisupervised multitask learning model and its interpretability for survival analysis","volume":"25","author":"Chi","year":"2021","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"2023012019093141800_btad025-B10","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1109\/TIP.2021.3139229","article-title":"Big-hypergraph factorization neural network for survival prediction from whole slide image","volume":"31","author":"Di","year":"2022","journal-title":"IEEE Trans. Image Process"},{"key":"2023012019093141800_btad025-B11","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1093\/bioinformatics\/btw344","article-title":"Evaluating the molecule-based prediction of clinical drug responses in cancer","volume":"32","author":"Ding","year":"2016","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B12","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1186\/s12859-019-3172-z","article-title":"MLW-gcForest: a multi-weighted gcForest model towards the staging of lung adenocarcinoma based on multi-modal genetic data","volume":"20","author":"Dong","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2023012019093141800_btad025-B13","first-page":"699","article-title":"Predicting the survival of cancer patients with multimodal graph neural network","author":"Gao","year":"2021","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform"},{"key":"2023012019093141800_btad025-B14","doi-asserted-by":"crossref","first-page":"e184","DOI":"10.1093\/bioinformatics\/btl230","article-title":"Predicting the prognosis of breast cancer by integrating clinical and microarray data with Bayesian networks","volume":"22","author":"Gevaert","year":"2006","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B15","first-page":"1122","author":"Hazarika","year":"2020"},{"key":"2023012019093141800_btad025-B16","doi-asserted-by":"crossref","first-page":"166","DOI":"10.3389\/fgene.2019.00166","article-title":"SALMON: survival analysis learning with multi-omics neural networks on breast cancer","volume":"10","author":"Huang","year":"2019","journal-title":"Front. Genet"},{"key":"2023012019093141800_btad025-B17","article-title":"Random survival forests","volume":"2, 841\u2013860.","author":"Ishwaran","year":"2008","journal-title":"Ann. Appl. Stat"},{"key":"2023012019093141800_btad025-B18","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s12874-018-0482-1","article-title":"DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network","volume":"18","author":"Katzman","year":"2018","journal-title":"BMC Med. Res. Methodol"},{"key":"2023012019093141800_btad025-B19","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1093\/bioinformatics\/btab581","article-title":"Deep learning-based classification of breast cancer cells using transmembrane receptor dynamics","volume":"38","author":"Kim","year":"2021","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B20","doi-asserted-by":"crossref","first-page":"e41","DOI":"10.5808\/GI.2019.17.4.e41","article-title":"Review of statistical methods for survival analysis using genomic data","volume":"17","author":"Lee","year":"2019","journal-title":"Genomics Inform"},{"key":"2023012019093141800_btad025-B21","first-page":"2247","author":"Liu","year":"2018"},{"key":"2023012019093141800_btad025-B22","first-page":"164","article-title":"Modality to modality translation: an adversarial representation learning and graph fusion network for multimodal fusion","volume":"34","author":"Mai","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"2023012019093141800_btad025-B23","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.1717139115","article-title":"Predicting cancer outcomes from histology and genomics using convolutional networks","volume":"115","author":"Mobadersany","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023012019093141800_btad025-B24","doi-asserted-by":"crossref","first-page":"2888","DOI":"10.1093\/bioinformatics\/btaa056","article-title":"Integrative analysis of cross-modal features for the prognosis prediction of clear cell renal cell carcinoma","volume":"36","author":"Ning","year":"2020","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TNNLS.2021.3112194","article-title":"Multi-constraint latent representation learning for prognosis analysis using multi-modal data","author":"Ning","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"2023012019093141800_btad025-B26","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1109\/TMI.2021.3108802","article-title":"Relation-aware shared representation learning for cancer prognosis analysis with auxiliary clinical variables and incomplete multi-modality data","volume":"41","author":"Ning","year":"2021","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2023012019093141800_btad025-B27","first-page":"4594","author":"Pan","year":"2016"},{"key":"2023012019093141800_btad025-B28","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/TMI.2019.2920608","article-title":"Integrative analysis of pathological images and multi-dimensional genomic data for early-stage cancer prognosis","volume":"39","author":"Shao","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2023012019093141800_btad025-B29","doi-asserted-by":"crossref","first-page":"3739","DOI":"10.1109\/TMI.2021.3097319","article-title":"Weakly supervised deep ordinal Cox model for survival prediction from whole-slide pathological images","volume":"40","author":"Shao","year":"2021","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2023012019093141800_btad025-B30","first-page":"1429","author":"Subramanian","year":"2021"},{"key":"2023012019093141800_btad025-B31","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1109\/TCBB.2018.2806438","article-title":"A multimodal deep neural network for human breast cancer prognosis prediction by integrating multi-dimensional data","volume":"16","author":"Sun","year":"2019","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform"},{"key":"2023012019093141800_btad025-B32","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.cmpb.2018.04.008","article-title":"Integrating genomic data and pathological images to effectively predict breast cancer clinical outcome","volume":"161","author":"Sun","year":"2018","journal-title":"Comput. Methods Programs Biomed"},{"key":"2023012019093141800_btad025-B33","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3","article-title":"The lasso method for variable selection in the Cox model","volume":"16","author":"Tibshirani","year":"1997","journal-title":"Stat. Med"},{"key":"2023012019093141800_btad025-B34","doi-asserted-by":"crossref","first-page":"1590","DOI":"10.1016\/j.csda.2008.05.021","article-title":"Survival prediction using gene expression data: a review and comparison","volume":"53","author":"van Wieringen","year":"2009","journal-title":"Comput. Stat. Data Anal"},{"key":"2023012019093141800_btad025-B35","first-page":"154","author":"Wang","year":"2017"},{"key":"2023012019093141800_btad025-B36","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.semcancer.2019.11.012","article-title":"Emergence in protein derived nanomedicine as anticancer therapeutics: more than a tour de force","volume":"69","author":"Wang","year":"2021","journal-title":"Semin. Cancer Biol"},{"key":"2023012019093141800_btad025-B37","doi-asserted-by":"crossref","first-page":"2963","DOI":"10.1093\/bioinformatics\/btab185","article-title":"GPDBN: deep bilinear network integrating both genomic data and pathological images for breast cancer prognosis prediction","volume":"37","author":"Wang","year":"2021","journal-title":"Bioinformatics"},{"key":"2023012019093141800_btad025-B38","article-title":"Jointly modeling deep video and compositional text to bridge vision and language in a unified framework","volume":"29, 2346\u20132352","author":"Xu","year":"2015","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"2023012019093141800_btad025-B39","doi-asserted-by":"crossref","first-page":"3261","DOI":"10.1109\/TCYB.2020.3009004","article-title":"Learning cross-modal common representations by private-shared subspaces separation","volume":"52","author":"Xu","year":"2020","journal-title":"IEEE Trans. Cybern"},{"key":"2023012019093141800_btad025-B40","doi-asserted-by":"crossref","first-page":"167","DOI":"10.4310\/SII.2013.v6.n2.a1","article-title":"A cocktail algorithm for solving the elastic net penalized Cox\u2019s regression in high dimensions","volume":"6","author":"Yang","year":"2013","journal-title":"Stat. Interface"},{"key":"2023012019093141800_btad025-B41","doi-asserted-by":"crossref","first-page":"101789","DOI":"10.1016\/j.media.2020.101789","article-title":"Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks","volume":"65","author":"Yao","year":"2020","journal-title":"Med. Image Anal"},{"key":"2023012019093141800_btad025-B42","doi-asserted-by":"crossref","first-page":"2130","DOI":"10.1093\/bib\/bby073","article-title":"Breast cancer prognosis signature: linking risk stratification to disease subtypes","volume":"20","author":"Yu","year":"2019","journal-title":"Brief. Bioinform"},{"key":"2023012019093141800_btad025-B43","doi-asserted-by":"crossref","first-page":"12474","DOI":"10.1038\/ncomms12474","article-title":"Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features","volume":"7","author":"Yu","year":"2016","journal-title":"Nat. Commun"},{"key":"2023012019093141800_btad025-B44","doi-asserted-by":"crossref","DOI":"10.1126\/scitranslmed.3004330","article-title":"Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling","volume":"4","author":"Yuan","year":"2012","journal-title":"Sci. Transl. Med"},{"key":"2023012019093141800_btad025-B45","first-page":"1103","author":"Zadeh","year":"2017"},{"key":"2023012019093141800_btad025-B46","doi-asserted-by":"crossref","first-page":"lqab015","DOI":"10.1093\/nargab\/lqab015","article-title":"Two-stage Cox-nnet: biologically interpretable neural-network model for prognosis prediction and its application in liver cancer survival using histopathology and transcriptomic data","volume":"3","author":"Zhan","year":"2021","journal-title":"NAR Genomics Bioinformatics"},{"key":"2023012019093141800_btad025-B47","doi-asserted-by":"crossref","first-page":"103194","DOI":"10.1016\/j.jbi.2019.103194","article-title":"LSCDFS-MKL: a multiple kernel based method for lung squamous cell carcinomas disease-free survival prediction with pathological and genomic data","volume":"94","author":"Zhang","year":"2019","journal-title":"J. Biomed. Inform"},{"key":"2023012019093141800_btad025-B48","doi-asserted-by":"crossref","first-page":"7504","DOI":"10.1111\/jcmm.15377","article-title":"A five-microRNA signature for individualized prognosis evaluation and radiotherapy guidance in patients with diffuse lower-grade glioma","volume":"24","author":"Zhang","year":"2020","journal-title":"J. Cell. Mol. Med"},{"key":"2023012019093141800_btad025-B49","doi-asserted-by":"crossref","first-page":"477","DOI":"10.3389\/fgene.2018.00477","article-title":"Deep learning-based multi-omics data integration reveals two prognostic subtypes in high-risk neuroblastoma","volume":"9","author":"Zhang","year":"2018","journal-title":"Front. Genet"},{"key":"2023012019093141800_btad025-B50","first-page":"bbac489","article-title":"Adaptive risk-aware sharable and individual subspace learning for cancer survival analysis with multi-modality data","author":"Zhao","year":"2022","journal-title":"Brief. Bioinform"},{"key":"2023012019093141800_btad025-B51","first-page":"6855","author":"Zhu","year":"2017"},{"key":"2023012019093141800_btad025-B52","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1038\/nmeth.2956","article-title":"TCGA-Assembler: open-source software for retrieving and processing TCGA data","volume":"11","author":"Zhu","year":"2014","journal-title":"Nat. Methods"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btad025\/48691600\/btad025.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/1\/btad025\/48805286\/btad025.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/1\/btad025\/48805286\/btad025.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,20]],"date-time":"2023-01-20T19:10:43Z","timestamp":1674241843000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btad025\/6986972"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,1,1]]},"references-count":51,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btad025","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,1,1]]},"published":{"date-parts":[[2023,1,1]]},"article-number":"btad025"}}