{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:08:47Z","timestamp":1775578127656,"version":"3.50.1"},"reference-count":40,"publisher":"Oxford University Press (OUP)","issue":"18","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61971393"],"award-info":[{"award-number":["61971393"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61871361"],"award-info":[{"award-number":["61871361"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61571414"],"award-info":[{"award-number":["61571414"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61471331"],"award-info":[{"award-number":["61471331"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Breast cancer is a very heterogeneous disease and there is an urgent need to design computational methods that can accurately predict the prognosis of breast cancer for appropriate therapeutic regime. Recently, deep learning-based methods have achieved great success in prognosis prediction, but many of them directly combine features from different modalities that may ignore the complex inter-modality relations. In addition, existing deep learning-based methods do not take intra-modality relations into consideration that are also beneficial to prognosis prediction. Therefore, it is of great importance to develop a deep learning-based method that can take advantage of the complementary information between intra-modality and inter-modality by integrating data from different modalities for more accurate prognosis prediction of breast cancer.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We present a novel unified framework named genomic and pathological deep bilinear network (GPDBN) for prognosis prediction of breast cancer by effectively integrating both genomic data and pathological images. In GPDBN, an inter-modality bilinear feature encoding module is proposed to model complex inter-modality relations for fully exploiting intrinsic relationship of the features across different modalities. Meanwhile, intra-modality relations that are also beneficial to prognosis prediction, are captured by two intra-modality bilinear feature encoding modules. Moreover, to take advantage of the complementary information between inter-modality and intra-modality relations, GPDBN further combines the inter- and intra-modality bilinear features by using a multi-layer deep neural network for final prognosis prediction. Comprehensive experiment results demonstrate that the proposed GPDBN significantly improves the performance of breast cancer prognosis prediction and compares favorably with existing methods.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availabilityand implementation<\/jats:title>\n                  <jats:p>GPDBN is freely available at https:\/\/github.com\/isfj\/GPDBN.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btab185","type":"journal-article","created":{"date-parts":[[2021,3,16]],"date-time":"2021-03-16T13:22:56Z","timestamp":1615900976000},"page":"2963-2970","source":"Crossref","is-referenced-by-count":112,"title":["GPDBN: deep bilinear network integrating both genomic data and pathological images for breast cancer prognosis prediction"],"prefix":"10.1093","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2379-1709","authenticated-orcid":false,"given":"Zhiqin","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}]},{"given":"Ruiqing","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"}]},{"given":"Minghui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"},{"name":"Centers for Biomedical Engineering, University of Science and Technology of China , Hefei AH230027, China"}]},{"given":"Ao","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China , Hefei AH230027, China"},{"name":"Centers for Biomedical Engineering, University of Science and Technology of China , Hefei AH230027, China"}]}],"member":"286","published-online":{"date-parts":[[2021,3,18]]},"reference":[{"key":"2023061310555551800_btab185-B1","doi-asserted-by":"crossref","first-page":"i446","DOI":"10.1093\/bioinformatics\/btz342","article-title":"Deep learning with multimodal representation for pancancer prognosis prediction","volume":"35","author":"Cheerla","year":"2019","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B2","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1093\/bioinformatics\/btz769","article-title":"Deep-learning approach to identifying cancer subtypes using high-dimensional genomic data","volume":"36","author":"Chen","year":"2020","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B3","first-page":"1","article-title":"Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"99","author":"Chen","year":"2020","journal-title":"IEEE Trans. Med. Imag"},{"key":"2023061310555551800_btab185-B4","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1093\/bioinformatics\/btx723","article-title":"Identification of topological features in renal tumor microenvironment associated with patient survival","volume":"34","author":"Cheng","year":"2018","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B5","doi-asserted-by":"crossref","first-page":"e91","DOI":"10.1158\/0008-5472.CAN-17-0313","article-title":"Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis","volume":"77","author":"Cheng","year":"2017","journal-title":"Cancer Res"},{"key":"2023061310555551800_btab185-B6","doi-asserted-by":"crossref","first-page":"2851","DOI":"10.1093\/bioinformatics\/bts528","article-title":"FSelector: a Ruby gem for feature selection","volume":"28","author":"Cheng","year":"2012","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B7","doi-asserted-by":"crossref","first-page":"e1006076","DOI":"10.1371\/journal.pcbi.1006076","article-title":"Cox-nnet: an artificial neural network method for prognosis prediction of high-throughput omics data","volume":"14","author":"Ching","year":"2018","journal-title":"PLoS Comput. Biol"},{"key":"2023061310555551800_btab185-B8","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.1038\/s41591-019-0583-3","article-title":"Deep learning-based classification of mesothelioma improves prediction of patient outcome","volume":"25","author":"Courtiol","year":"2019","journal-title":"Nat. Med"},{"key":"2023061310555551800_btab185-B9","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1093\/bioinformatics\/btw344","article-title":"Evaluating the molecule-based prediction of clinical drug responses in cancer","volume":"32","author":"Ding","year":"2016","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B10","first-page":"457","author":"Fukui","year":"2016"},{"key":"2023061310555551800_btab185-B11","first-page":"6639","author":"Gao","year":"2019"},{"key":"2023061310555551800_btab185-B12","first-page":"317","author":"Gao","year":"2016"},{"key":"2023061310555551800_btab185-B13","doi-asserted-by":"crossref","first-page":"e184","DOI":"10.1093\/bioinformatics\/btl230","article-title":"Predicting the prognosis of breast cancer by integrating clinical and microarray data with Bayesian networks","volume":"22","author":"Gevaert","year":"2006","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B14","doi-asserted-by":"crossref","first-page":"391","DOI":"10.3816\/CBC.2005.n.043","article-title":"The global breast cancer burden: variations in epidemiology and survival","volume":"6","author":"Hortobagyi","year":"2005","journal-title":"Clin. Breast Cancer"},{"key":"2023061310555551800_btab185-B15","first-page":"12136","article-title":"Deep multimodal multilinear fusion with high-order polynomial pooling","author":"Hou","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"2023061310555551800_btab185-B16","doi-asserted-by":"crossref","first-page":"166","DOI":"10.3389\/fgene.2019.00166","article-title":"SALMON: survival analysis learning with multi-omics neural networks on breast cancer","volume":"10","author":"Huang","year":"2019","journal-title":"Front. Genet"},{"key":"2023061310555551800_btab185-B17","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1038\/nmeth.4627","article-title":"Using deep learning to model the hierarchical structure and function of a cell","volume":"15","author":"Ma","year":"2018","journal-title":"Nat. Methods"},{"key":"2023061310555551800_btab185-B18","doi-asserted-by":"crossref","first-page":"E2970","DOI":"10.1073\/pnas.1717139115","article-title":"Predicting cancer outcomes from histology and genomics using convolutional networks","volume":"115","author":"Mobadersany","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023061310555551800_btab185-B19","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.cmpb.2017.06.001","article-title":"Computer-aided prediction of axillary lymph node status in breast cancer using tumor surrounding tissue features in ultrasound images","volume":"146","author":"Moon","year":"2017","journal-title":"Comput. Methods Programs Biomed"},{"key":"2023061310555551800_btab185-B20","first-page":"689","author":"Ngiam","year":"2011"},{"key":"2023061310555551800_btab185-B21","first-page":"551","author":"Nguyen","year":"2013"},{"key":"2023061310555551800_btab185-B22","doi-asserted-by":"crossref","first-page":"2888","DOI":"10.1093\/bioinformatics\/btaa056","article-title":"Integrative analysis of cross-modal features for the prognosis prediction of clear cell renal cell carcinoma","volume":"36","author":"Ning","year":"2020","journal-title":"Bioinformatics"},{"key":"2023061310555551800_btab185-B23","doi-asserted-by":"crossref","first-page":"1812","DOI":"10.1016\/S0140-6736(11)61539-0","article-title":"Gene expression profiling in breast cancer: classification, prognostication, and prediction","volume":"378","author":"Reis-Filho","year":"2011","journal-title":"Lancet"},{"key":"2023061310555551800_btab185-B24","first-page":"1","article-title":"Deep multi-instance learning using multi-modal data for diagnosis of lymphocytosis","volume":"99","author":"Sahasrabudhe","year":"2020","journal-title":"IEEE J. Biomed. Health Inf"},{"key":"2023061310555551800_btab185-B25","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/TMI.2019.2920608","article-title":"Integrative analysis of pathological images and multi-dimensional genomic data for early-stage cancer prognosis","volume":"39","author":"Shao","year":"2019","journal-title":"IEEE Trans. Med. Imag"},{"key":"2023061310555551800_btab185-B26","doi-asserted-by":"crossref","first-page":"101795","DOI":"10.1016\/j.media.2020.101795","article-title":"Multi-task multi-modal learning for joint diagnosis and prognosis of human cancers","volume":"65","author":"Shao","year":"2020","journal-title":"Med. Image Anal"},{"key":"2023061310555551800_btab185-B27","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.cmpb.2018.04.008","article-title":"Integrating genomic data and pathological images to effectively predict breast cancer clinical outcome","volume":"161","author":"Sun","year":"2018","journal-title":"Comput. Methods Programs Biomed"},{"key":"2023061310555551800_btab185-B28","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1109\/TCBB.2018.2806438","article-title":"A multimodal deep neural network for human breast cancer prognosis prediction by integrating multi-dimensional data","volume":"16","author":"Sun","year":"2018","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinf"},{"key":"2023061310555551800_btab185-B29","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1162\/089976600300015349","article-title":"Separating style and content with bilinear models","volume":"12","author":"Tenenbaum","year":"2000","journal-title":"Neural Comput"},{"key":"2023061310555551800_btab185-B30","doi-asserted-by":"crossref","first-page":"1999","DOI":"10.1056\/NEJMoa021967","article-title":"A gene-expression signature as a predictor of survival in breast cancer","volume":"347","author":"Van De Vijver","year":"2002","journal-title":"N. Engl. J. Med"},{"key":"2023061310555551800_btab185-B31","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1016\/S0140-6736(05)17947-1","article-title":"Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer","volume":"365","author":"Wang","year":"2005","journal-title":"Lancet"},{"key":"2023061310555551800_btab185-B32","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1109\/TMI.2015.2458702","article-title":"Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images","volume":"35","author":"Xu","year":"2016","journal-title":"IEEE Trans. Med. Imag"},{"key":"2023061310555551800_btab185-B33","first-page":"928","author":"Xu","year":"2012"},{"key":"2023061310555551800_btab185-B34","first-page":"406","author":"Yao","year":"2017"},{"key":"2023061310555551800_btab185-B35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/ncomms12474","article-title":"Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features","volume":"7","author":"Yu","year":"2016","journal-title":"Nat. Commun"},{"key":"2023061310555551800_btab185-B36","first-page":"1821","author":"Yu","year":"2017"},{"key":"2023061310555551800_btab185-B37","doi-asserted-by":"crossref","first-page":"157ra143","DOI":"10.1126\/scitranslmed.3004330","article-title":"Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling","volume":"4","author":"Yuan","year":"2012","journal-title":"Sci. Transl. Med"},{"key":"2023061310555551800_btab185-B38","first-page":"1114","author":"Zadeh","year":"2017"},{"key":"2023061310555551800_btab185-B39","first-page":"544","author":"Zhu","year":"2016"},{"key":"2023061310555551800_btab185-B40","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1038\/nmeth.2956","article-title":"TCGA-assembler: open-source software for retrieving and processing TCGA data","volume":"11","author":"Zhu","year":"2014","journal-title":"Nat. Methods"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btab185\/36849592\/btab185.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/18\/2963\/50579022\/btab185.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/18\/2963\/50579022\/btab185.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T10:57:50Z","timestamp":1686653870000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/18\/2963\/6178291"}},"subtitle":[],"editor":[{"given":"Zhiyong","family":"Lu","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,3,18]]},"references-count":40,"journal-issue":{"issue":"18","published-print":{"date-parts":[[2021,9,29]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btab185","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,9,15]]},"published":{"date-parts":[[2021,3,18]]}}}