{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T02:19:00Z","timestamp":1776305940869,"version":"3.50.1"},"reference-count":205,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T00:00:00Z","timestamp":1721865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Cancer research encompasses data across various scales, modalities, and resolutions, from screening and diagnostic imaging to digitized histopathology slides to various types of molecular data and clinical records. The integration of these diverse data types for personalized cancer care and predictive modeling holds the promise of enhancing the accuracy and reliability of cancer screening, diagnosis, and treatment. Traditional analytical methods, which often focus on isolated or unimodal information, fall short of capturing the complex and heterogeneous nature of cancer data. The advent of deep neural networks has spurred the development of sophisticated multimodal data fusion techniques capable of extracting and synthesizing information from disparate sources. Among these, Graph Neural Networks (GNNs) and Transformers have emerged as powerful tools for multimodal learning, demonstrating significant success. This review presents the foundational principles of multimodal learning including oncology data modalities, taxonomy of multimodal learning, and fusion strategies. We delve into the recent advancements in GNNs and Transformers for the fusion of multimodal data in oncology, spotlighting key studies and their pivotal findings. We discuss the unique challenges of multimodal learning, such as data heterogeneity and integration complexities, alongside the opportunities it presents for a more nuanced and comprehensive understanding of cancer. Finally, we present some of the latest comprehensive multimodal pan-cancer data sources. By surveying the landscape of multimodal data integration in oncology, our goal is to underline the transformative potential of multimodal GNNs and Transformers. Through technological advancements and the methodological innovations presented in this review, we aim to chart a course for future research in this promising field. This review may be the first that highlights the current state of multimodal modeling applications in cancer using GNNs and transformers, presents comprehensive multimodal oncology data sources, and sets the stage for multimodal evolution, encouraging further exploration and development in personalized cancer care.<\/jats:p>","DOI":"10.3389\/frai.2024.1408843","type":"journal-article","created":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T14:18:26Z","timestamp":1721917106000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":97,"title":["Multimodal data integration for oncology in the era of deep neural networks: a review"],"prefix":"10.3389","volume":"7","author":[{"given":"Asim","family":"Waqas","sequence":"first","affiliation":[]},{"given":"Aakash","family":"Tripathi","sequence":"additional","affiliation":[]},{"given":"Ravi P.","family":"Ramachandran","sequence":"additional","affiliation":[]},{"given":"Paul A.","family":"Stewart","sequence":"additional","affiliation":[]},{"given":"Ghulam","family":"Rasool","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,7,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774","article-title":"Gpt-4 technical report","author":"Achiam","year":"2023","journal-title":"arXiv"},{"key":"B2","doi-asserted-by":"publisher","first-page":"919046","DOI":"10.3389\/fmedt.2022.919046","article-title":"Failure detection in deep neural networks for medical imaging","volume":"4","author":"Ahmed","year":"","journal-title":"Front. Med. Technol"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1007\/s00034-023-02454-8","article-title":"Transformers in time-series analysis: a tutorial","author":"Ahmed","year":"","journal-title":"arXiv"},{"key":"B4","doi-asserted-by":"publisher","first-page":"102027","DOI":"10.1016\/j.compmedimag.2021.102027","article-title":"A survey on graph-based deep learning for computational histopathology","volume":"95","author":"Ahmedt-Aristizabal","year":"2022","journal-title":"Comp. Med. Imag. Graph"},{"key":"B5","first-page":"1","author":"Akter","year":"2022"},{"key":"B6","unstructured":"Flamingo: a visual language model for few-shot learning\n          \n          23716\n          23736\n          \n            \n              Alayrac\n              J.-B.\n            \n            \n              Donahue\n              J.\n            \n            \n              Luc\n              P.\n            \n            \n              Miech\n              A.\n            \n            \n              Barr\n              I.\n            \n            \n              Hasson\n              Y.\n            \n          \n          Adv. Neural Inf. Process. Syst\n          35\n          2022"},{"key":"B7","first-page":"04034","author":"Alghamdi","year":"2023"},{"key":"B8","first-page":"7","author":"Al-jabery","year":"2020"},{"key":"B9","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.comcom.2022.09.029","article-title":"Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case","volume":"196","author":"Almasan","year":"2022","journal-title":"Comput. Commun"},{"key":"B10","author":"Anand","year":"2020"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13059-017-1189-z","article-title":"DeepCpG: accurate prediction of single-cell DNA methylation states using deep learning","volume":"18","author":"Angermueller","year":"2017","journal-title":"Genome Biol"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1200\/CCI.17.00118","article-title":"Oncologists' views regarding the role of electronic health records in care coordination","volume":"2","author":"Asan","year":"2018","journal-title":"JCO Clini. Cancer Inform"},{"key":"B13","doi-asserted-by":"publisher","first-page":"107637","DOI":"10.1016\/j.patcog.2020.107637","article-title":"Hypergraph convolution and hypergraph attention","volume":"110","author":"Bai","year":"2021","journal-title":"Pattern Recognit"},{"key":"B14","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal machine learning: a survey and taxonomy","volume":"41","author":"Baltru\u0161aitis","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B15","doi-asserted-by":"publisher","first-page":"81656","DOI":"10.1109\/ACCESS.2023.3301160","article-title":"Efficient scopeformer: towards scalable and rich feature extraction for intracranial hemorrhage detection","volume":"11","author":"Barhoumi","year":"2023","journal-title":"IEEE Access"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.00830","article-title":"Medaide: Leveraging large language models for on-premise medical assistance on edge devices","author":"Basit","year":"2024","journal-title":"arXiv"},{"key":"B17","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1038\/s41568-021-00408-3","article-title":"Harnessing multimodal data integration to advance precision oncology","volume":"22","author":"Boehm","year":"2021","journal-title":"Nat. Rev. Cancer"},{"key":"B18","author":"Bommasani","year":"2022","journal-title":"On the Opportunities and Risks of Foundation Models"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNNLS.2022.3229161","article-title":"Deep neural networks and tabular data: a survey","volume":"35","author":"Borisov","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1260374","DOI":"10.3389\/fonc.2023.1260374","article-title":"Ai\/ml advances in non-small cell lung cancer biomarker discovery","volume":"13","author":"\u00c7al\u0131\u015fkan","year":"2023","journal-title":"Front. Oncol"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1458","DOI":"10.1038\/s41587-022-01284-4","article-title":"Multi-omics single-cell data integration and regulatory inference with graph-linked embedding","volume":"40","author":"Cao","year":"2022","journal-title":"Nat. Biotechnol"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e218","DOI":"10.1002\/mp.13764","article-title":"Computer-aided diagnosis in the era of deep learning","volume":"47","author":"Chan","year":"2020","journal-title":"Med. Phys"},{"key":"B23","first-page":"772","article-title":"\u201cLymph node gross tumor volume detection in oncology imaging via relationship learning using graph neural network,\u201d","volume-title":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part VII 23","author":"Chao","year":"2020"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btad643","article-title":"Explainable multilayer graph neural network for cancer gene prediction","author":"Chatzianastasis","year":"2023","journal-title":"arXiv"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1289669","DOI":"10.3389\/frai.2023.1289669","article-title":"Trends and hotspots in research on medical images with deep learning: a bibliometric analysis from 2013 to 2023","volume":"6","author":"Chen","year":"2023","journal-title":"Front. Artif. Intellig"},{"key":"B26","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","article-title":"Pathomic Fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"41","author":"Chen","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"B27","doi-asserted-by":"crossref","first-page":"3995","DOI":"10.1109\/ICCV48922.2021.00398","article-title":"\u201cMultimodal co-attention transformer for survival prediction in gigapixel whole slide images,\u201d","author":"Chen","year":"2021","journal-title":"2021 IEEE\/CVF International Conference on Computer Vision (ICCV)"},{"key":"B28","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1145\/3097983.3098126","article-title":"\u201cGRAM: graph-based attention model for healthcare representation learning,\u201d","volume-title":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Choi","year":"2017"},{"key":"B29","first-page":"657","article-title":"\u201cCo-graph attention reasoning based imaging and clinical features integration for lymph node metastasis prediction,\u201d","volume-title":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part V 24","author":"Cui","year":"2021"},{"key":"B30","article-title":"Safe rlhf: Safe reinforcement learning from human feedback","author":"Dai","year":"2023","journal-title":"arXiv"},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.1109\/ICECA.2018.8474912","article-title":"\u201cFeature extraction by using deep learning: a survey,\u201d","author":"Dara","year":"2018","journal-title":"2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA"},{"key":"B32","unstructured":"Convolutional neural networks on graphs with fast localized spectral filtering\n          \n          3844\n          3852\n          \n            \n              Defferrard\n              M.\n            \n            \n              Bresson\n              X.\n            \n            \n              Vandergheynst\n              P.\n            \n          \n          Adv. Neural Inf. Process. Syst\n          29\n          2016"},{"key":"B33","doi-asserted-by":"publisher","first-page":"4669","DOI":"10.1109\/TSP.2021.3096804","article-title":"PremiUm-CNN: propagating uncertainty towards robust convolutional neural networks","volume":"69","author":"Dera","year":"2021","journal-title":"IEEE Trans. Signal Proc"},{"key":"B34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/MLSP.2019.8918747","article-title":"\u201cExtended variational inference for propagating uncertainty in convolutional neural networks,\u201d","volume-title":"2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)","author":"Dera","year":"2019"},{"key":"B35","first-page":"3767","article-title":"\u201cETA prediction with graph neural networks in google maps,\u201d","author":"Derrow-Pinion","year":"2021","journal-title":"Proceedings of the 30th ACM International Conference on Information"},{"key":"B36","volume-title":"Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding","author":"Devlin","year":"2019"},{"key":"B37","volume-title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","author":"Dosovitskiy","year":"2020"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1912.07517","article-title":"Zoom in to where it matters: a hierarchical graph based model for mammogram analysis","author":"Du","year":"2019","journal-title":"arXiv"},{"key":"B39","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1038\/s42256-023-00624-6","article-title":"Multimodal learning with graphs","volume":"5","author":"Ektefaie","year":"2023","journal-title":"Nat. Mach. Intell"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-12915-x","article-title":"Network curvature as a hallmark of brain structural connectivity","volume":"10","author":"Farooq","year":"2019","journal-title":"Nat. Commun"},{"key":"B41","doi-asserted-by":"crossref","DOI":"10.5772\/intechopen.111651","volume-title":"Graph Neural Networks and Reinforcement Learning: A Survey","author":"Fathinezhad","year":"2023"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-020-03724-x","article-title":"DPDDI: a deep predictor for drug-drug interactions","volume":"21","author":"Feng","year":"2020","journal-title":"BMC Bioinformat"},{"key":"B43","first-page":"30","article-title":"\u201cProtein interface prediction using graph convolutional networks,\u201d","volume-title":"Adv. Neural Inf. Process. Syst","author":"Fout","year":"2017"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-07757-5","article-title":"Combining graph neural networks and spatio-temporal disease models to improve the prediction of weekly covid-19 cases in germany","author":"Fritz","year":"2022","journal-title":"Sci. Rep"},{"key":"B45","doi-asserted-by":"publisher","first-page":"4291","DOI":"10.1109\/TNNLS.2020.3019893","article-title":"Attention in natural language processing","volume":"32","author":"Galassi","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B46","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1158\/1078-0432.CCR-22-0390","article-title":"Facts and hopes on the use of artificial intelligence for predictive immunotherapy biomarkers in cancer","volume":"29","author":"Ghaffari Laleh","year":"2023","journal-title":"Clini.Cancer Res"},{"key":"B47","first-page":"1263","article-title":"\u201cNeural message passing for quantum chemistry,\u201d","volume-title":"International Conference on Machine Learning","author":"Gilmer","year":"2017"},{"key":"B48","doi-asserted-by":"crossref","DOI":"10.1111\/liv.15974","article-title":"\u201cOptimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes,\u201d","volume-title":"Liver International","author":"Giuffr\u00e8","year":"2024"},{"key":"B49","first-page":"2086","article-title":"\u201cTowards explaining the effects of data preprocessing on machine learning,\u201d","volume-title":"IEEE 35th International Conference on Data Engineering (ICDE)","author":"Gonzalez Zelaya","year":"2019"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1109","DOI":"10.1056\/NEJMp1607591","article-title":"Toward a shared vision for cancer genomic data","volume":"375","author":"Grossman","year":"2016","journal-title":"New Engl. J. Med"},{"key":"B51","doi-asserted-by":"publisher","first-page":"e13654","DOI":"10.1111\/exsy.13654","article-title":"When geoscience meets generative ai and large language models: foundations, trends, and future challenges","volume":"2024","author":"Hadid","year":"2024","journal-title":"Expert Syst"},{"key":"B52","first-page":"30","article-title":"\u201cInductive representation learning on large graphs,\u201d","volume-title":"Adv. Neural Inf. Process. Syst","author":"Hamilton","year":"2017"},{"key":"B53","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/978-3-031-01588-5","article-title":"Graph representation learning","volume":"14","author":"Hamilton","year":"2020","journal-title":"Synth. Lect. Artif. Intellig. Mach. Learn"},{"key":"B54","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TPAMI.2022.3152247","article-title":"A survey on vision transformer","volume":"45","author":"Han","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.02469","article-title":"Vision-language models for medical report generation and visual question answering: a review","author":"Hartsock","year":"2024","journal-title":"arXiv"},{"key":"B56","doi-asserted-by":"publisher","first-page":"23","DOI":"10.3389\/frma.2018.00023","article-title":"Dimensions: building context for search and evaluation","volume":"3","author":"Hook","year":"2018","journal-title":"Front. Res. Metrics Anal"},{"key":"B57","first-page":"1439","article-title":"\u201cUnit: Multimodal multitask learning with a unified transformer,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Hu","year":"2021"},{"key":"B58","first-page":"562","article-title":"\u201cEdge-variational graph convolutional networks for uncertainty-aware disease prediction,\u201d","volume-title":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part VII 23","author":"Huang","year":"2020"},{"key":"B59","volume-title":"What Makes Multi-Modal Learning Better Than Single (Provably)","author":"Huang","year":"2021"},{"key":"B60","first-page":"9226","article-title":"\u201cModality competition: What makes joint training of multi-modal network fail in deep learning? (provably),\u201d","volume-title":"International Conference on Machine Learning","author":"Huang","year":"2022"},{"key":"B61","doi-asserted-by":"publisher","first-page":"884749","DOI":"10.3389\/frai.2022.884749","article-title":"A survey on human cancer categorization based on deep learning","volume":"5","author":"Ibrahim","year":"2022","journal-title":"Front. Artif. Intellig"},{"key":"B62","doi-asserted-by":"publisher","first-page":"2395","DOI":"10.3390\/healthcare10122395","article-title":"Breast cancer dataset, classification and detection using deep learning","volume":"10","author":"Iqbal","year":"2022","journal-title":"Healthcare"},{"key":"B63","doi-asserted-by":"publisher","first-page":"86570","DOI":"10.1109\/ACCESS.2019.2925041","article-title":"Mitochondrial organelle movement classification (fission and fusion) via convolutional neural network approach","volume":"7","author":"Iqbal","year":"2019","journal-title":"IEEE Access"},{"key":"B64","doi-asserted-by":"crossref","first-page":"0281","DOI":"10.1109\/CCWC54503.2022.9720752","article-title":"\u201cPrivacy-preserving federated learning model for healthcare data,\u201d","volume-title":"2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC)","author":"Islam","year":"2022"},{"key":"B65","author":"Jaegle","year":"2021","journal-title":"Perceiver IO: A General Architecture for Structured Inputs"},{"key":"B66","doi-asserted-by":"publisher","first-page":"e1006555","DOI":"10.1371\/journal.pcbi.1006555","article-title":"Building gene regulatory networks from scATAC-seq and scRNA-seq using linked self organizing maps","volume":"15","author":"Jansen","year":"2019","journal-title":"PLoS Comput. Biol"},{"key":"B67","first-page":"9938","article-title":"\u201cMitigating modality collapse in multimodal VAEs via impartial optimization,\u201d","volume-title":"International Conference on Machine Learning","author":"Javaloy","year":"2022"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1101\/2020.09.17.301879","article-title":"DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome","author":"Ji","year":"2020","journal-title":"bioRxiv"},{"key":"B69","first-page":"4904","article-title":"\u201cScaling up visual and vision-language representation learning with noisy text supervision,\u201d","volume-title":"International Conference on Machine Learning","author":"Jia","year":"2021"},{"key":"B70","article-title":"Graph convolutional reinforcement learning","author":"Jiang","year":"2018","journal-title":"arXiv"},{"key":"B71","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1038\/s41568-022-00502-0","article-title":"Big Data in basic and translational cancer research","volume":"22","author":"Jiang","year":"2022","journal-title":"Nat. Rev. Cancer"},{"key":"B72","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1109\/TAI.2022.3194869","article-title":"Graph representation learning meets computer vision: a survey","volume":"4","author":"Jiao","year":"2022","journal-title":"IEEE Trans. Artif. Intellig"},{"key":"B73","article-title":"Heterogeneous graph neural networks using self-supervised reciprocally contrastive learning","author":"Jin","year":"2022","journal-title":"arXiv"},{"key":"B74","doi-asserted-by":"publisher","first-page":"18800","DOI":"10.1038\/s41598-021-98408-8","article-title":"Multimodal deep learning models for the prediction of pathologic response to neoadjuvant chemotherapy in breast cancer","volume":"11","author":"Joo","year":"2021","journal-title":"Sci. Rep"},{"key":"B75","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1142\/9789811250477_0034","article-title":"\u201cMulti-omic graph transformers for cancer classification and interpretation,\u201d","volume-title":"Pacific Symposium On Biocomputing 2022","author":"Kaczmarek","year":"2021"},{"key":"B76","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1007\/978-3-030-68238-5_48","article-title":"\u201cLate Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition,\u201d","volume-title":"Computer Vision-ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16","author":"Kalfaoglu","year":"2020"},{"key":"B77","doi-asserted-by":"publisher","first-page":"565","DOI":"10.3390\/diagnostics10080565","article-title":"Multimodal brain tumor classification using deep learning and robust feature selection: a machine learning application for radiologists","volume":"10","author":"Khan","year":"2020","journal-title":"Diagnostics"},{"key":"B78","doi-asserted-by":"publisher","first-page":"1202990","DOI":"10.3389\/frai.2023.1202990","article-title":"Identifying the role of vision transformer for skin cancer\u2013a scoping review","volume":"6","author":"Khan","year":"2023","journal-title":"Front. Artif. Intellig"},{"key":"B79","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf","year":"2016","journal-title":"arXiv"},{"key":"B80","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"B81","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2019","journal-title":"Bioinformatics"},{"key":"B82","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13059-022-02739-2","article-title":"A benchmark study of deep learning-based multi-omics data fusion methods for cancer","volume":"23","author":"Leng","year":"2022","journal-title":"Genome Biol"},{"key":"B83","volume-title":"BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension","author":"Lewis","year":"2019"},{"key":"B84","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1038\/s41551-022-00942-x","article-title":"Graph representation learning in biomedicine and healthcare","volume":"6","author":"Li","year":"2022","journal-title":"Nat. Biomed. Eng"},{"key":"B85","first-page":"5652","article-title":"\u201cSelfdoc: self-supervised document representation learning,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Li","year":""},{"key":"B86","doi-asserted-by":"publisher","first-page":"bbab109","DOI":"10.1093\/bib\/bbab109","article-title":"An effective self-supervised framework for learning expressive molecular global representations to drug discovery","volume":"22","author":"Li","year":"","journal-title":"Brief. Bioinform"},{"key":"B87","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.09258","article-title":"Explainability in graph neural networks: An experimental survey","author":"Li","year":"2022","journal-title":"arXiv"},{"key":"B88","doi-asserted-by":"publisher","first-page":"105027","DOI":"10.1016\/j.compedu.2024.105027","article-title":"Aligning open educational resources to new taxonomies: How ai technologies can help and in which scenarios","volume":"216","author":"Li","year":"2024","journal-title":"Comp. Educ"},{"key":"B89","doi-asserted-by":"publisher","first-page":"e80547","DOI":"10.7554\/eLife.80547","article-title":"Early stage NSCLS patients' prognostic prediction with multi-information using transformer and graph neural network model","volume":"11","author":"Lian","year":"2022","journal-title":"Elife"},{"key":"B90","doi-asserted-by":"publisher","first-page":"4","DOI":"10.21037\/qims-21-919","article-title":"TransConver: transformer and convolution parallel network for developing automatic brain tumor segmentation in MRI images","volume":"12","author":"Liang","year":"2022","journal-title":"Quant. Imaging Med. Surg"},{"key":"B91","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2209.03430","article-title":"Foundations and recent trends in multimodal machine learning: principles, challenges, and open questions","author":"Liang","year":"2022","journal-title":"arXiv"},{"key":"B92","doi-asserted-by":"publisher","first-page":"1095","DOI":"10.1016\/j.ccell.2022.09.012","article-title":"Artificial intelligence for multimodal data integration in oncology","volume":"40","author":"Lipkova","year":"2022","journal-title":"Cancer Cell"},{"key":"B93","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal"},{"key":"B94","doi-asserted-by":"publisher","first-page":"2008","DOI":"10.3390\/ijms22042008","article-title":"Therapeutic advances in oncology","volume":"22","author":"Liu","year":"2021","journal-title":"Int. J. Mol. Sci"},{"key":"B95","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.irbm.2020.12.002","article-title":"A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data","volume":"43","author":"Liu","year":"2022","journal-title":"IRBM"},{"key":"B96","doi-asserted-by":"crossref","first-page":"1380","DOI":"10.1109\/TNNLS.2021.3105284","article-title":"Multitask learning for visual question answering","volume":"34","author":"Ma","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B97","first-page":"443","article-title":"\u201cNeuGraph: parallel deep neural network computation on large graphs,\u201d","volume-title":"USENIX Annual Technical Conference","author":"Ma","year":"2019"},{"key":"B98","first-page":"343","article-title":"\u201cBrain tumor classification with multimodal MR and pathology images,\u201d","volume-title":"Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part II 5","author":"Ma","year":"2020"},{"key":"B99","first-page":"1202","article-title":"\u201cA unified view on graph neural networks as graph signal denoising,\u201d","author":"Ma","year":"2021","journal-title":"Proceedings of the 30th ACM International Conference on Information"},{"key":"B100","doi-asserted-by":"crossref","DOI":"10.1017\/9781108924184","volume-title":"Deep Learning on Graphs","author":"Ma","year":"2021"},{"key":"B101","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/srep26094","article-title":"Deep patient: an unsupervised representation to predict the future of patients from the electronic health records","volume":"6","author":"Miotto","year":"2016","journal-title":"Sci. Rep"},{"key":"B102","doi-asserted-by":"publisher","first-page":"87","DOI":"10.3390\/genes10020087","article-title":"Machine learning and integrative analysis of biomedical big data","volume":"10","author":"Mirza","year":"2019","journal-title":"Genes"},{"key":"B103","first-page":"429","article-title":"\u201cMultimodal priors guided segmentation of liver lesions in MRI using mutual information based graph co-attention networks,\u201d","volume-title":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part IV 23","author":"Mo","year":"2020"},{"key":"B104","doi-asserted-by":"publisher","first-page":"1247261","DOI":"10.3389\/frai.2023.1247261","article-title":"Editorial: Ai applications for diagnosis of breast cancer","volume":"6","author":"Muhammad","year":"2023","journal-title":"Front. Artif. Intellig"},{"key":"B105","doi-asserted-by":"publisher","first-page":"1329737","DOI":"10.3389\/frai.2024.1329737","article-title":"Artificial neural network-assisted prediction of radiobiological indices in head and neck cancer","volume":"7","author":"Muhammad","year":"2024","journal-title":"Front. Artif. Intellig"},{"key":"B106","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.17794","article-title":"Vision through the veil: Differential privacy in federated learning for medical image classification","author":"Nampalle","year":"2023","journal-title":"arXiv"},{"key":"B107","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1093\/bioinformatics\/btaa921","article-title":"GraphDTA: predicting drug-target binding affinity with graph neural networks","volume":"37","author":"Nguyen","year":"2021","journal-title":"Bioinformatics"},{"key":"B108","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1109\/TETCI.2022.3222545","article-title":"Reinforcement learning on graphs: A survey","volume":"7","author":"Nie","year":"2023","journal-title":"IEEE Trans. Emerg. Topics Comp. Intellig"},{"key":"B109","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1109\/MSP.2022.3142719","article-title":"Robust explainability: a tutorial on gradient-based attribution methods for deep neural networks","volume":"39","author":"Nielsen","year":"2022","journal-title":"IEEE Signal Process. Mag"},{"key":"B110","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3637323","article-title":"ehealth data security and privacy: Perspectives from diverse stakeholders in malawi","volume":"8","author":"Orii","year":"2024","journal-title":"Proc. ACM on Human-Comp. Interact"},{"key":"B111","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1109\/TNNLS.2020.2979670","article-title":"A survey of the usages of deep learning for natural language processing","volume":"32","author":"Otter","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B112","first-page":"5516","article-title":"\u201cUnsupervised hyperbolic representation learning via message passing auto-encoders,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Park","year":"2021"},{"key":"B113","doi-asserted-by":"publisher","first-page":"1839","DOI":"10.3390\/biom12121839","article-title":"Deep-learning algorithm and concomitant biomarker identification for NSCLC prediction using multi-omics data integration","volume":"12","author":"Park","year":"2022","journal-title":"Biomolecules"},{"key":"B114","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1007\/978-3-030-60365-6_20","article-title":"\u201cHACT-net: a hierarchical cell-to-tissue graph neural network for histopathological image classification,\u201d","volume-title":"Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis: Second International Workshop, UNSURE 2020, and Third International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 8, 2020, Proceedings 2","author":"Pati","year":"2020"},{"key":"B115","doi-asserted-by":"publisher","first-page":"7346","DOI":"10.1038\/s41467-022-33407-5","article-title":"Federated learning enables big data for rare cancer boundary detection","volume":"13","author":"Pati","year":"2022","journal-title":"Nat. Commun"},{"key":"B116","doi-asserted-by":"publisher","first-page":"137856","DOI":"10.1109\/ACCESS.2020.3012603","article-title":"Development and application of matrix variate restricted boltzmann machine","volume":"8","author":"Qi","year":"2020","journal-title":"IEEE Access"},{"key":"B117","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1515\/dx-2018-0036","article-title":"Electronic health records, communication, and data sharing: challenges and opportunities for improving the diagnostic process","volume":"6","author":"Quinn","year":"2019","journal-title":"Diagnosis"},{"key":"B118","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"B119","unstructured":"Radford\n              A.\n            \n            \n              Narasimhan\n              K.\n            \n            \n              Salimans\n              T.\n            \n            \n              Sutskever\n              I.\n            \n          \n          Improving Language Understanding by Generative Pre-Training\n          \n          2018"},{"key":"B120","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"B121","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2301.02242","article-title":"Graph contrastive learning for multi-omics data","author":"Rajadhyaksha","year":"2023","journal-title":"arXiv"},{"key":"B122","doi-asserted-by":"publisher","first-page":"102393","DOI":"10.1016\/j.isci.2021.102393","article-title":"Imputing single-cell RNA-seq data by combining graph convolution and autoencoder neural networks","volume":"24","author":"Rao","year":"2021","journal-title":"Iscience"},{"key":"B123","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-021-00455-y","article-title":"Med-BERT: Pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction","volume":"4","author":"Rasmy","year":"2021","journal-title":"NPJ Digital Med"},{"key":"B124","volume-title":"Explainability Methods for Transformer-based Artificial Neural Networks: a Comparative Analysis","author":"Remmer","year":"2022"},{"key":"B125","first-page":"10684","article-title":"\u201cHigh-resolution image synthesis with latent diffusion models,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Rombach","year":"2022"},{"key":"B126","doi-asserted-by":"publisher","first-page":"333","DOI":"10.3322\/caac.21713","article-title":"Molecular imaging in oncology: current impact and future directions","volume":"72","author":"Rowe","year":"2022","journal-title":"CA Cancer J. Clin"},{"key":"B127","first-page":"3472","article-title":"\u201cMOOMIN: deep molecular omics network for anti-cancer drug combination therapy,\u201d","author":"Rozemberczki","year":"2022","journal-title":"Proceedings of the 31st ACM International Conference on Information"},{"key":"B128","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-08637-3_3","article-title":"Secure multi-party computation based privacy preserving data analysis in healthcare iot systems","author":"\u015eahinba\u015f","year":"2021","journal-title":"arXiv"},{"key":"B129","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1038\/s42256-023-00618-4","article-title":"Biological research and self-driving labs in deep space supported by artificial intelligence","volume":"5","author":"Sanders","year":"2023","journal-title":"Nat. Mach. Intellig"},{"key":"B130","article-title":"Dynamic graph representation learning via self-attention networks","author":"Sankar","year":"2018","journal-title":"arXiv"},{"key":"B131","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1007\/978-3-030-70650-0_2","article-title":"\u201cExploring graph-based neural networks for automatic brain tumor segmentation,\u201d","volume-title":"From Data to Models and Back: 9th International Symposium, DataMod 2020, Virtual Event, October 20, 2020, Revised Selected Papers 9","author":"Saueressig","year":"2021"},{"key":"B132","article-title":"BLOOM: A 176B-parameter open-access multilingual language model","author":"Scao","year":"2022","journal-title":"arXiv"},{"key":"B133","doi-asserted-by":"publisher","first-page":"788740","DOI":"10.3389\/fonc.2021.788740","article-title":"Multimodal deep learning for prognosis prediction in renal cancer","volume":"11","author":"Schulz","year":"2021","journal-title":"Front. Oncol"},{"key":"B134","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/825","article-title":"Pre-training of graph augmented transformers for medication recommendation","author":"Shang","year":"2019","journal-title":"arXiv"},{"key":"B135","article-title":"\u201cTransMIL: transformer based correlated multiple instance learning for whole slide image classification,\u201d","author":"Shao","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B136","article-title":"\u201cGraph convolutional networks for cervical cell classification,\u201d","volume-title":"MICCAI 2019 Computational Pathology Workshop COMPAY","author":"Shi","year":"2019"},{"key":"B137","doi-asserted-by":"publisher","first-page":"1247195","DOI":"10.3389\/frai.2023.1247195","article-title":"Multimodal deep learning for liver cancer applications: a scoping review","volume":"6","author":"Siam","year":"2023","journal-title":"Front. Artif. Intellig"},{"key":"B138","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3322\/caac.21763","article-title":"Cancer statistics, 2023","volume":"73","author":"Siegel","year":"2023","journal-title":"CA Cancer J. Clin"},{"key":"B139","volume-title":"FLAVA: A Foundational Language and Vision Alignment Model","author":"Singh","year":"2022"},{"key":"B140","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1038\/s41416-021-01387-w","article-title":"Radiomics and radiogenomics in gliomas: a contemporary update","volume":"125","author":"Singh","year":"2021","journal-title":"Br. J. Cancer"},{"key":"B141","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3543848","article-title":"Multimodal classification: current landscape, taxonomy and future directions","volume":"55","author":"Sleeman","year":"2022","journal-title":"ACM Comp. Surv"},{"key":"B142","doi-asserted-by":"publisher","first-page":"979","DOI":"10.1007\/s11548-021-02391-4","article-title":"Multiview multimodal network for breast cancer diagnosis in contrast-enhanced spectral mammography images","volume":"16","author":"Song","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg"},{"key":"B143","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-24172-y","article-title":"scGCN is a graph convolutional networks algorithm for knowledge transfer in single cell omics","author":"Song","year":"2021","journal-title":"Nat. Commun"},{"key":"B144","doi-asserted-by":"publisher","first-page":"i919","DOI":"10.1093\/bioinformatics\/btaa843","article-title":"SCIM: universal single-cell matching with unpaired feature sets","volume":"36","author":"Stark","year":"2020","journal-title":"Bioinformatics"},{"key":"B145","first-page":"843","article-title":"\u201cRevisiting unreasonable effectiveness of data in deep learning era,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Sun","year":"2017"},{"key":"B146","doi-asserted-by":"publisher","first-page":"4764","DOI":"10.3390\/s21144764","article-title":"Multi-Modal Adaptive Fusion Transformer Network for the estimation of depression level","volume":"21","author":"Sun","year":"2021","journal-title":"Sensors"},{"key":"B147","first-page":"99","article-title":"\u201cHuman-in-the-loop interaction for continuously improving generative model in conversational agent for behavioral intervention,\u201d","volume-title":"Companion Proceedings of the 28th International Conference on Intelligent User Interfaces","author":"Sun","year":"2023"},{"key":"B148","doi-asserted-by":"publisher","first-page":"1796","DOI":"10.3390\/cancers13081796","article-title":"Multi-view data integration methods for radiotherapy structure name standardization","volume":"13","author":"Syed","year":"2021","journal-title":"Cancers"},{"key":"B149","doi-asserted-by":"publisher","first-page":"1285037","DOI":"10.3389\/frai.2024.1285037","article-title":"Machine learning-based classifiers to predict metastasis in colorectal cancer patients","volume":"7","author":"Talebi","year":"2024","journal-title":"Front. Artif. Intellig"},{"key":"B150","first-page":"3458","article-title":"\u201cMMT: multi-way multi-modal transformer for multimodal learning,\u201d","volume-title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22","author":"Tang","year":"2022"},{"key":"B151","doi-asserted-by":"publisher","DOI":"10.1158\/1538-7445.AM2020-LB-242","article-title":"Abstract lb-242: Proteomic data commons: a resource for proteogenomic analysis","author":"Thangudu","year":"2020","journal-title":"Cancer Res"},{"key":"B152","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1145\/2812802","article-title":"YFCC100M: The new data in multimedia research","volume":"59","author":"Thomee","year":"2016","journal-title":"Commun. ACM"},{"key":"B153","doi-asserted-by":"publisher","first-page":"4164","DOI":"10.1002\/mp.14327","article-title":"Graph-convolutional-network-based interactive prostate segmentation in MR images","volume":"47","author":"Tian","year":"2020","journal-title":"Med. Phys"},{"key":"B154","doi-asserted-by":"publisher","first-page":"47563","DOI":"10.1109\/ACCESS.2023.3275126","article-title":"Radiopathomics: Multimodal learning in non-small cell lung cancer for adaptive radiotherapy","volume":"11","author":"Tortora","year":"2023","journal-title":"IEEE Access"},{"key":"B155","doi-asserted-by":"publisher","first-page":"1634","DOI":"10.3390\/s24051634","article-title":"Building flexible, scalable, and machine learning-ready multimodal oncology datasets","volume":"24","author":"Tripathi","year":"","journal-title":"Sensors"},{"key":"B156","article-title":"Honeybee: a scalable modular framework for creating multimodal oncology datasets with foundational embedding models","author":"Tripathi","year":"","journal-title":"arXiv"},{"key":"B157","doi-asserted-by":"publisher","first-page":"4905","DOI":"10.1158\/1538-7445.AM2024-4905","article-title":"Multimodal transformer model improves survival prediction in lung cancer compared to unimodal approaches","volume":"84","author":"Tripathi","year":"","journal-title":"Cancer Res"},{"key":"B158","doi-asserted-by":"publisher","first-page":"101062","DOI":"10.1016\/j.imu.2022.101062","article-title":"RadGenNets: Deep learning-based radiogenomics model for gene mutation prediction in lung cancer","volume":"33","author":"Tripathi","year":"2022","journal-title":"Inform. Med. Unlocked"},{"key":"B159","doi-asserted-by":"publisher","first-page":"4610","DOI":"10.1002\/9781119850830.ch3","article-title":"RAN-GNNs: breaking the capacity limits of graph neural networks","volume":"34","author":"Valsesia","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B160","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.3390\/cancers16061100","article-title":"Machine learning meets cancer","volume":"16","author":"Varlamova","year":"2024","journal-title":"Cancers"},{"key":"B161","first-page":"30","article-title":"\u201cAttention is all you need,\u201d","volume-title":"Adv. Neural Inf. Process. Syst","author":"Vaswani","year":"2017"},{"key":"B162","article-title":"Graph attention networks","author":"Veli\u010dkovi\u0107","year":"2017","journal-title":"arXiv"},{"key":"B163","first-page":"12225","article-title":"PGM-explainer: probabilistic graphical model explanations for graph neural networks","volume":"33","author":"Vu","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B164","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10462-022-10321-2","article-title":"A survey of graph neural networks in various learning paradigms: methods, applications, and challenges","volume":"56","author":"Waikhom","year":"2022","journal-title":"Artif. Intellig. Rev"},{"key":"B165","doi-asserted-by":"publisher","first-page":"46335","DOI":"10.1109\/ACCESS.2020.2974101","article-title":"Feature extraction and analysis of natural language processing for deep learning english language","volume":"8","author":"Wang","year":"","journal-title":"IEEE Access"},{"key":"B166","first-page":"239","article-title":"\u201cWeakly supervised prostate TMA classification via graph convolutional networks,\u201d","volume-title":"2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)","author":"Wang","year":""},{"key":"B167","doi-asserted-by":"publisher","first-page":"1882","DOI":"10.1038\/s41467-021-22197-x","article-title":"scGNN is a novel graph neural network framework for single-cell RNA-Seq analyses","volume":"12","author":"Wang","year":"2021","journal-title":"Nat. Commun"},{"key":"B168","doi-asserted-by":"publisher","first-page":"e1005324","DOI":"10.1371\/journal.pcbi.1005324","article-title":"Accurate de novo prediction of protein contact map by ultra-deep learning model","volume":"13","author":"Wang","year":"2017","journal-title":"PLoS Comput. Biol"},{"key":"B169","doi-asserted-by":"publisher","first-page":"3445","DOI":"10.1038\/s41467-021-23774-w","article-title":"MOGONET integrates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification","volume":"12","author":"Wang","year":"2021","journal-title":"Nat. Commun"},{"key":"B170","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1038\/s41698-022-00285-5","article-title":"Cell graph neural networks enable the precise prediction of patient survival in gastric cancer","volume":"6","author":"Wang","year":"2022","journal-title":"NPJ Prec. Oncol"},{"key":"B171","doi-asserted-by":"publisher","first-page":"100255","DOI":"10.1016\/j.labinv.2023.100255","article-title":"Revolutionizing digital pathology with the power of generative artificial intelligence and foundation models","volume":"2023","author":"Waqas","year":"2023","journal-title":"Lab. Investigat"},{"key":"B172","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1007\/978-3-030-71676-9_13","article-title":"\u201cBrain tumor segmentation and surveillance with deep artificial neural networks,\u201d","volume-title":"Deep Learning for Biomedical Data Analysis","author":"Waqas","year":"2021"},{"key":"B173","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1038\/s44172-022-00043-2","article-title":"Exploring robust architectures for deep artificial neural networks","volume":"1","author":"Waqas","year":"2022","journal-title":"Commun. Eng"},{"key":"B174","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.08226","article-title":"SeNMo: a self-normalizing deep learning model for enhanced multi-omics data analysis in oncology","author":"Waqas","year":"","journal-title":"arXiv preprint"},{"key":"B175","doi-asserted-by":"publisher","DOI":"10.6004\/jnccn.2023.7137","article-title":"Embedding-based multimodal learning on pan-squamous cell carcinomas for improved survival outcomes","author":"Waqas","year":"","journal-title":"arXiv"},{"key":"B176","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01374","article-title":"\u201cCombating noisy labels by agreement: A joint training method with co-regularization,\u201d","author":"Wei","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B177","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1145\/3343031.3351034","article-title":"\u201cMMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video,\u201d","volume-title":"Proceedings of the 27th ACM International Conference on Multimedia","author":"Wei","year":"2019"},{"key":"B178","doi-asserted-by":"crossref","first-page":"4153","DOI":"10.1145\/3534678.3539213","article-title":"\u201cGraph neural networks for multimodal single-cell data integration,\u201d","volume-title":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Wen","year":"2022"},{"key":"B179","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B180","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1109\/TITS.2020.3013234","article-title":"Multimodal end-to-end autonomous driving","volume":"23","author":"Xiao","year":"2020","journal-title":"IEEE Trans. Intellig. Transp. Syst"},{"key":"B181","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-87199-4_16","article-title":"\u201cCotr: Efficiently bridging cnn and transformer for 3d medical image segmentation,\u201d","volume-title":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part III 24","author":"Xie","year":"2021"},{"key":"B182","doi-asserted-by":"publisher","first-page":"12113","DOI":"10.1109\/TPAMI.2023.3275156","article-title":"Multimodal learning with transformers: a survey","volume":"45","author":"Xu","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intellig"},{"key":"B183","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1093\/bioinformatics\/btab706","article-title":"SMILE: mutual information learning for integration of single-cell omics data","volume":"38","author":"Xu","year":"2022","journal-title":"Bioinformatics"},{"key":"B184","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-20249-2","article-title":"Multi-domain translation between single-cell imaging and sequencing data using autoencoders","author":"Yang","year":"2021","journal-title":"Nat. Commun"},{"key":"B185","first-page":"441","article-title":"\u201cHTML: hierarchical transformer-based multi-task learning for volatility prediction,\u201d","volume-title":"Proceedings of The Web Conference 2020, WWW '20","author":"Yang","year":"2020"},{"key":"B186","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3450352","article-title":"HGAT: Heterogeneous graph attention networks for semi-supervised short text classification","volume":"39","author":"Yang","year":"2021","journal-title":"ACM Trans. Inform. Syst"},{"key":"B187","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1111\/exd.13777","article-title":"Multimodal skin lesion classification using deep learning","volume":"27","author":"Yap","year":"2018","journal-title":"Exp. Dermatol"},{"key":"B188","doi-asserted-by":"publisher","first-page":"bbab340","DOI":"10.1093\/bib\/bbab340","article-title":"Graph representation learning in bioinformatics: trends, methods and applications","volume":"23","author":"Yi","year":"2022","journal-title":"Brief. Bioinform"},{"key":"B189","article-title":"\u201cGNNExplainer: generating explanations for graph neural networks,\u201d","author":"Ying","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B190","doi-asserted-by":"publisher","first-page":"450","DOI":"10.36548\/jtcsst.2023.4.006","article-title":"Genomic data analysis with variant of secure multi-party computation technique","volume":"5","author":"Yogi","year":"2024","journal-title":"J. Trends Comp. Sci. Smart Technol"},{"key":"B191","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-017-11817-6","article-title":"Predicting clinical outcomes from large scale cancer genomic profiles with deep survival models","volume":"7","author":"Yousefi","year":"2017","journal-title":"Sci. Rep"},{"key":"B192","article-title":"\u201cCoca: Contrastive captioners are image-text foundation models,\u201d","volume-title":"Transactions on Machine Learning Research","author":"Yu","year":"2022"},{"key":"B193","first-page":"12241","article-title":"\u201cOn explainability of graph neural networks via subgraph explorations,\u201d","volume-title":"International Conference on Machine Learning","author":"Yuan","year":"2021"},{"key":"B194","doi-asserted-by":"crossref","DOI":"10.1109\/BIBM49941.2020.9313569","article-title":"\u201cAccurately clustering single-cell RNA-seq data by capturing structural relations between cells through graph convolutional network,\u201d","volume-title":"2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)","author":"Zeng","year":"2020"},{"key":"B195","article-title":"Trustworthy graph neural networks: Aspects, methods and trends","author":"Zhang","year":"2022","journal-title":"arXiv"},{"key":"B196","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.1109\/TNNLS.2020.2990746","article-title":"Learning adversarial transformer for symbolic music generation","volume":"34","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B197","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2020.102439","article-title":"Improved breast cancer classification through combining graph convolutional network and convolutional neural network","author":"Zhang","year":"2021","journal-title":"Inform. Proc. Manage"},{"key":"B198","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1186\/s13059-022-02706-x","article-title":"scDART: integrating unmatched scRNA-seq and scATAC-seq data and learning cross-modality relationship simultaneously","volume":"23","author":"Zhang","year":"2022","journal-title":"Genome Biol"},{"key":"B199","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1016\/j.neucom.2021.10.039","article-title":"Hierarchical multimodal transformer to summarize videos","volume":"468","author":"Zhao","year":"2022","journal-title":"Neurocomputing"},{"key":"B200","doi-asserted-by":"publisher","DOI":"10.1145\/3649447","author":"Zhao","year":"2024","journal-title":"Deep Multimodal Data Fusion"},{"key":"B201","doi-asserted-by":"publisher","first-page":"4055","DOI":"10.1109\/TVCG.2023.3247169","article-title":"Accurate registration of cross-modality geometry via consistent clustering","volume":"30","author":"Zhao","year":"2023","journal-title":"IEEE Trans. Visualizat. Comp. Graph"},{"key":"B202","doi-asserted-by":"crossref","first-page":"6057","DOI":"10.1109\/WACV56688.2023.00601","article-title":"\u201cAnticipative feature fusion transformer for multi-modal action anticipation,\u201d","volume-title":"2023 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV)","author":"Zhong","year":"2023"},{"key":"B203","volume-title":"DFTR: Depth-supervised Fusion Transformer for Salient Object Detection","author":"Zhu","year":"2022"},{"key":"B204","first-page":"1218","article-title":"\u201cA robustly optimized BERT pre-training approach with post-training,\u201d","volume-title":"Proceedings of the 20th Chinese National Conference on Computational Linguistics","author":"Zhuang","year":"2021"},{"key":"B205","doi-asserted-by":"publisher","first-page":"i457","DOI":"10.1093\/bioinformatics\/bty294","article-title":"Modeling polypharmacy side effects with graph convolutional networks","volume":"34","author":"Zitnik","year":"2018","journal-title":"Bioinformatics"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1408843\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T14:19:19Z","timestamp":1721917159000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1408843\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,25]]},"references-count":205,"alternative-id":["10.3389\/frai.2024.1408843"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1408843","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,25]]},"article-number":"1408843"}}