{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T10:44:41Z","timestamp":1776077081150,"version":"3.50.1"},"reference-count":97,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000009","name":"Foundation for the National Institutes of Health","doi-asserted-by":"publisher","award":["R37- CA214955"],"award-info":[{"award-number":["R37- CA214955"]}],"id":[{"id":"10.13039\/100000009","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000009","name":"Foundation for the National Institutes of Health","doi-asserted-by":"publisher","award":["T32GM070449"],"award-info":[{"award-number":["T32GM070449"]}],"id":[{"id":"10.13039\/100000009","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008637","name":"Center for Strategic Scientific Initiatives, National Cancer Institute","doi-asserted-by":"publisher","award":["CCSG P30 CA04659"],"award-info":[{"award-number":["CCSG P30 CA04659"]}],"id":[{"id":"10.13039\/100008637","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning (ML) applications in medical artificial intelligence (AI) systems have shifted from traditional and statistical methods to increasing application of deep learning models. This survey navigates the current landscape of multimodal ML, focusing on its profound impact on medical image analysis and clinical decision support systems. Emphasizing challenges and innovations in addressing multimodal representation, fusion, translation, alignment, and co-learning, the paper explores the transformative potential of multimodal models for clinical predictions. It also highlights the need for principled assessments and practical implementation of such models, bringing attention to the dynamics between decision support systems and healthcare providers and personnel. Despite advancements, challenges such as data biases and the scarcity of \u201cbig data\u201d in many biomedical domains persist. We conclude with a discussion on principled innovation and collaborative efforts to further the mission of seamless integration of multimodal ML models into biomedical practice.\n<\/jats:p>","DOI":"10.1007\/s11263-024-02032-8","type":"journal-article","created":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T13:01:48Z","timestamp":1713877308000},"page":"3753-3769","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects"],"prefix":"10.1007","volume":"132","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6694-2701","authenticated-orcid":false,"given":"Elisa","family":"Warner","sequence":"first","affiliation":[]},{"given":"Joonsang","family":"Lee","sequence":"additional","affiliation":[]},{"given":"William","family":"Hsu","sequence":"additional","affiliation":[]},{"given":"Tanveer","family":"Syeda-Mahmood","sequence":"additional","affiliation":[]},{"given":"Charles E.","family":"Kahn","sequence":"additional","affiliation":[]},{"given":"Olivier","family":"Gevaert","sequence":"additional","affiliation":[]},{"given":"Arvind","family":"Rao","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,4,23]]},"reference":[{"key":"2032_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104418","volume":"135","author":"M Abdar","year":"2021","unstructured":"Abdar, M., Samami, M., Mahmoodabad, S. D., Doan, T., Mazoure, B., Hashemifesharaki, R., Liu, L., Khosravi, A., Acharya, U. R., Makarenkov, V., & Nahavandi, S. (2021). Uncertainty quantification in skin cancer classification using three-way decision-based Bayesian deep learning. Computers in Biology and Medicine, 135, 104418. https:\/\/doi.org\/10.1016\/j.compbiomed.2021.104418","journal-title":"Computers in Biology and Medicine"},{"issue":"24","key":"2032_CR2","doi-asserted-by":"publisher","first-page":"2285","DOI":"10.1056\/nejmp1907407","volume":"381","author":"AS Adamson","year":"2019","unstructured":"Adamson, A. S., & Welch, H. G. (2019). Machine learning and the cancer-diagnosis problem\u2014No gold standard. New England Journal of Medicine, 381(24), 2285\u20132287. https:\/\/doi.org\/10.1056\/nejmp1907407","journal-title":"New England Journal of Medicine"},{"key":"2032_CR3","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-017-0430-8","author":"JS Ancker","year":"2017","unstructured":"Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making. https:\/\/doi.org\/10.1186\/s12911-017-0430-8","journal-title":"BMC Medical Informatics and Decision Making"},{"key":"2032_CR4","doi-asserted-by":"publisher","unstructured":"Azcona, E. A., Besson, P., Wu, Y., Punjabi, A., Martersteck, A., Dravid, A., Parrish, T. B., Bandt, S. K., & Katsaggelos, A. K. (2020). Interpretation of brain morphology in association to Alzheimer\u2019s disease dementia classification using graph convolutional networks on triangulated meshes. In Shape in medical imaging (pp. 95\u2013107). Springer. https:\/\/doi.org\/10.1007\/978-3-030-61056-2_8","DOI":"10.1007\/978-3-030-61056-2_8"},{"key":"2032_CR5","unstructured":"Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In Y. Bengio, Y. LeCun (Eds.), 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference track proceedings. arxiv:1409.0473."},{"issue":"2","key":"2032_CR6","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/tpami.2018.2798607","volume":"41","author":"T Baltrusaitis","year":"2019","unstructured":"Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423\u2013443. https:\/\/doi.org\/10.1109\/tpami.2018.2798607","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2032_CR7","unstructured":"Benchetrit, Y., Banville, H., & King, J.-R. (2023). Brain decoding: Toward real-time reconstruction of visual perception. arXiv:2310.19812"},{"issue":"8","key":"2032_CR8","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/tpami.2013.50","volume":"35","author":"Y Bengio","year":"2013","unstructured":"Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798\u20131828. https:\/\/doi.org\/10.1109\/tpami.2013.50","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2032_CR9","unstructured":"Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y., Manassra, W., Dhariwal, P., Chu, C., Jiao, Y., & Ramesh, A. (2023). Improving image generation with better captions. https:\/\/cdn.openai.com\/papers\/dall-e-3.pdf."},{"key":"2032_CR10","doi-asserted-by":"publisher","unstructured":"Bhalodia, R., Hatamizadeh, A., Tam, L., Xu, Z., Wang, X., Turkbey, E., & Xu, D. (2021). Improving pneumonia localization via cross-attention on medical images and\u00a0reports. In Medical image computing and computer assisted intervention\u2014MICCAI 2021 (pp. 571\u2013581). Springer. https:\/\/doi.org\/10.1007\/978-3-030-87196-3_53","DOI":"10.1007\/978-3-030-87196-3_53"},{"key":"2032_CR11","doi-asserted-by":"publisher","unstructured":"B\u00f4ne, A., Vernhet, P., Colliot, O., & Durrleman, S. (2020). Learning joint shape and appearance representations with metamorphic auto-encoders. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 202\u2013211). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59710-8_20","DOI":"10.1007\/978-3-030-59710-8_20"},{"key":"2032_CR12","doi-asserted-by":"publisher","unstructured":"Bui, T. D., Nguyen, M., Le, N., & Luu, K. (2020). Flow-based deformation guidance for unpaired multi-contrast MRI image-to-image translation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 728\u2013737). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_70","DOI":"10.1007\/978-3-030-59713-9_70"},{"issue":"6","key":"2032_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3272127.3275046","volume":"37","author":"K Cao","year":"2018","unstructured":"Cao, K., Liao, J., & Yuan, L. (2018). CariGANs. ACM Transactions on Graphics, 37(6), 1\u201314. https:\/\/doi.org\/10.1145\/3272127.3275046","journal-title":"ACM Transactions on Graphics"},{"key":"2032_CR14","doi-asserted-by":"publisher","unstructured":"Carbonell, E.L., Shen, Y., Yang, X., & Ke, J. (2023). COVID-19 pneumonia classification with\u00a0transformer from\u00a0incomplete modalities. In Lecture notes in computer science (pp. 379\u2013388). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43904-9_37","DOI":"10.1007\/978-3-031-43904-9_37"},{"key":"2032_CR15","doi-asserted-by":"publisher","unstructured":"Chauhan, G., Liao, R., Wells, W., Andreas, J., Wang, X., Berkowitz, S., Horng, S., Szolovits, P., & Golland, P. (2020). Joint modeling of chest radiographs and radiology reports for pulmonary edema assessment. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 529\u2013539). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_51","DOI":"10.1007\/978-3-030-59713-9_51"},{"key":"2032_CR16","doi-asserted-by":"publisher","unstructured":"Chen, Z., Guo, Q., Yeung, L. K. T., Chan, D. T. M., Lei, Z., Liu, H., & Wang, J. (2023). Surgical video captioning with\u00a0mutual-modal concept alignment. In Lecture notes in computer science (pp. 24\u201334). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43996-4_3","DOI":"10.1007\/978-3-031-43996-4_3"},{"issue":"6","key":"2032_CR17","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1038\/s41551-021-00751-8","volume":"5","author":"RJ Chen","year":"2021","unstructured":"Chen, R. J., Lu, M. Y., Chen, T. Y., Williamson, D. F. K., & Mahmood, F. (2021). Synthetic data in machine learning for medicine and healthcare. Nature Biomedical Engineering, 5(6), 493\u2013497. https:\/\/doi.org\/10.1038\/s41551-021-00751-8","journal-title":"Nature Biomedical Engineering"},{"key":"2032_CR18","doi-asserted-by":"publisher","DOI":"10.1109\/tmi.2020.3021387","author":"RJ Chen","year":"2020","unstructured":"Chen, R. J., Lu, M. Y., Wang, J., Williamson, D. F. K., Rodig, S. J., Lindeman, N. I., & Mahmood, F. (2020). Pathomic fusion: An integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis. IEEE Transactions on Medical Imaging. https:\/\/doi.org\/10.1109\/tmi.2020.3021387","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"2032_CR19","doi-asserted-by":"publisher","unstructured":"Chiou, E., Giganti, F., Punwani, S., Kokkinos, I., & Panagiotaki, E. (2020). Harnessing uncertainty in domain adaptation for MRI prostate lesion segmentation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 510\u2013520). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59710-8_50","DOI":"10.1007\/978-3-030-59710-8_50"},{"key":"2032_CR20","unstructured":"Choi, E., Biswal, S., Malin, B., Duke, J., Stewart, W. F., & Sun, J. (2017). Generating multi-label discrete patient records using generative adversarial networks. In F. Doshi-Velez, J. Fackler, D. Kale, R. Ranganath, B. Wallace, J. Wiens (Eds.), Proceedings of the 2nd machine learning for healthcare conference. Proceedings of machine learning research (Vol. 68, pp. 286\u2013305). PMLR. https:\/\/proceedings.mlr.press\/v68\/choi17a.html"},{"key":"2032_CR21","doi-asserted-by":"publisher","unstructured":"Cui, C., Liu, H., Liu, Q., Deng, R., Asad, Z., Wang, Y., Zhao, S., Yang, H., Landman, B. A., & Huo, Y. (2022). Survival prediction of\u00a0brain cancer with\u00a0incomplete radiology, pathology, genomic, and\u00a0demographic data. In Lecture notes in computer science (pp. 626\u2013635). Springer. https:\/\/doi.org\/10.1007\/978-3-031-16443-9_60","DOI":"10.1007\/978-3-031-16443-9_60"},{"key":"2032_CR22","doi-asserted-by":"publisher","unstructured":"Daza, L., Castillo, A., Escobar, M., Valencia, S., Pinz\u00f3n, B., & Arbel\u00e1ez, P. (2020). LUCAS: LUng CAncer screening with multimodal biomarkers. In Multimodal learning for clinical decision support and clinical image-based procedures (pp. 115\u2013124). Springer. https:\/\/doi.org\/10.1007\/978-3-030-60946-7_12","DOI":"10.1007\/978-3-030-60946-7_12"},{"key":"2032_CR23","unstructured":"Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. In North American chapter of the association for computational linguistics. https:\/\/aclanthology.org\/N19-1423.pdf"},{"key":"2032_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118016","volume":"207","author":"D Dong","year":"2022","unstructured":"Dong, D., Fu, G., Li, J., Pei, Y., & Chen, Y. (2022). An unsupervised domain adaptation brain CT segmentation method across image modalities and diseases. Expert Systems with Applications, 207, 118016. https:\/\/doi.org\/10.1016\/j.eswa.2022.118016","journal-title":"Expert Systems with Applications"},{"key":"2032_CR25","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth $$16\\times 16$$ words: Transformers for image recognition at scale. In International conference on learning representations. arxiv:2010.11929"},{"issue":"4","key":"2032_CR26","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1056\/nejm199801223380406","volume":"338","author":"RS Evans","year":"1998","unstructured":"Evans, R. S., Pestotnik, S. L., Classen, D. C., Clemmer, T. P., Weaver, L. K., Orme, J. F., Lloyd, J. F., & Burke, J. P. (1998). A computer-assisted management program for antibiotics and other antiinfective agents. New England Journal of Medicine, 338(4), 232\u2013238. https:\/\/doi.org\/10.1056\/nejm199801223380406","journal-title":"New England Journal of Medicine"},{"key":"2032_CR27","doi-asserted-by":"publisher","unstructured":"Guo, P., Wang, P., Zhou, J., Patel, V.M., & Jiang, S. (2020). Lesion mask-based simultaneous synthesis of anatomic and molecular MR images using a GAN. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 104\u2013113). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_11","DOI":"10.1007\/978-3-030-59713-9_11"},{"key":"2032_CR28","doi-asserted-by":"publisher","unstructured":"Habib, G., Kiryati, N., Sklair-Levy, M., Shalmon, A., Neiman, O. H., Weidenfeld, R. F., Yagil, Y., Konen, E., & Mayer, A. (2020). Automatic breast lesion classification by joint neural analysis of mammography and ultrasound. In Multimodal learning for clinical decision support and clinical image-based procedures (pp. 125\u2013135). Springer. https:\/\/doi.org\/10.1007\/978-3-030-60946-7_13","DOI":"10.1007\/978-3-030-60946-7_13"},{"key":"2032_CR29","doi-asserted-by":"publisher","unstructured":"Hamghalam, M., Frangi, A.F., Lei, B., & Simpson, A. L. (2021). Modality completion via gaussian process prior variational autoencoders for multi-modal glioma segmentation. In: Medical image computing and computer assisted intervention\u2014MICCAI 2021 (pp. 442\u2013452). Springer. https:\/\/doi.org\/10.1007\/978-3-030-87234-2_42","DOI":"10.1007\/978-3-030-87234-2_42"},{"key":"2032_CR30","doi-asserted-by":"publisher","unstructured":"Hu, M., Maillard, M., Zhang, Y., Ciceri, T., Barbera, G. L., Bloch, I., & Gori, P. (2020). Knowledge distillation from multi-modal to mono-modal segmentation networks. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 772\u2013781). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59710-8_75","DOI":"10.1007\/978-3-030-59710-8_75"},{"key":"2032_CR31","doi-asserted-by":"publisher","unstructured":"Hu, S., Shen, Y., Wang, S., & Lei, B. (2020). Brain MR to PET synthesis via bidirectional generative adversarial network. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 698\u2013707). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_67","DOI":"10.1007\/978-3-030-59713-9_67"},{"key":"2032_CR32","doi-asserted-by":"publisher","unstructured":"Hu, M., Song, T., Gu, Y., Luo, X., Chen, J., Chen, Y., Zhang, Y., & Zhang, S. (2021). Fully test-time adaptation for image segmentation. In Medical image computing and computer assisted intervention\u2014MICCAI 2021 (pp. 251\u2013260). Springer. https:\/\/doi.org\/10.1007\/978-3-030-87199-4_24","DOI":"10.1007\/978-3-030-87199-4_24"},{"key":"2032_CR33","doi-asserted-by":"publisher","first-page":"2031","DOI":"10.1109\/tifs.2020.3047753","volume":"16","author":"Z Huang","year":"2021","unstructured":"Huang, Z., Chen, S., Zhang, J., & Shan, H. (2021). PFA-GAN: Progressive face aging with generative adversarial network. IEEE Transactions on Information Forensics and Security, 16, 2031\u20132045. https:\/\/doi.org\/10.1109\/tifs.2020.3047753","journal-title":"IEEE Transactions on Information Forensics and Security"},{"key":"2032_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102536","volume":"81","author":"M Jafari","year":"2022","unstructured":"Jafari, M., Francis, S., Garibaldi, J. M., & Chen, X. (2022). LMISA: A lightweight multi-modality image segmentation network via domain adaptation using gradient magnitude and shape constraint. Medical Image Analysis, 81, 102536. https:\/\/doi.org\/10.1016\/j.media.2022.102536","journal-title":"Medical Image Analysis"},{"key":"2032_CR35","doi-asserted-by":"publisher","unstructured":"Jiang, J., & Veeraraghavan, H. (2020). Unified cross-modality feature disentangler for unsupervised multi-domain MRI abdomen organs segmentation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 347\u2013358). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_34","DOI":"10.1007\/978-3-030-59713-9_34"},{"key":"2032_CR36","doi-asserted-by":"publisher","unstructured":"Khosravi, B., Rouzrokh, P., Kremers, H. M., Larson, D. R., Johnson, Q. J., Faghani, S., Kremers, W. K., Erickson, B. J., Sierra, R. J., Taunton, M. J., & Wyles, C. C. (2022). Patient-specific hip arthroplasty dislocation risk calculator: An explainable multimodal machine learning\u2013based approach. Radiology: Artificial Intelligence. https:\/\/doi.org\/10.1148\/ryai.220067","DOI":"10.1148\/ryai.220067"},{"key":"2032_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2019.106816","volume":"142","author":"Y Kwon","year":"2020","unstructured":"Kwon, Y., Won, J.-H., Kim, B. J., & Paik, M. C. (2020). Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis, 142, 106816. https:\/\/doi.org\/10.1016\/j.csda.2019.106816","journal-title":"Computational Statistics & Data Analysis"},{"key":"2032_CR38","doi-asserted-by":"crossref","unstructured":"Lambert, J., Sener, O., & Savarese, S. (2018). Deep learning under privileged information using heteroscedastic dropout. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). https:\/\/openaccess.thecvf.com\/content_cvpr_2018\/papers\/Lambert_Deep_Learning_Under_CVPR_2018_paper.pdf","DOI":"10.1109\/CVPR.2018.00926"},{"key":"2032_CR39","doi-asserted-by":"publisher","unstructured":"Leroy, A., Cafaro, A., Gessain, G., Champagnac, A., Gr\u00e9goire, V., Deutsch, E., Lepetit, V., & Paragios, N. (2023). StructuRegNet: Structure-guided multimodal 2D-3D registration. In Lecture notes in computer science (pp. 771\u2013780). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43999-5_73","DOI":"10.1007\/978-3-031-43999-5_73"},{"key":"2032_CR40","doi-asserted-by":"publisher","unstructured":"Li, T. Z., Still, J. M., Xu, K., Lee, H. H., Cai, L. Y., Krishnan, A. R., Gao, R., Khan, M. S., Antic, S., Kammer, M., Sandler, K. L., Maldonado, F., Landman, B. A., & Lasko, T. A. (2023) Longitudinal multimodal transformer integrating imaging and\u00a0latent clinical signatures from\u00a0routine EHRs for\u00a0pulmonary nodule classification. In Lecture notes in computer science (pp. 649\u2013659). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43895-0_61","DOI":"10.1007\/978-3-031-43895-0_61"},{"key":"2032_CR41","doi-asserted-by":"publisher","unstructured":"Li, H., Yang, F., Xing, X., Zhao, Y., Zhang, J., Liu, Y., Han, M., Huang, J., Wang, L., & Yao, J. (2021). Multi-modal multi-instance learning using weakly correlated histopathological images and tabular clinical information. In Medical image computing and computer assisted intervention\u2014MICCAI 2021 (pp. 529\u2013539). Springer. https:\/\/doi.org\/10.1007\/978-3-030-87237-3_51","DOI":"10.1007\/978-3-030-87237-3_51"},{"key":"2032_CR42","doi-asserted-by":"publisher","unstructured":"Liao, W., Hu, K., Yang, M. Y., & Rosenhahn, B. (2022). Text to image generation with semantic-spatial aware GAN. In 2022 IEEE\/CVF conference on computer vision and pattern recognition (CVPR). IEEE. https:\/\/doi.org\/10.1109\/cvpr52688.2022.01765","DOI":"10.1109\/cvpr52688.2022.01765"},{"key":"2032_CR43","doi-asserted-by":"publisher","unstructured":"Liu, H., Michelini, P.N., & Zhu, D. (2018). Artsy-GAN: A style transfer system with improved quality, diversity and performance. In 2018 24th international conference on pattern recognition (ICPR). IEEE. https:\/\/doi.org\/10.1109\/icpr.2018.8546172","DOI":"10.1109\/icpr.2018.8546172"},{"key":"2032_CR44","doi-asserted-by":"publisher","unstructured":"Liu, Z., Wei, J., Li, R., & Zhou, J. (2023). SFusion: Self-attention based n-to-one multimodal fusion block. In Lecture notes in computer science (pp. 159\u2013169). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43895-0_15","DOI":"10.1007\/978-3-031-43895-0_15"},{"issue":"19","key":"2032_CR45","doi-asserted-by":"publisher","first-page":"1415","DOI":"10.1161\/circulationaha.122.060291","volume":"146","author":"SA Lubitz","year":"2022","unstructured":"Lubitz, S. A., Faranesh, A. Z., Selvaggi, C., Atlas, S. J., McManus, D. D., Singer, D. E., Pagoto, S., McConnell, M. V., Pantelopoulos, A., & Foulkes, A. S. (2022). Detection of atrial fibrillation in a large population using wearable devices: The Fitbit heart study. Circulation, 146(19), 1415\u20131424. https:\/\/doi.org\/10.1161\/circulationaha.122.060291","journal-title":"Circulation"},{"issue":"6","key":"2032_CR46","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","volume":"5","author":"MY Lu","year":"2021","unstructured":"Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021). Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6), 555\u2013570. https:\/\/doi.org\/10.1038\/s41551-020-00682-w","journal-title":"Nature Biomedical Engineering"},{"issue":"6","key":"2032_CR47","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","volume":"5","author":"MY Lu","year":"2021","unstructured":"Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021). Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6), 555\u2013570. https:\/\/doi.org\/10.1038\/s41551-020-00682-w","journal-title":"Nature Biomedical Engineering"},{"key":"2032_CR48","unstructured":"Metz, R. (2022). AI won an art contest, and artists are furious. Warner Bros. Discovery. https:\/\/www.cnn.com\/2022\/09\/03\/tech\/ai-art-fair-winner-controversy\/index.html"},{"key":"2032_CR49","doi-asserted-by":"publisher","unstructured":"Moradi, M., Syeda-Mahmood, T., & Hor, S. (2016). Tree-based transforms for privileged learning. In Machine learning in medical imaging (pp. 188\u2013195). Springer. https:\/\/doi.org\/10.1007\/978-3-319-47157-0_23","DOI":"10.1007\/978-3-319-47157-0_23"},{"key":"2032_CR50","doi-asserted-by":"publisher","unstructured":"Neubauer, T., Wimmer, M., Berg, A., Major, D., Lenis, D., Beyer, T., Saponjski, J., & B\u00fchler, K. (2020). Soft tissue sarcoma co-segmentation in combined MRI and PET\/CT data. In Multimodal learning for clinical decision support and clinical image-based procedures (pp. 97\u2013105). Springer. https:\/\/doi.org\/10.1007\/978-3-030-60946-7_10","DOI":"10.1007\/978-3-030-60946-7_10"},{"key":"2032_CR51","unstructured":"OpenAI: GPT-4 Technical Report. (2023). arXiv:2303.08774."},{"key":"2032_CR52","doi-asserted-by":"publisher","unstructured":"Oppenlaender, J. (2022). The creativity of text-to-image generation. In Proceedings of the 25th international academic mindtrek conference. ACM. https:\/\/doi.org\/10.1145\/3569219.3569352","DOI":"10.1145\/3569219.3569352"},{"key":"2032_CR53","doi-asserted-by":"crossref","unstructured":"Palsson, S., Agustsson, E., Timofte, R., & Van\u00a0Gool, L. (2018). Generative adversarial style transfer networks for face aging. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) workshops. https:\/\/openaccess.thecvf.com\/content_cvpr_2018_workshops\/papers\/w41\/Palsson_Generative_Adversarial_Style_CVPR_2018_paper.pdf","DOI":"10.1109\/CVPRW.2018.00282"},{"key":"2032_CR54","unstructured":"Parmar, N. J., Vaswani, A., Uszkoreit, J., Kaiser, L., Shazeer, N., Ku, A., & Tran, D. (2018). Image transformer. In International conference on machine learning (ICML). http:\/\/proceedings.mlr.press\/v80\/parmar18a.html"},{"key":"2032_CR55","doi-asserted-by":"publisher","DOI":"10.1109\/tmi.2023.3346285","author":"C Pei","year":"2023","unstructured":"Pei, C., Wu, F., Yang, M., Pan, L., Ding, W., Dong, J., Huang, L., & Zhuang, X. (2023). Multi-source domain adaptation for medical image segmentation. IEEE Transactions on Medical Imaging. https:\/\/doi.org\/10.1109\/tmi.2023.3346285","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"2032_CR56","doi-asserted-by":"publisher","unstructured":"Perino, A. C., Gummidipundi, S. E., Lee, J., Hedlin, H., Garcia, A., Ferris, T., Balasubramanian, V., Gardner, R.M., Cheung, L., Hung, G., Granger, C. B., Kowey, P., Rumsfeld, J. S., Russo, A. M., True\u00a0Hills, M., Talati, N., Nag, D., Tsay, D., Desai, S., Desai, M., Mahaffey, K. W., Turakhia, M. P., & Perez, M. V. (2021). Arrhythmias other than atrial fibrillation in those with an irregular pulse detected with a smartwatch: Findings from the Apple heart study. Circulation: Arrhythmia and Electrophysiology. https:\/\/doi.org\/10.1161\/circep.121.010063","DOI":"10.1161\/circep.121.010063"},{"issue":"4","key":"2032_CR57","doi-asserted-by":"publisher","first-page":"389","DOI":"10.3390\/electronics10040389","volume":"10","author":"E Piacentino","year":"2021","unstructured":"Piacentino, E., Guarner, A., & Angulo, C. (2021). Generating synthetic ECGs using GANs for anonymizing healthcare data. Electronics, 10(4), 389. https:\/\/doi.org\/10.3390\/electronics10040389","journal-title":"Electronics"},{"key":"2032_CR58","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In M. Meila & T. Zhang (Eds.), Proceedings of the 38th international conference on machine learning. Proceedings of machine learning research (Vol. 139, pp. 8748\u20138763). PMLR. https:\/\/proceedings.mlr.press\/v139\/radford21a.html"},{"key":"2032_CR59","unstructured":"Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv:2204.06125."},{"issue":"4","key":"2032_CR60","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1016\/s2589-7500(22)00029-2","volume":"4","author":"S Reddy","year":"2022","unstructured":"Reddy, S. (2022). Explainability and artificial intelligence in medicine. The Lancet Digital Health, 4(4), 214\u2013215. https:\/\/doi.org\/10.1016\/s2589-7500(22)00029-2","journal-title":"The Lancet Digital Health"},{"key":"2032_CR61","unstructured":"Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016). Generative adversarial text to image synthesis. In: M. F. Balcan & K. Q. Weinberger (Eds.), Proceedings of the 33rd international conference on machine learning. Proceedings of machine learning research (Vol. 48, pp. 1060\u20131069). PMLR. https:\/\/proceedings.mlr.press\/v48\/reed16.html"},{"key":"2032_CR62","doi-asserted-by":"publisher","unstructured":"Rudie, J. D., Calabrese, E., Saluja, R., Weiss, D., Colby, J. B., Cha, S., Hess, C. P., Rauschecker, A. M., Sugrue, L. P., & Villanueva-Meyer, J. E. (2022). Longitudinal assessment of posttreatment diffuse glioma tissue volumes with three-dimensional convolutional neural networks. Radiology: Artificial Intelligence. https:\/\/doi.org\/10.1148\/ryai.210243","DOI":"10.1148\/ryai.210243"},{"issue":"1","key":"2032_CR63","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1111\/nup.12112","volume":"17","author":"H Ruiter","year":"2015","unstructured":"Ruiter, H., Liaschenko, J., & Angus, J. (2015). Problems with the electronic health record. Nursing Philosophy, 17(1), 49\u201358. https:\/\/doi.org\/10.1111\/nup.12112","journal-title":"Nursing Philosophy"},{"issue":"3","key":"2032_CR64","doi-asserted-by":"publisher","first-page":"784","DOI":"10.1109\/jbhi.2020.3008601","volume":"25","author":"E Sabeti","year":"2021","unstructured":"Sabeti, E., Drews, J., Reamaroon, N., Warner, E., Sjoding, M. W., Gryak, J., & Najarian, K. (2021). Learning using partially available privileged information and label uncertainty: Application in detection of acute respiratory distress syndrome. IEEE Journal of Biomedical and Health Informatics, 25(3), 784\u2013796. https:\/\/doi.org\/10.1109\/jbhi.2020.3008601","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"2032_CR65","doi-asserted-by":"publisher","DOI":"10.1007\/s00138-020-01058-5","author":"TA Shaikh","year":"2020","unstructured":"Shaikh, T. A., Ali, R., & Beg, M. M. S. (2020). Transfer learning privileged information fuels CAD diagnosis of breast cancer. Machine Vision and Applications. https:\/\/doi.org\/10.1007\/s00138-020-01058-5","journal-title":"Machine Vision and Applications"},{"key":"2032_CR66","doi-asserted-by":"publisher","unstructured":"Shin, H.-C., , Ihsani, A., Xu, Z., Mandava, S., Sreenivas, S. T., Forster, C., & Cha, J. (2020). GANDALF: Generative adversarial networks with discriminator-adaptive loss fine-tuning for Alzheimer\u2019s disease diagnosis from MRI. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 688\u2013697). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_66","DOI":"10.1007\/978-3-030-59713-9_66"},{"key":"2032_CR67","doi-asserted-by":"publisher","DOI":"10.1056\/cat.20.0212","author":"C Silcox","year":"2020","unstructured":"Silcox, C., Dentzer, S., & Bates, D. W. (2020). AI-enabled clinical decision support software: A \u201ctrust and value checklist\u2019\u2019 for clinicians. NEJM Catalyst. https:\/\/doi.org\/10.1056\/cat.20.0212","journal-title":"NEJM Catalyst"},{"key":"2032_CR68","doi-asserted-by":"publisher","unstructured":"Sonsbeek, T., & Worring, M. (2020). Towards automated diagnosis with attentive multi-modal learning using electronic health records and chest X-rays. In Multimodal learning for clinical decision support and clinical image-based procedures (pp. 106\u2013114). Springer. https:\/\/doi.org\/10.1007\/978-3-030-60946-7_11","DOI":"10.1007\/978-3-030-60946-7_11"},{"issue":"84","key":"2032_CR69","first-page":"2949","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava, N., & Salakhutdinov, R. (2014). Multimodal learning with deep Boltzmann machines. Journal of Machine Learning Research, 15(84), 2949\u20132980.","journal-title":"Journal of Machine Learning Research"},{"key":"2032_CR70","doi-asserted-by":"crossref","unstructured":"Sung, M., Lee, J., Yi, S. S., Jeon, M., Kim, S., & Kang, J. (2021). Can language models be biomedical knowledge bases? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (pp. 4723\u20134734). Association for Computational Linguistics. ACL. arXiv:2109.07154. https:\/\/aclanthology.org\/2021.emnlp-main.388.pdf","DOI":"10.18653\/v1\/2021.emnlp-main.388"},{"key":"2032_CR71","doi-asserted-by":"publisher","unstructured":"Takagi, Y., & Nishimoto, S. (2023). High-resolution image reconstruction with latent diffusion models from human brain activity. In 2023 IEEE\/CVF conference on computer vision and pattern recognition (CVPR). IEEE. https:\/\/doi.org\/10.1109\/cvpr52729.2023.01389","DOI":"10.1109\/cvpr52729.2023.01389"},{"issue":"4","key":"2032_CR72","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2023.100729","volume":"4","author":"R Tinn","year":"2023","unstructured":"Tinn, R., Cheng, H., Gu, Y., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2023). Fine-tuning large neural language models for biomedical natural language processing. Patterns, 4(4), 100729. https:\/\/doi.org\/10.1016\/j.patter.2023.100729","journal-title":"Patterns"},{"key":"2032_CR73","doi-asserted-by":"publisher","unstructured":"Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019). Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th annual meeting of the association for computational linguistics. Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/p19-1656","DOI":"10.18653\/v1\/p19-1656"},{"issue":"12","key":"2032_CR74","doi-asserted-by":"publisher","first-page":"6690","DOI":"10.1002\/mp.12625","volume":"44","author":"H-H Tseng","year":"2017","unstructured":"Tseng, H.-H., Luo, Y., Cui, S., Chien, J.-T., Haken, R. K. T., & Naqa, I. E. (2017). Deep reinforcement learning for automated radiation adaptation in lung cancer. Medical Physics, 44(12), 6690\u20136705. https:\/\/doi.org\/10.1002\/mp.12625","journal-title":"Medical Physics"},{"key":"2032_CR75","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03828-1","author":"K Tunyasuvunakool","year":"2021","unstructured":"Tunyasuvunakool, K., Adler, J., Wu, Z., Green, T., Zielinski, M., \u017d\u00eddek, A., Bridgland, A., Cowie, A., Meyer, C., Laydon, A., Velankar, S., Kleywegt, G. J., Bateman, A., Evans, R., Pritzel, A., Figurnov, M., Ronneberger, O., Bates, R., Kohl, S. A. A., & Hassabis, D. (2021). Highly accurate protein structure prediction for the human proteome. Nature. https:\/\/doi.org\/10.1038\/s41586-021-03828-1","journal-title":"Nature"},{"issue":"5\u20136","key":"2032_CR76","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.neunet.2009.06.042","volume":"22","author":"V Vapnik","year":"2009","unstructured":"Vapnik, V., & Vashist, A. (2009). A new learning paradigm: Learning using privileged information. Neural Networks, 22(5\u20136), 544\u2013557. https:\/\/doi.org\/10.1016\/j.neunet.2009.06.042","journal-title":"Neural Networks"},{"key":"2032_CR77","doi-asserted-by":"publisher","unstructured":"Varsavsky, T., Orbes-Arteaga, M., Sudre, C. H., Graham, M. S., Nachev, P., & Cardoso, M. J. (2020). Test-time unsupervised domain adaptation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 428\u2013436). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59710-8_42","DOI":"10.1007\/978-3-030-59710-8_42"},{"key":"2032_CR78","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. CoRR arXiv:1706.03762"},{"key":"2032_CR79","doi-asserted-by":"publisher","unstructured":"Vivar, G., Mullakaeva, K., Zwergal, A., Navab, N., & Ahmadi, S.-A. (2020). Peri-diagnostic decision support through cost-efficient feature acquisition at test-time. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 572\u2013581). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_55","DOI":"10.1007\/978-3-030-59713-9_55"},{"key":"2032_CR80","doi-asserted-by":"publisher","unstructured":"Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., & Li, J. (2021). TransBTS: Multimodal brain tumor segmentation using transformer. In Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2021 (pp. 109\u2013119). Springer. https:\/\/doi.org\/10.1007\/978-3-030-87193-2_11","DOI":"10.1007\/978-3-030-87193-2_11"},{"key":"2032_CR81","doi-asserted-by":"publisher","unstructured":"Wang, H., Ma, C., Zhang, J., Zhang, Y., Avery, J., Hull, L., & Carneiro, G. (2023). Learnable cross-modal knowledge distillation for\u00a0multi-modal learning with\u00a0missing modality. In Lecture notes in computer science (pp. 216\u2013226). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43901-8_21","DOI":"10.1007\/978-3-031-43901-8_21"},{"key":"2032_CR82","doi-asserted-by":"publisher","unstructured":"Warner, E., Al-Turkestani, N., Bianchi, J., Gurgel, M. L., Cevidanes, L., & Rao, A. (2022). Predicting osteoarthritis of the temporomandibular joint using random forest with\u00a0privileged information. In Ethical and philosophical issues in medical imaging, multimodal learning and fusion across scales for clinical decision support, and topological data analysis for biomedical imaging (pp. 77\u201386). Springer. https:\/\/doi.org\/10.1007\/978-3-031-23223-7_7","DOI":"10.1007\/978-3-031-23223-7_7"},{"key":"2032_CR83","unstructured":"World Health Organization. (2024). Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models, Geneva. https:\/\/www.who.int\/publications\/i\/item\/9789240084759"},{"key":"2032_CR84","doi-asserted-by":"publisher","unstructured":"Xing, X., Chen, Z., Zhu, M., Hou, Y., Gao, Z., & Yuan, Y. (2022). Discrepancy and\u00a0gradient-guided multi-modal knowledge distillation for\u00a0pathological glioma grading. In Lecture notes in computer science (pp. 636\u2013646. Springer. https:\/\/doi.org\/10.1007\/978-3-031-16443-9_61","DOI":"10.1007\/978-3-031-16443-9_61"},{"key":"2032_CR85","doi-asserted-by":"publisher","unstructured":"Xiong, J., He, A. W., Fu, M., Hu, X., Zhang, Y., Liu, C., Zhao, X., & Ge, Z. (2020). Improve unseen domain generalization via enhanced local color transformation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 433\u2013443). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_42","DOI":"10.1007\/978-3-030-59713-9_42"},{"key":"2032_CR86","doi-asserted-by":"publisher","unstructured":"Xue, Y., Feng, S., Zhang, Y., Zhang, X., & Wang, Y. (2020). Dual-task self-supervision for cross-modality domain adaptation. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 408\u2013417). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59710-8_40","DOI":"10.1007\/978-3-030-59710-8_40"},{"key":"2032_CR87","doi-asserted-by":"publisher","unstructured":"Yang, J., Chen, J., Kuang, K., Lin, T., He, J., & Ni, B. (2020). MIA-prognosis: A deep learning framework to predict therapy response. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 211\u2013220). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_21","DOI":"10.1007\/978-3-030-59713-9_21"},{"key":"2032_CR88","doi-asserted-by":"publisher","unstructured":"Yang, Y., Wang, N., Yang, H., Sun, J., & Xu, Z. (2020). Model-driven deep attention network for ultra-fast compressive sensing MRI guided by cross-contrast MR image. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 188\u2013198). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_19","DOI":"10.1007\/978-3-030-59713-9_19"},{"key":"2032_CR89","doi-asserted-by":"publisher","unstructured":"Zhang, Y., He, N., Yang, J., Li, Y., Wei, D., Huang, Y., Zhang, Y., He, Z., & Zheng, Y. (2022). mmFormer: Multimodal medical transformer for\u00a0incomplete multimodal learning of\u00a0brain tumor segmentation. In Lecture notes in computer science (pp. 107\u2013117). Springer. https:\/\/doi.org\/10.1007\/978-3-031-16443-9_11","DOI":"10.1007\/978-3-031-16443-9_11"},{"key":"2032_CR90","unstructured":"Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C.P. (2022). Contrastive learning of medical visual representations from paired images and text. In Proceedings of machine learning research (Vol. 182, pp. 1\u201324). Machine Learning for Healthcare, PMLR. https:\/\/proceedings.mlr.press\/v182\/zhang22a\/zhang22a.pdf"},{"key":"2032_CR91","doi-asserted-by":"publisher","unstructured":"Zhang, L., Na, S., Liu, T., Zhu, D., & Huang, J. (2023). Multimodal deep fusion in\u00a0hyperbolic space for\u00a0mild cognitive impairment study. In Lecture notes in computer science (pp. 674\u2013684). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43904-9_65","DOI":"10.1007\/978-3-031-43904-9_65"},{"key":"2032_CR92","doi-asserted-by":"publisher","first-page":"104830","DOI":"10.1109\/access.2020.2999750","volume":"8","author":"F Zhang","year":"2020","unstructured":"Zhang, F., & Wang, C. (2020). MSGAN: Generative adversarial networks for image seasonal style transfer. IEEE Access, 8, 104830\u2013104840. https:\/\/doi.org\/10.1109\/access.2020.2999750","journal-title":"IEEE Access"},{"key":"2032_CR93","doi-asserted-by":"publisher","unstructured":"Zhou, T., Fu, H., Zhang, Y., Zhang, C., Lu, X., Shen, J., & Shao, L. (2020). M2net: Multi-modal multi-channel network for overall survival time prediction of brain tumor patients. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 221\u2013231). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_22","DOI":"10.1007\/978-3-030-59713-9_22"},{"key":"2032_CR94","doi-asserted-by":"publisher","unstructured":"Zhou, Y., Yang, G., Zhou, Y., Ding, D., & Zhao, J. (2023). Representation, alignment, fusion: A generic transformer-based framework for\u00a0multi-modal glaucoma recognition. In Lecture notes in computer science (pp. 704\u2013713). Springer. https:\/\/doi.org\/10.1007\/978-3-031-43990-2_66","DOI":"10.1007\/978-3-031-43990-2_66"},{"key":"2032_CR95","doi-asserted-by":"crossref","unstructured":"Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv:1703.10593.","DOI":"10.1109\/ICCV.2017.244"},{"key":"2032_CR96","doi-asserted-by":"publisher","unstructured":"Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE international conference on computer vision (ICCV). IEEE. https:\/\/doi.org\/10.1109\/iccv.2017.244","DOI":"10.1109\/iccv.2017.244"},{"key":"2032_CR97","doi-asserted-by":"publisher","unstructured":"Zhu, Y., Tang, Y., Tang, Y., Elton, D. C., Lee, S., Pickhardt, P. J., & Summers, R. M. (2020). Cross-domain medical image translation by shared latent gaussian mixture model. In Medical image computing and computer assisted intervention\u2014MICCAI 2020 (pp. 379\u2013389). Springer. https:\/\/doi.org\/10.1007\/978-3-030-59713-9_37","DOI":"10.1007\/978-3-030-59713-9_37"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-024-02032-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-024-02032-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-024-02032-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,27]],"date-time":"2024-08-27T07:34:30Z","timestamp":1724744070000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-024-02032-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,23]]},"references-count":97,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["2032"],"URL":"https:\/\/doi.org\/10.1007\/s11263-024-02032-8","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,23]]},"assertion":[{"value":"30 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The Authors declare no competing financial interests but the following competing non-financial interests: A.R. serves as a member for Voxel Analytics, LLC. C.E.K.\u2019s institution receives salary support for service as editor of Radiology: Artificial Intelligence.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}