{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T14:17:11Z","timestamp":1793888231316,"version":"4.8.0"},"reference-count":63,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,9,23]],"date-time":"2023-09-23T00:00:00Z","timestamp":1695427200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,23]],"date-time":"2023-09-23T00:00:00Z","timestamp":1695427200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["2022.09212.PTDC"],"award-info":[{"award-number":["2022.09212.PTDC"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"name":"UNESCO Chair on AI&XR"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This study investigates the effects of including patients\u2019 clinical information on the performance of deep learning (DL) classifiers for disease location in chest X-ray images. Although current classifiers achieve high performance using chest X-ray images alone, consultations with practicing radiologists indicate that clinical data is highly informative and essential for interpreting medical images and making proper diagnoses. In this work, we propose a novel architecture consisting of two fusion methods that enable the model to simultaneously process patients\u2019 clinical data (structured data) and chest X-rays (image data). Since these data modalities are in different dimensional spaces, we propose a spatial arrangement strategy, <jats:italic>spatialization<\/jats:italic>, to facilitate the multimodal learning process in a Mask R-CNN model. We performed an extensive experimental evaluation using MIMIC-Eye, a dataset comprising different modalities: MIMIC-CXR (chest X-ray images), MIMIC IV-ED (patients\u2019 clinical data), and REFLACX (annotations of disease locations in chest X-rays). Results show that incorporating patients\u2019 clinical data in a DL model together with the proposed fusion methods improves the disease localization in chest X-rays by 12% in terms of Average Precision compared to a standard Mask R-CNN using chest X-rays alone. Further ablation studies also emphasize the importance of multimodal DL architectures and the incorporation of patients\u2019 clinical data in disease localization. In the interest of fostering scientific reproducibility, the architecture proposed within this investigation has been made publicly accessible(<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/ChihchengHsieh\/multimodal-abnormalities-detection\">https:\/\/github.com\/ChihchengHsieh\/multimodal-abnormalities-detection<\/jats:ext-link>).<\/jats:p>","DOI":"10.1038\/s41598-023-41463-0","type":"journal-article","created":{"date-parts":[[2023,9,23]],"date-time":"2023-09-23T10:02:01Z","timestamp":1695463321000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["MDF-Net for abnormality detection by fusing X-rays with clinical data"],"prefix":"10.1038","volume":"13","author":[{"given":"Chihcheng","family":"Hsieh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Isabel Blanco","family":"Nobre","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandra Costa","family":"Sousa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chun","family":"Ouyang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Margot","family":"Brereton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacinto C.","family":"Nascimento","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joaquim","family":"Jorge","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catarina","family":"Moreira","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,9,23]]},"reference":[{"issue":"2129\u20132154","key":"41463_CR1","doi-asserted-by":"publisher","first-page":"1990","DOI":"10.1016\/s0140-6736(22)00532-3","volume":"399","author":"A Haakenstad","year":"2022","unstructured":"Haakenstad, A. et al. Measuring the availability of human resources for health and its relationship to universal health coverage for 204 countries and territories from, to 2019: A systematic analysis for the global burden of disease study 2019. Lancet 399(2129\u20132154), 1990. https:\/\/doi.org\/10.1016\/s0140-6736(22)00532-3 (2022).","journal-title":"Lancet"},{"key":"41463_CR2","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.101562","author":"G Maicas","year":"2019","unstructured":"Maicas, G., Bradley, A. P., Nascimento, J. C., Reid, I. & Carneiro, G. Pre and post-hoc diagnosis and interpretation of malignancy from breast DCE-MRI. Med. Image Anal.https:\/\/doi.org\/10.1016\/j.media.2019.101562 (2019).","journal-title":"Med. Image Anal."},{"key":"41463_CR3","doi-asserted-by":"publisher","first-page":"2045","DOI":"10.1038\/s41598-019-48995-4","volume":"9","author":"L Shen","year":"2019","unstructured":"Shen, L. et al. Deep learning to improve breast cancer detection on screening mammography. Sci. Rep. 9, 2045\u20132322 (2019).","journal-title":"Sci. Rep."},{"key":"41463_CR4","doi-asserted-by":"publisher","first-page":"H390","DOI":"10.1152\/ajpheart.00416.2020","volume":"321","author":"X Liu","year":"2020","unstructured":"Liu, X. et al. Deep learning-based automated left ventricular ejection fraction assessment using 2-d echocardiography. J. Physiol. Heart Circ. Physiol. 321, H390\u2013H399 (2020).","journal-title":"J. Physiol. Heart Circ. Physiol."},{"key":"41463_CR5","doi-asserted-by":"publisher","first-page":"8167","DOI":"10.1109\/TPAMI.2021.3113077","volume":"44","author":"DO Medley","year":"2022","unstructured":"Medley, D. O., Santiago, C. & Nascimento, J. C. Cycoseg: A cyclic collaborative framework for automated medical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 44, 8167\u20138182. https:\/\/doi.org\/10.1109\/TPAMI.2021.3113077 (2022).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"41463_CR6","doi-asserted-by":"publisher","first-page":"17485","DOI":"10.1038\/s41598-021-96707-8","volume":"11","author":"T-C Pham","year":"2021","unstructured":"Pham, T.-C., Luong, C.-M., Hoang, V.-D. & Doucet, A. Ai outperformed every dermatologist in dermoscopic melanoma diagnosis, using an optimized deep-CNN architecture with custom mini-batch logic and loss function. Sci. Rep. 11, 17485 (2021).","journal-title":"Sci. Rep."},{"key":"41463_CR7","doi-asserted-by":"publisher","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","volume":"29","author":"H Haenssle","year":"2018","unstructured":"Haenssle, H. et al. Man against machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 29, 1836\u20131842 (2018).","journal-title":"Ann. Oncol."},{"key":"41463_CR8","doi-asserted-by":"crossref","unstructured":"Irvin, J. et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI Conference on Artificial Intelligence 590\u2013597 (2019).","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"41463_CR9","unstructured":"Rajpurkar, P. et\u00a0al. Chexnet: Radiologist-level pneumonia detection on chest X-rays with deep learning. CoRR abs\/1711.05225 1711.05225 (2017)."},{"key":"41463_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pmed.1002686","volume":"15","author":"P Rajpurkar","year":"2018","unstructured":"Rajpurkar, P. et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists. PLoS Med. 15, 1\u201317. https:\/\/doi.org\/10.1371\/journal.pmed.1002686 (2018).","journal-title":"PLoS Med."},{"key":"41463_CR11","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1016\/j.crad.2018.05.015","volume":"73","author":"E Yates","year":"2018","unstructured":"Yates, E., Yates, L. & Harvey, H. Machine learning \u201cred dot\u2019\u2019: open-source, cloud, deep convolutional neural networks in chest radiograph binary normality classification. Clin. Radiol. 73, 827\u2013831. https:\/\/doi.org\/10.1016\/j.crad.2018.05.015 (2018).","journal-title":"Clin. Radiol."},{"key":"41463_CR12","doi-asserted-by":"publisher","unstructured":"Moreira, C. et\u00a0al. Comparing visual search patterns in chest x-ray diagnostics. In Proceedings of the 2023 Symposium on Eye Tracking Research and Applications, ETRA \u201923 (Association for Computing Machinery, 2023). https:\/\/doi.org\/10.1145\/3588015.3588403.","DOI":"10.1145\/3588015.3588403"},{"key":"41463_CR13","unstructured":"Rahimi, S., Oktay, O., Alvarez-Valle, J. & Bharadwaj, S. Addressing the exorbitant cost of labeling medical images with active learning. In International Conference on Machine Learning in Medical Imaging and Analysis (2021)."},{"key":"41463_CR14","first-page":"931","volume":"51","author":"R Guidotti","year":"2018","unstructured":"Guidotti, R. et al. A survey of methods for explaining black box models. ACM Comput. Surv. 51, 931\u20139342 (2018).","journal-title":"ACM Comput. Surv."},{"key":"41463_CR15","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/3233231","volume":"61","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z. C. The mythos of model interpretability. ACM Commun. 61, 36\u201343 (2018).","journal-title":"ACM Commun."},{"key":"41463_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","volume":"267","author":"T Miller","year":"2019","unstructured":"Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artif. intell. 267, 1\u201338 (2019).","journal-title":"Artif. intell."},{"key":"41463_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2021.113561","volume":"150","author":"C Moreira","year":"2021","unstructured":"Moreira, C. et al. Linda-bn: An interpretable probabilistic approach for demystifying black-box predictive models. Decis. Support Syst. 150, 113561 (2021).","journal-title":"Decis. Support Syst."},{"key":"41463_CR18","doi-asserted-by":"crossref","unstructured":"Sindhgatta, R., Ouyang, C. & Moreira, C. Exploring interpretability for predictive process analytics. In Proceedings of the 18th International Conference on Service Oriented Computing (ICSOC) (2020).","DOI":"10.1007\/978-3-030-65310-1_31"},{"key":"41463_CR19","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.inffus.2021.11.003","volume":"81","author":"Y-L Chou","year":"2022","unstructured":"Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C. & Jorge, J. Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Inf. Fusion 81, 59\u201383 (2022).","journal-title":"Inf. Fusion"},{"key":"41463_CR20","doi-asserted-by":"crossref","unstructured":"Singh, A., Sengupta, S. & Lakshminarayanan, V. Explainable deep learning models in medical image analysis. J. Imaging. 6 (2020).","DOI":"10.3390\/jimaging6060052"},{"key":"41463_CR21","unstructured":"Zhuang, F. et\u00a0al. A comprehensive survey on transfer learning. CoRR abs\/1911.02685 1911.02685 (2019)."},{"key":"41463_CR22","doi-asserted-by":"crossref","unstructured":"Yuan, Z., Yan, Y., Sonka, M. & Yang, T. Robust deep AUC maximization: A new surrogate loss and empirical studies on medical image classification. CoRR 2012.03173 2012.03173 (2020).","DOI":"10.1109\/ICCV48922.2021.00303"},{"key":"41463_CR23","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0273-z","author":"Y-X Tang","year":"2020","unstructured":"Tang, Y.-X. et al. Automated abnormality classification of chest radiographs using deep convolutional neural networks. NPJ Digit. Med.https:\/\/doi.org\/10.1038\/s41746-020-0273-z (2020).","journal-title":"NPJ Digit. Med."},{"key":"41463_CR24","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P. & Girshick, R.\u00a0B. Mask R-CNN. CoRR 1703.06870 (2017).","DOI":"10.1109\/ICCV.2017.322"},{"key":"41463_CR25","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.106874","volume":"158","author":"M Liu","year":"2023","unstructured":"Liu, M. et al. Aa-wgan: Attention augmented Wasserstein generative adversarial network with application to fundus retinal vessel segmentation. Comput. Biol. Med. 158, 106874 (2023).","journal-title":"Comput. Biol. Med."},{"key":"41463_CR26","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.106457","volume":"152","author":"P Wu","year":"2023","unstructured":"Wu, P. et al. Aggn: Attention-based glioma grading network with multi-scale feature extraction and multi-modal information fusion. Comput. Biol. Med. 152, 106457 (2023).","journal-title":"Comput. Biol. Med."},{"key":"41463_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118029","volume":"207","author":"H Li","year":"2022","unstructured":"Li, H., Zeng, N., Wu, P. & Clawson, K. Cov-net: A computer-aided diagnosis method for recognizing covid-19 from chest X-ray images via machine vision. Expert Syst. Appl. 207, 118029 (2022).","journal-title":"Expert Syst. Appl."},{"key":"41463_CR28","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-021-02166-7","author":"K Bayoudh","year":"2021","unstructured":"Bayoudh, K., Knani, R., Hamdaoui, F. & Mtibaa, A. A survey on deep multimodal learning for computer vision: Advances, trends, applications, and datasets. Vis. Comput.https:\/\/doi.org\/10.1007\/s00371-021-02166-7 (2021).","journal-title":"Vis. Comput."},{"key":"41463_CR29","first-page":"4835","volume":"33","author":"Y Wang","year":"2020","unstructured":"Wang, Y. et al. Deep multimodal fusion by channel exchanging. Adv. Neural Inf. Process. Syst. 33, 4835\u20134845 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"41463_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-019-0105-1","volume":"2","author":"MA Badgeley","year":"2019","unstructured":"Badgeley, M. A. et al. Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit. Med. 2, 1\u201310 (2019).","journal-title":"NPJ Digit. Med."},{"key":"41463_CR31","doi-asserted-by":"publisher","first-page":"1449","DOI":"10.1109\/JPROC.2015.2460697","volume":"103","author":"D Lahat","year":"2015","unstructured":"Lahat, D., Adali, T. & Jutten, C. Multimodal data fusion: An overview of methods, challenges, and prospects. Proc. IEEE 103, 1449\u20131477 (2015).","journal-title":"Proc. IEEE"},{"key":"41463_CR32","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/MSP.2017.2738401","volume":"34","author":"D Ramachandram","year":"2017","unstructured":"Ramachandram, D. & Taylor, G. W. Deep multimodal learning: A survey on recent advances and trends. IEEE Signal Process. Mag. 34, 96\u2013108. https:\/\/doi.org\/10.1109\/MSP.2017.2738401 (2017).","journal-title":"IEEE Signal Process. Mag."},{"key":"41463_CR33","unstructured":"Smilkov, D., Thorat, N., Kim, B., Vi\u00e9gas, F.\u00a0B. & Wattenberg, M. Smoothgrad: removing noise by adding noise. CoRR 1706.03825 (2017)."},{"key":"41463_CR34","doi-asserted-by":"publisher","unstructured":"Lu\u00eds, A. et\u00a0al. Integrating eye-gaze data into cxr dl approaches: A preliminary study. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) 196\u2013199. https:\/\/doi.org\/10.1109\/VRW58643.2023.00048 (2023).","DOI":"10.1109\/VRW58643.2023.00048"},{"key":"41463_CR35","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105253","volume":"144","author":"MA Azam","year":"2022","unstructured":"Azam, M. A. et al. A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics. Comput. Biol. Med. 144, 105253. https:\/\/doi.org\/10.1016\/j.compbiomed.2022.105253 (2022).","journal-title":"Comput. Biol. Med."},{"key":"41463_CR36","doi-asserted-by":"crossref","unstructured":"Moon, J.\u00a0H., Lee, H., Shin, W., Kim, Y.-H. & Choi, E. Multi-modal understanding and generation for medical images and text via vision-language pre-training. IEEE J. Biomed. Health Inform. 1\u201310 (2022).","DOI":"10.2196\/preprints.31613"},{"key":"41463_CR37","doi-asserted-by":"crossref","unstructured":"Yan, B. & Pei, M. Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (2022).","DOI":"10.1609\/aaai.v36i3.20204"},{"key":"41463_CR38","doi-asserted-by":"crossref","unstructured":"Chen, Z., Li, G. & Wan, X. Align, reason and learn: Enhancing medical vision-and-language pre-training with knowledge. In Proceedings of the 30th ACM International Conference on Multimedia 5152\u20135161 (2022).","DOI":"10.1145\/3503161.3547948"},{"key":"41463_CR39","unstructured":"Oakden-Rayner, L. et\u00a0al. Producing radiologist-quality reports for interpretable deep learning. In IEEE 16th International Symposium on Biomedical Imaging (2019)."},{"key":"41463_CR40","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1002\/jmrs.424","volume":"68","author":"C Castillo","year":"2021","unstructured":"Castillo, C., Steffens, T., Sim, L. & Caffery, L. The effect of clinical information on radiology reporting: A systematic review. J. Med. Radiat. Sci. 68, 60\u201374. https:\/\/doi.org\/10.1002\/jmrs.424 (2021).","journal-title":"J. Med. Radiat. Sci."},{"key":"41463_CR41","unstructured":"Johnson, A. E.\u00a0W. et\u00a0al. MIMIC-CXR-JPG: A large publicly available database of labeled chest radiographs. CoRR (2019). 1901.07042."},{"key":"41463_CR42","doi-asserted-by":"publisher","unstructured":"Johnson, A. et\u00a0al. Mimic-iv. https:\/\/doi.org\/10.13026\/S6N6-XD98 (2021).","DOI":"10.13026\/S6N6-XD98"},{"key":"41463_CR43","unstructured":"Hsieh, C. et\u00a0al. Mimic-eye: Integrating mimic datasets with reflacx and eye gaze for multimodal deep learning applications. PhysioNet (version 1.0.0) (2023)."},{"key":"41463_CR44","doi-asserted-by":"publisher","unstructured":"Johnson, A. E.\u00a0W., Pollard, T., Mark, R., Berkowitz, S. & Horng, S. The mimic-cxr database. https:\/\/doi.org\/10.13026\/C2JT1Q (2019).","DOI":"10.13026\/C2JT1Q"},{"key":"41463_CR45","doi-asserted-by":"publisher","unstructured":"Johnson, A. et\u00a0al. Mimic-iv-ed. https:\/\/doi.org\/10.13026\/77Z6-9W59 (2021).","DOI":"10.13026\/77Z6-9W59"},{"key":"41463_CR46","unstructured":"Howard, A.\u00a0G. et\u00a0al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR 1704.04861. abs\/1704.04861 (2017)."},{"key":"41463_CR47","unstructured":"Howard, A. et\u00a0al. Searching for mobilenetv3. CoRR 1905.02244. 1905.02244 (2019)."},{"key":"41463_CR48","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S.\u00a0K., Girshick, R.\u00a0B. & Farhadi, A. You only look once: Unified, real-time object detection. CoRR 1506.02640. 1506.02640 (2015).","DOI":"10.1109\/CVPR.2016.91"},{"key":"41463_CR49","doi-asserted-by":"crossref","unstructured":"Liu, W. et\u00a0al. Ssd: Single shot multibox detector. In Computer Vision\u2014ECCV 2016 (eds. Leibe, B., Matas, J., Sebe, N. & Welling, M.) 21\u201337 (Springer International Publishing, 2016).","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"41463_CR50","unstructured":"Carion, N. et\u00a0al. End-to-end object detection with transformers. CoRR 2005.12872. 2005.12872 (2020)."},{"key":"41463_CR51","doi-asserted-by":"crossref","unstructured":"Schweitzer, D. & Agrawal, R. Multi-class object detection from aerial images using mask r-cnn. In 2018 IEEE International Conference on Big Data (Big Data) 3470\u20133477 (2018).","DOI":"10.1109\/BigData.2018.8622536"},{"key":"41463_CR52","doi-asserted-by":"publisher","unstructured":"Liu, H. & Bhanu, B. Pose-guided r-cnn for jersey number recognition in sports. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2457\u20132466. https:\/\/doi.org\/10.1109\/CVPRW.2019.00301 (2019).","DOI":"10.1109\/CVPRW.2019.00301"},{"key":"41463_CR53","doi-asserted-by":"publisher","DOI":"10.1016\/j.ecoinf.2022.101593","volume":"69","author":"CR Conrady","year":"2022","unstructured":"Conrady, C. R., \u015eebnem, E., Attwood, C. G., Roberson, L. A. & de Vos, L. Automated detection and classification of southern African roman seabream using mask r-cnn. Ecol. Inform. 69, 101593 (2022).","journal-title":"Ecol. Inform."},{"key":"41463_CR54","doi-asserted-by":"publisher","unstructured":"Zeiler, M.\u00a0D., Krishnan, D., Taylor, G.\u00a0W. & Fergus, R. Deconvolutional networks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2528\u20132535. https:\/\/doi.org\/10.1109\/CVPR.2010.5539957 (2010).","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"41463_CR55","doi-asserted-by":"publisher","unstructured":"Dumoulin, V. & Visin, F. A guide to convolution arithmetic for deep learning. https:\/\/doi.org\/10.48550\/ARXIV.1603.07285 (2016).","DOI":"10.48550\/ARXIV.1603.07285"},{"key":"41463_CR56","doi-asserted-by":"publisher","unstructured":"Albawi, S., Mohammed, T.\u00a0A. & Al-Zawi, S. Understanding of a convolutional neural network. In International Conference on Engineering and Technology (ICET) 1\u20136.https:\/\/doi.org\/10.1109\/ICEngTechnol.2017.8308186 (2017).","DOI":"10.1109\/ICEngTechnol.2017.8308186"},{"key":"41463_CR57","first-page":"818","volume":"2014","author":"MD Zeiler","year":"2014","unstructured":"Zeiler, M. D. & Fergus, R. Visualizing and understanding convolutional networks. Comput. Vis. ECCV 2014, 818\u2013833 (2014).","journal-title":"Comput. Vis. ECCV"},{"key":"41463_CR58","unstructured":"Ren, S., He, K., Girshick, R. & Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems, vol.\u00a028 (Curran Associates, Inc., 2015)."},{"key":"41463_CR59","unstructured":"Kendall, A., Gal, Y. & Cipolla, R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. CoRR 1705.07115 (2017)."},{"key":"41463_CR60","doi-asserted-by":"crossref","unstructured":"German National Cohort (GNC) Consortium geschaeftsstelle@ nationale-kohorte. de. The german national cohort: Aims, study design and organization. Eur. J. Epidemiol. 29, 371\u2014382 (2014).","DOI":"10.1007\/s10654-014-9890-7"},{"key":"41463_CR61","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001779","volume":"31","author":"C Sudlow","year":"2015","unstructured":"Sudlow, C. et al. Uk biobank: An open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 31, e1001779 (2015).","journal-title":"PLoS Med."},{"key":"41463_CR62","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41597-022-01441-z","volume":"9","author":"R Bigolin Lanfredi","year":"2022","unstructured":"Bigolin Lanfredi, R. et al. Reflacx, a dataset of reports and eye-tracking data for localization of abnormalities in chest X-rays. Sci. Data 9, 350 (2022).","journal-title":"Sci. Data"},{"key":"41463_CR63","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.103823","volume":"123","author":"Z Wang","year":"2020","unstructured":"Wang, Z., Cui, Z. & Zhu, Y. Multi-modal medical image fusion by laplacian pyramid and adaptive sparse representation. Comput. Biol. Med. 123, 103823 (2020).","journal-title":"Comput. Biol. Med."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-41463-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-41463-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-41463-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T16:20:55Z","timestamp":1700324455000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-41463-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,23]]},"references-count":63,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["41463"],"URL":"https:\/\/doi.org\/10.1038\/s41598-023-41463-0","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,23]]},"assertion":[{"value":"20 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"15873"}}