{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,13]],"date-time":"2026-11-13T03:18:22Z","timestamp":1794539902119,"version":"4.9.1"},"reference-count":104,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T00:00:00Z","timestamp":1758758400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T00:00:00Z","timestamp":1758758400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Artificial intelligence (AI) has become a fundamental tool for assisting clinicians in analyzing ophthalmic images, such as optical coherence tomography (OCT). However, developing AI models often requires extensive annotation, and existing models tend to underperform on independent, unseen data. Foundation models (FMs), large AI models trained on vast, unlabeled datasets, have shown promise in overcoming these challenges. Nonetheless, available FMs for ophthalmology lack extensive validation, especially for segmentation tasks, and focus on a single imaging modality. In this context, we propose MIRAGE, a novel multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO) images. Additionally, we propose a new evaluation benchmark with OCT\/SLO classification and segmentation tasks. The comparison with general and specialized FMs and segmentation methods shows the superiority of MIRAGE in both types of tasks, highlighting its suitability as a basis for the development of robust AI systems for retinal OCT image analysis.<\/jats:p>","DOI":"10.1038\/s41746-025-01852-3","type":"journal-article","created":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T10:10:03Z","timestamp":1758795003000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis"],"prefix":"10.1038","volume":"8","author":[{"given":"Jos\u00e9","family":"Morano","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Botond","family":"Fazekas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emese","family":"S\u00fckei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald","family":"Fecso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taha","family":"Emre","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Gumpinger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Georg","family":"Faustmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marzieh","family":"Oghbaie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ursula","family":"Schmidt-Erfurth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hrvoje","family":"Bogunovi\u0107","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,9,25]]},"reference":[{"key":"1852_CR1","doi-asserted-by":"publisher","first-page":"e489","DOI":"10.1016\/S2214-109X(20)30488-5","volume":"9","author":"MJ Burton","year":"2021","unstructured":"Burton, M. J. et al. The lancet global health commission on global eye health: vision beyond 2020. Lancet Glob. Health 9, e489\u2013e551 (2021).","journal-title":"Lancet Glob. Health"},{"key":"1852_CR2","doi-asserted-by":"crossref","unstructured":"Kanski, J. J. & Bowling, B. Clinical Ophthalmology: A Systematic Approach 7th edn (Elsevier Health Sciences, 2011).","DOI":"10.1016\/B978-0-7020-4093-1.00019-7"},{"key":"1852_CR3","doi-asserted-by":"crossref","unstructured":"Mohammadpour, M. Diagnostics in Ocular Imaging: Cornea, Retina, Glaucoma and Orbit (Springer Nature, 2020).","DOI":"10.1007\/978-3-030-54863-6"},{"key":"1852_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.preteyeres.2015.07.007","volume":"50","author":"U Schmidt-Erfurth","year":"2016","unstructured":"Schmidt-Erfurth, U. & Waldstein, S. M. A paradigm shift in imaging biomarkers in neovascular age-related macular degeneration. Prog. Retinal Eye Res. 50, 1\u201324 (2016).","journal-title":"Prog. Retinal Eye Res."},{"key":"1852_CR5","doi-asserted-by":"crossref","unstructured":"Schuman, J. S., Fujimoto, J. G., Duker, J. & Ishikawa, H. Optical Coherence Tomography of Ocular Diseases (CRC Press, 2024).","DOI":"10.1201\/9781003525455"},{"key":"1852_CR6","doi-asserted-by":"publisher","first-page":"100031","DOI":"10.1016\/j.xops.2021.100031","volume":"1","author":"S Bemme","year":"2021","unstructured":"Bemme, S. et al. Reliability of subjective assessment of spectral-domain OCT pathologic features by multiple raters in retinal vein occlusion. Ophthalmol. Sci. 1, 100031 (2021).","journal-title":"Ophthalmol. Sci."},{"key":"1852_CR7","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1016\/j.oret.2023.07.021","volume":"7","author":"S Schmitz-Valckenberg","year":"2023","unstructured":"Schmitz-Valckenberg, S. et al. Interreader agreement and longitudinal progression of incomplete\/complete retinal pigment epithelium and outer retinal atrophy in age-related macular degeneration. Ophthalmol. Retin. 7, 1059\u20131068 (2023).","journal-title":"Ophthalmol. Retin."},{"key":"1852_CR8","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","volume":"24","author":"J De Fauw","year":"2018","unstructured":"De Fauw, J. et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med. 24, 1342\u20131350 (2018).","journal-title":"Nat. Med."},{"key":"1852_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.preteyeres.2018.07.004","volume":"67","author":"U Schmidt-Erfurth","year":"2018","unstructured":"Schmidt-Erfurth, U., Sadeghipour, A., Gerendas, B. S., Waldstein, S. M. & Bogunovi\u0107, H. Artificial intelligence in retina. Prog. Retinal Eye Res. 67, 1\u201329 (2018).","journal-title":"Prog. Retinal Eye Res."},{"key":"1852_CR10","doi-asserted-by":"crossref","unstructured":"Yim, J. et al. Predicting conversion to wet age-related macular degeneration using deep learning. Nat. Med. 26, 892\u2013899 (2020).","DOI":"10.1038\/s41591-020-0867-7"},{"key":"1852_CR11","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1136\/bjophthalmol-2018-313173","volume":"103","author":"DSW Ting","year":"2019","unstructured":"Ting, D. S. W. et al. Artificial intelligence and deep learning in ophthalmology. Br. J. Ophthalmol. 103, 167\u2013175 (2019).","journal-title":"Br. J. Ophthalmol."},{"key":"1852_CR12","doi-asserted-by":"publisher","first-page":"100972","DOI":"10.1016\/j.preteyeres.2021.100972","volume":"86","author":"U Schmidt-Erfurth","year":"2022","unstructured":"Schmidt-Erfurth, U. et al. AI-based monitoring of retinal fluid in disease activity and under therapy. Prog. Retinal Eye Res. 86, 100972 (2022).","journal-title":"Prog. Retinal Eye Res."},{"key":"1852_CR13","doi-asserted-by":"publisher","first-page":"101305","DOI":"10.1016\/j.preteyeres.2024.101305","volume":"103","author":"GS Reiter","year":"2024","unstructured":"Reiter, G. S. et al. AI in the clinical management of GA: a novel therapeutic universe requires novel tools. Prog. Retinal Eye Res. 103, 101305 (2024).","journal-title":"Prog. Retinal Eye Res."},{"key":"1852_CR14","doi-asserted-by":"crossref","unstructured":"Paschali, M., Conjeti, S., Navarro, F. & Navab, N. Generalizability vs. robustness: investigating medical imaging networks using adversarial examples. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention 493\u2013501 (Springer, 2018).","DOI":"10.1007\/978-3-030-00928-1_56"},{"key":"1852_CR15","doi-asserted-by":"crossref","unstructured":"Hemelings, R. et al. A generalizable deep learning regression model for automated glaucoma screening from fundus images. NPJ Digital Med. 6, 112 (2023).","DOI":"10.1038\/s41746-023-00857-0"},{"key":"1852_CR16","doi-asserted-by":"crossref","unstructured":"Yang, Y., Zhang, H., Gichoya, J. W., Katabi, D. & Ghassemi, M. The limits of fair medical imaging AI in real-world generalization. Nat. Med. 30, 2838\u20132848 (2024).","DOI":"10.1038\/s41591-024-03113-4"},{"key":"1852_CR17","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1109\/MSP.2021.3134634","volume":"39","author":"L Ericsson","year":"2022","unstructured":"Ericsson, L., Gouk, H., Loy, C. C. & Hospedales, T. M. Self-supervised representation learning: Introduction, advances, and challenges. IEEE Signal Process. Mag. 39, 42\u201362 (2022).","journal-title":"IEEE Signal Process. Mag."},{"key":"1852_CR18","doi-asserted-by":"crossref","unstructured":"He, K. et al. Masked autoencoders are scalable vision learners. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 15979\u201315988 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"1852_CR19","doi-asserted-by":"publisher","first-page":"193907","DOI":"10.1109\/ACCESS.2020.3031549","volume":"8","author":"PH Le-Khac","year":"2020","unstructured":"Le-Khac, P. H., Healy, G. & Smeaton, A. F. Contrastive representation learning: a framework and review. IEEE Access 8, 193907\u2013193934 (2020).","journal-title":"IEEE Access"},{"key":"1852_CR20","unstructured":"Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. International Conference on Machine Learning 8748\u20138763 (PMLR, 2021)."},{"key":"1852_CR21","doi-asserted-by":"crossref","unstructured":"Caron, M. et al. Emerging properties in self-supervised vision transformers. In Proc. IEEE\/CVF International Conference on Computer Vision, pp. 9650\u20139660 (IEEE, 2021).","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"1852_CR22","doi-asserted-by":"crossref","unstructured":"Girdhar, R. et al. ImageBind: one embedding space to bind them all. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 15180\u201315190 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01457"},{"key":"1852_CR23","doi-asserted-by":"crossref","unstructured":"Bachmann, R., Mizrahi, D., Atanov, A. & Zamir, A. MultiMAE: multi-modal multi-task masked autoencoders. In Proc. European Conference on Computer Vision 348\u2013367 (Springer, 2022).","DOI":"10.1007\/978-3-031-19836-6_20"},{"key":"1852_CR24","doi-asserted-by":"crossref","unstructured":"Morano, J., Hervella, \u00c1. S., Barreira, N., Novo, J. & Rouco, J. Multimodal transfer learning-based approaches for retinal vascular segmentation. In ECAI 2020 1866\u20131873 (IOS Press, 2020).","DOI":"10.3233\/FAIA200303"},{"key":"1852_CR25","doi-asserted-by":"publisher","first-page":"4023","DOI":"10.1109\/TMI.2020.3008871","volume":"39","author":"X Li","year":"2020","unstructured":"Li, X., Jia, M., Islam, M. T., Yu, L. & Xing, L. Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis. IEEE Trans. Med. Imaging 39, 4023\u20134033 (2020).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"1852_CR26","doi-asserted-by":"crossref","unstructured":"Morano, J. et al. Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (eds Greenspan, H. et al.) 589\u2013599 (Springer Nature Switzerland, Cham, 2023).","DOI":"10.1007\/978-3-031-43901-8_56"},{"key":"1852_CR27","doi-asserted-by":"publisher","first-page":"103357","DOI":"10.1016\/j.media.2024.103357","volume":"99","author":"J Silva-Rodr\u00edguez","year":"2025","unstructured":"Silva-Rodr\u00edguez, J., Chakor, H., Kobbi, R., Dolz, J. & Ben Ayed, I. A foundation language-image model of the retina (FLAIR): encoding expert knowledge in text supervision. Med. Image Anal. 99, 103357 (2025).","journal-title":"Med. Image Anal."},{"key":"1852_CR28","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-78515-y","volume":"14","author":"E S\u00fckei","year":"2024","unstructured":"S\u00fckei, E. et al. Multi-modal representation learning in retinal imaging using self-supervised learning for enhanced clinical predictions. Sci. Rep. 14, 26802 (2024).","journal-title":"Sci. Rep."},{"key":"1852_CR29","unstructured":"Dosovitskiy, A. et al. An image is worth 16x16 words: transformers for image recognition at scale. In Proc. International Conference on Learning Representations (Curran Associates, Inc., 2021)."},{"key":"1852_CR30","unstructured":"Bommasani, R. et al. On the opportunities and risks of foundation models. Preprint at https:\/\/arxiv.org\/abs\/2108.07258 (2021)."},{"key":"1852_CR31","doi-asserted-by":"publisher","first-page":"AIoa2300138","DOI":"10.1056\/AIoa2300138","volume":"1","author":"T Tu","year":"2024","unstructured":"Tu, T. et al. Towards generalist biomedical AI. NEJM AI 1, AIoa2300138 (2024).","journal-title":"NEJM AI"},{"key":"1852_CR32","unstructured":"Oquab, M. et al. DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res. (2024)."},{"key":"1852_CR33","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1038\/s41586-023-06555-x","volume":"622","author":"Y Zhou","year":"2023","unstructured":"Zhou, Y. et al. A foundation model for generalizable disease detection from retinal images. Nature 622, 156\u2013163 (2023).","journal-title":"Nature"},{"key":"1852_CR34","doi-asserted-by":"publisher","first-page":"AIoa2300221","DOI":"10.1056\/AIoa2300221","volume":"1","author":"J Qiu","year":"2024","unstructured":"Qiu, J. et al. Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence. NEJM AI 1, AIoa2300221 (2024).","journal-title":"NEJM AI"},{"key":"1852_CR35","unstructured":"Shi, D. et al. EyeFound: a multimodal generalist foundation model for ophthalmic imaging. Preprint at https:\/\/arxiv.org\/abs\/2405.11338 (2024)."},{"key":"1852_CR36","doi-asserted-by":"crossref","unstructured":"Shi, D. et al. A multimodal visual\u2013language foundation model for computational ophthalmology. npj Digit. Med. 8, 381 (2025).","DOI":"10.1038\/s41746-025-01772-2"},{"key":"1852_CR37","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. ImageNet: A large-scale hierarchical image database. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition 248\u2013255 (IEEE, 2009).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"1852_CR38","doi-asserted-by":"crossref","unstructured":"Cai, Z., Lin, L., He, H., Cheng, P. & Tang, X. Uni4Eye++: a general masked image modeling multi-modal pre-training framework for ophthalmic image classification and segmentation. IEEE Trans. Med. Imaging (2024).","DOI":"10.1109\/TMI.2024.3422102"},{"key":"1852_CR39","doi-asserted-by":"crossref","unstructured":"Dong, X. et al. MaskCLIP: masked self-distillation advances contrastive language-image pretraining. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 10995\u201311005 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01058"},{"key":"1852_CR40","doi-asserted-by":"publisher","first-page":"102918","DOI":"10.1016\/j.media.2023.102918","volume":"89","author":"MA Mazurowski","year":"2023","unstructured":"Mazurowski, M. A. et al. Segment anything model for medical image analysis: an experimental study. Med. Image Anal. 89, 102918 (2023).","journal-title":"Med. Image Anal."},{"key":"1852_CR41","doi-asserted-by":"publisher","first-page":"103061","DOI":"10.1016\/j.media.2023.103061","volume":"92","author":"Y Huang","year":"2024","unstructured":"Huang, Y. et al. Segment anything model for medical images? Med. Image Anal. 92, 103061 (2024).","journal-title":"Med. Image Anal."},{"key":"1852_CR42","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-44824-z","volume":"15","author":"J Ma","year":"2024","unstructured":"Ma, J. et al. Segment anything in medical images. Nat. Commun. 15, 654 (2024).","journal-title":"Nat. Commun."},{"key":"1852_CR43","unstructured":"Zhu, J., Qi, Y. & Wu, J. Medical SAM 2: segment medical images as video via Segment Anything Model 2. Preprint at https:\/\/arxiv.org\/abs\/2408.00874 (2024)."},{"key":"1852_CR44","doi-asserted-by":"crossref","unstructured":"Kirillov, A. et al. Segment anything. In Proc. IEEE\/CVF International Conference on Computer Vision, pp. 4015\u20134026 (IEEE, 2023).","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"1852_CR45","doi-asserted-by":"crossref","unstructured":"Fazekas, B., Morano, J., Lachinov, D., Aresta, G. & Bogunovi\u0107, H. Adapting segment anything model (SAM) for retinal OCT. In Ophthalmic Medical Image Analysis (eds Antony, B. et al.) 92\u2013101 (Springer Nature Switzerland, Cham, 2023).","DOI":"10.1007\/978-3-031-44013-7_10"},{"key":"1852_CR46","doi-asserted-by":"crossref","unstructured":"Gerendas, B. S. et al. Validation of an automated fluid algorithm on real-world data of neovascular age-related macular degeneration over five years. Retina 42, 1673\u20131682 (2022).","DOI":"10.1097\/IAE.0000000000003557"},{"key":"1852_CR47","doi-asserted-by":"publisher","first-page":"1495","DOI":"10.1109\/TMI.2008.923966","volume":"27","author":"MK Garvin","year":"2008","unstructured":"Garvin, M. K. et al. Intraretinal layer segmentation of macular optical coherence tomography images using optimal 3-d graph search. IEEE Trans. Med. Imaging 27, 1495\u20131505 (2008).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"1852_CR48","doi-asserted-by":"publisher","first-page":"2403","DOI":"10.1364\/BOE.2.002403","volume":"2","author":"B Antony","year":"2011","unstructured":"Antony, B. et al. Automated 3-d method for the correction of axial artifacts in spectral-domain optical coherence tomography images. Biomed. Opt. Express 2, 2403 (2011).","journal-title":"Biomed. Opt. Express"},{"key":"1852_CR49","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.ophtha.2013.07.013","volume":"121","author":"S Farsiu","year":"2014","unstructured":"Farsiu, S. et al. Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography. Ophthalmology 121, 162\u2013172 (2014).","journal-title":"Ophthalmology"},{"key":"1852_CR50","doi-asserted-by":"publisher","first-page":"102938","DOI":"10.1016\/j.media.2023.102938","volume":"90","author":"J Wu","year":"2023","unstructured":"Wu, J. et al. GAMMA challenge: glaucoma grading from multi-modality images. Med. Image Anal. 90, 102938 (2023).","journal-title":"Med. Image Anal."},{"key":"1852_CR51","doi-asserted-by":"crossref","unstructured":"Luo, Y., Shi, M., Tian, Y., Elze, T. & Wang, M. Harvard glaucoma detection and progression: a multimodal multitask dataset and generalization-reinforced semi-supervised learning. In Proc. IEEE\/CVF International Conference on Computer Vision 20471\u201320482 https:\/\/ophai.hms.harvard.edu\/datasets\/harvard-gdp1000\/ (2023).","DOI":"10.1109\/ICCV51070.2023.01872"},{"key":"1852_CR52","unstructured":"Kermany, D. Labeled optical coherence tomography (OCT) and chest X-ray images for classification https:\/\/data.mendeley.com\/datasets\/rscbjbr9sj\/2 (2018)."},{"key":"1852_CR53","doi-asserted-by":"publisher","first-page":"1122\u20131131.e9","DOI":"10.1016\/j.cell.2018.02.010","volume":"172","author":"DS Kermany","year":"2018","unstructured":"Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122\u20131131.e9 (2018).","journal-title":"Cell"},{"key":"1852_CR54","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1109\/TMI.2017.2780115","volume":"37","author":"R Rasti","year":"2017","unstructured":"Rasti, R., Rabbani, H., Mehridehnavi, A. & Hajizadeh, F. Macular OCT classification using a multi-scale convolutional neural network ensemble. IEEE Trans. Med. Imaging 37, 1024\u20131034 (2017).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"1852_CR55","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-024-03182-7","volume":"11","author":"M Kulyabin","year":"2024","unstructured":"Kulyabin, M. et al. OCTDL: Optical coherence tomography dataset for image-based deep learning methods. Sci. Data 11, 365 (2024).","journal-title":"Sci. Data"},{"key":"1852_CR56","doi-asserted-by":"publisher","first-page":"106532","DOI":"10.1016\/j.compeleceng.2019.106532","volume":"81","author":"P Gholami","year":"2020","unstructured":"Gholami, P., Roy, P., Parthasarathy, M. K. & Lakshminarayanan, V. OCTID: Optical coherence tomography image database. Comput. Electr. Eng. 81, 106532 (2020).","journal-title":"Comput. Electr. Eng."},{"key":"1852_CR57","unstructured":"Prabhushankar, M. et al. OLIVES dataset: ophthalmic labels for investigating visual eye semantics. In Advances in Neural Information Processing System, Vol. 35, pp. 9201\u20139216, https:\/\/zenodo.org\/records\/7105232 (Curran Associates, Inc., 2022)."},{"key":"1852_CR58","doi-asserted-by":"publisher","first-page":"102452","DOI":"10.1016\/j.compmedimag.2024.102452","volume":"118","author":"M Oghbaie","year":"2024","unstructured":"Oghbaie, M., Ara\u00fajo, T., Schmidt-Erfurth, U. & Bogunovi\u0107, H. VLFATRollout: fully transformer-based classifier for retinal OCT volumes. Comput. Med. Imaging Graph. 118, 102452 (2024).","journal-title":"Comput. Med. Imaging Graph."},{"key":"1852_CR59","first-page":"989","volume":"65","author":"A Rashno","year":"2018","unstructured":"Rashno, A. et al. Fully automated segmentation of fluid\/cyst regions in optical coherence tomography images with diabetic macular edema using neutrosophic sets and graph algorithms. IEEE Trans. Biomed. Eng. 65, 989\u20131001 (2018).","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"1852_CR60","doi-asserted-by":"publisher","first-page":"3568","DOI":"10.1364\/BOE.5.003568","volume":"5","author":"PP Srinivasan","year":"2014","unstructured":"Srinivasan, P. P. et al. Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. Biomed. Opt. Express 5, 3568\u20133577 (2014).","journal-title":"Biomed. Opt. Express"},{"key":"1852_CR61","doi-asserted-by":"publisher","first-page":"1172","DOI":"10.1364\/BOE.6.001172","volume":"6","author":"SJ Chiu","year":"2015","unstructured":"Chiu, S. J. et al. Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. Biomed. Opt. Express 6, 1172\u20131194 (2015).","journal-title":"Biomed. Opt. Express"},{"key":"1852_CR62","doi-asserted-by":"crossref","unstructured":"Melin\u0161\u010dak, M., Radmilovi\u010d, M., Vatavuk, Z. & Lon\u010dari\u0107, S. AROI: annotated retinal OCT images database. In Proc. International Convention on Information, Communication and Electronic Technology 371\u2013376 https:\/\/ipg.fer.hr\/ipg\/resources\/oct_image_database (2021).","DOI":"10.23919\/MIPRO52101.2021.9596934"},{"key":"1852_CR63","doi-asserted-by":"publisher","first-page":"1858","DOI":"10.1109\/TMI.2019.2901398","volume":"38","author":"H Bogunovi\u0107","year":"2019","unstructured":"Bogunovi\u0107, H. et al. RETOUCH: The retinal OCT fluid detection and segmentation benchmark and challenge. IEEE Trans. Med. Imaging 38, 1858\u20131874 (2019).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"1852_CR64","doi-asserted-by":"crossref","unstructured":"Fang, H. et al. Dataset and evaluation algorithm design for GOALS challenge. In Ophthalmic Medical Image Analysis (eds Antony, B. et al) 135\u2013142 (Springer International Publishing, Cham, 2022).","DOI":"10.1007\/978-3-031-16525-2_14"},{"key":"1852_CR65","doi-asserted-by":"publisher","first-page":"2013","DOI":"10.1038\/s41433-021-01747-z","volume":"36","author":"PTA Bui","year":"2022","unstructured":"Bui, P. T. A. et al. Fundus autofluorescence and optical coherence tomography biomarkers associated with the progression of geographic atrophy secondary to age-related macular degeneration. Eye 36, 2013\u20132019 (2022).","journal-title":"Eye"},{"key":"1852_CR66","doi-asserted-by":"crossref","unstructured":"Liu, Z. et al. A ConvNet for the 2020s. In Proc, IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976\u201311986 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"1852_CR67","doi-asserted-by":"crossref","unstructured":"He, Y. et al. SwinUNETR-V2: stronger swin transformers with stagewise convolutions for 3D medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention 416\u2013426 (Springer, 2023).","DOI":"10.1007\/978-3-031-43901-8_40"},{"key":"1852_CR68","doi-asserted-by":"crossref","unstructured":"Roy, S. et al. MedNeXt: transformer-driven scaling of convnets for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention 405\u2013415 (Springer, 2023).","DOI":"10.1007\/978-3-031-43901-8_39"},{"key":"1852_CR69","doi-asserted-by":"publisher","first-page":"103280","DOI":"10.1016\/j.media.2024.103280","volume":"97","author":"J Chen","year":"2024","unstructured":"Chen, J. et al. TransUNet: rethinking the U-Net architecture design for medical image segmentation through the lens of transformers. Med. Image Anal. 97, 103280 (2024).","journal-title":"Med. Image Anal."},{"key":"1852_CR70","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","volume":"18","author":"F Isensee","year":"2021","unstructured":"Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203\u2013211 (2021).","journal-title":"Nat. Methods"},{"key":"1852_CR71","doi-asserted-by":"crossref","unstructured":"Isensee, F. et al. nnU-Net revisited: a call for rigorous validation in 3D medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention 488\u2013498 (Springer, 2024).","DOI":"10.1007\/978-3-031-72114-4_47"},{"key":"1852_CR72","doi-asserted-by":"crossref","unstructured":"Liu, Z. et al. Swin transformer: hierarchical vision transformer using shifted windows. In Proc. IEEE\/CVF International Conference on Computer Vision, pp. 10012\u201310022 (2021).","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"1852_CR73","unstructured":"Vaswani, A. et al. Attention is all you need. In Advances in neural information processing systems, Vol. 30, pp. 6000\u20136010 (Curran Associates, Inc., 2017)."},{"key":"1852_CR74","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention 234\u2013241 (Springer, 2015).","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"1852_CR75","doi-asserted-by":"publisher","first-page":"101856","DOI":"10.1016\/j.media.2020.101856","volume":"68","author":"Y He","year":"2021","unstructured":"He, Y. et al. Structured layer surface segmentation for retina OCT using fully convolutional regression networks. Med. Image Anal. 68, 101856 (2021).","journal-title":"Med. Image Anal."},{"key":"1852_CR76","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/JBHI.2022.3217962","volume":"27","author":"B Fazekas","year":"2023","unstructured":"Fazekas, B. et al. Segmentation of Bruch\u2019s membrane in retinal OCT with AMD using anatomical priors and uncertainty quantification. IEEE J. Biomed. Health Inform. 27, 41\u201352 (2023).","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"1852_CR77","doi-asserted-by":"publisher","first-page":"124970","DOI":"10.1016\/j.eswa.2024.124970","volume":"256","author":"J Morano","year":"2024","unstructured":"Morano, J., Aresta, G. & Bogunovi\u0107, H. RRWNet: recursive refinement network for effective retinal artery\/vein segmentation and classification. Expert Syst. Appl. 256, 124970 (2024).","journal-title":"Expert Syst. Appl."},{"key":"1852_CR78","doi-asserted-by":"crossref","unstructured":"Azad, R. et al. Medical image segmentation review: the success of U-Net. IEEE Trans. Pattern Anal. Mach. Intell. 46, 10076\u201310095 (2024).","DOI":"10.1109\/TPAMI.2024.3435571"},{"key":"1852_CR79","doi-asserted-by":"crossref","unstructured":"Singh, A. et al. FLAVA: a foundational language and vision alignment model. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 15638\u201315650 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01519"},{"key":"1852_CR80","doi-asserted-by":"publisher","first-page":"850","DOI":"10.1038\/s41591-024-02857-3","volume":"30","author":"RJ Chen","year":"2024","unstructured":"Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850\u2013862 (2024).","journal-title":"Nat. Med."},{"key":"1852_CR81","doi-asserted-by":"publisher","first-page":"1459","DOI":"10.1016\/S0140-6736(22)02609-5","volume":"401","author":"RH Guymer","year":"2023","unstructured":"Guymer, R. H. & Campbell, T. G. Age-related macular degeneration. Lancet 401, 1459\u20131472 (2023).","journal-title":"Lancet"},{"key":"1852_CR82","unstructured":"Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, pp. 4171\u20134186 (Association for Computational Linguistics, 2019)."},{"key":"1852_CR83","unstructured":"Brown, T. et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, pp. 1877\u20131901 (Curran Associates, Inc., 2020)."},{"key":"1852_CR84","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","volume":"616","author":"M Moor","year":"2023","unstructured":"Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259\u2013265 (2023).","journal-title":"Nature"},{"key":"1852_CR85","doi-asserted-by":"publisher","first-page":"102996","DOI":"10.1016\/j.media.2023.102996","volume":"91","author":"S Zhang","year":"2024","unstructured":"Zhang, S. & Metaxas, D. On the challenges and perspectives of foundation models for medical image analysis. Med. Image Anal. 91, 102996 (2024).","journal-title":"Med. Image Anal."},{"key":"1852_CR86","doi-asserted-by":"publisher","first-page":"3421","DOI":"10.1109\/JBHI.2020.3001019","volume":"24","author":"Y George","year":"2020","unstructured":"George, Y. et al. Attention-guided 3D-CNN framework for glaucoma detection and structural-functional association using volumetric images. IEEE J. Biomed. health Inform. 24, 3421\u20133430 (2020).","journal-title":"IEEE J. Biomed. health Inform."},{"key":"1852_CR87","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1167\/tvst.10.6.32","volume":"10","author":"AC Lin","year":"2021","unstructured":"Lin, A. C., Lee, C. S., Blazes, M., Lee, A. Y. & Gorin, M. B. Assessing the clinical utility of expanded macular OCTs using machine learning. Transl. Vis. Sci. Technol. 10, 32\u201332 (2021).","journal-title":"Transl. Vis. Sci. Technol."},{"key":"1852_CR88","doi-asserted-by":"publisher","first-page":"103019","DOI":"10.1016\/j.media.2023.103019","volume":"91","author":"H Liu","year":"2024","unstructured":"Liu, H. et al. Simultaneous alignment and surface regression using hybrid 2D-3D networks for 3D coherent layer segmentation of retinal OCT images with full and sparse annotations. Med. Image Anal. 91, 103019 (2024).","journal-title":"Med. Image Anal."},{"key":"1852_CR89","doi-asserted-by":"publisher","first-page":"108586","DOI":"10.1016\/j.cmpb.2025.108586","volume":"261","author":"B Fazekas","year":"2025","unstructured":"Fazekas, B. et al. SD-LayerNet: Robust and label-efficient retinal layer segmentation via anatomical priors. Comput. Methods Prog. Biomed. 261, 108586 (2025).","journal-title":"Comput. Methods Prog. Biomed."},{"key":"1852_CR90","doi-asserted-by":"publisher","first-page":"2204","DOI":"10.1364\/BOE.417212","volume":"12","author":"J Li","year":"2021","unstructured":"Li, J. et al. Multi-scale GCN-assisted two-stage network for joint segmentation of retinal layers and discs in peripapillary OCT images. Biomed. Opt. Express 12, 2204\u20132220 (2021).","journal-title":"Biomed. Opt. Express"},{"key":"1852_CR91","unstructured":"Hu, E. J. et al. LoRA: low-rank adaptation of large language models. Proc. International Conference on Learning Representations (2021)."},{"key":"1852_CR92","unstructured":"Steiner, A. P. et al. How to train your ViT? Data, augmentation, and regularization in vision transformers. Trans. Mach. Learn. Res. (2022)."},{"key":"1852_CR93","doi-asserted-by":"crossref","unstructured":"Jeong, D., Garg, S., Lipton, Z. C. & Oberst, M. Medical adaptation of large language and vision-language models: Are we making progress? In Proc. 2024 Conference on Empirical Methods in Natural Language Processing 12143\u201312170 (2024).","DOI":"10.18653\/v1\/2024.emnlp-main.677"},{"key":"1852_CR94","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1001\/jamaophthalmol.2024.1165","volume":"142","author":"F Antaki","year":"2024","unstructured":"Antaki, F., Chopra, R. & Keane, P. A. Vision-language models for feature detection of macular diseases on optical coherence tomography. JAMA Ophthalmol. 142, 573\u2013576 (2024).","journal-title":"JAMA Ophthalmol."},{"key":"1852_CR95","doi-asserted-by":"publisher","first-page":"3627","DOI":"10.1364\/BOE.8.003627","volume":"8","author":"AG Roy","year":"2017","unstructured":"Roy, A. G. et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed. Opt. Express 8, 3627\u20133642 (2017).","journal-title":"Biomed. Opt. Express"},{"key":"1852_CR96","unstructured":"Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. In Advances in neural information processing systems, Vol. 32, pp. 8026\u20138037 (Curran Associates, Inc., 2019)."},{"key":"1852_CR97","unstructured":"Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In Proc. International Conference on Learning Representations (Curran Associates, Inc., 2018)."},{"key":"1852_CR98","unstructured":"Goyal, P. et al. Accurate, large minibatch SGD: training ImageNet in 1 hour. Preprint at https:\/\/arxiv.org\/abs\/1706.02677 (2017)."},{"key":"1852_CR99","doi-asserted-by":"crossref","unstructured":"Ranftl, R., Bochkovskiy, A. & Koltun, V. Vision transformers for dense prediction. In Proc. IEEE\/CVF International Conference on Computer Vision, pp. 12179\u201312188 (IEEE, 2021).","DOI":"10.1109\/ICCV48922.2021.01196"},{"key":"1852_CR100","unstructured":"Provost, F. & Domingos, P. Well-Trained PETs: Improving Probability Estimation Trees CeDER Working Paper IS-00-04 (Stern School of Business, New York University, 2000)."},{"key":"1852_CR101","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861\u2013874 (2006).","journal-title":"Pattern Recognit. Lett."},{"key":"1852_CR102","unstructured":"Mosley, L. A balanced approach to the multi-class imbalance problem [PhD thesis]. Iowa State University of Science and Technology (2013)."},{"key":"1852_CR103","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar, J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1\u201330 (2006).","journal-title":"J. Mach. Learn. Res."},{"key":"1852_CR104","doi-asserted-by":"publisher","first-page":"1251","DOI":"10.1109\/TMI.2009.2013851","volume":"28","author":"T Heimann","year":"2009","unstructured":"Heimann, T. et al. Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans. Med. Imaging 28, 1251\u20131265 (2009).","journal-title":"IEEE Trans. Med. Imaging"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01852-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01852-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01852-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T11:04:18Z","timestamp":1758798258000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01852-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,25]]},"references-count":104,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1852"],"URL":"https:\/\/doi.org\/10.1038\/s41746-025-01852-3","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,25]]},"assertion":[{"value":"2 December 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"576"}}