{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,12]],"date-time":"2026-11-12T06:13:55Z","timestamp":1794464035552,"version":"4.9.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This single-center retrospective study evaluated the performance of four multimodal large language models (MLLMs) (ChatGPT-4o, Claude 3.5 Sonnet, Google Gemini 1.5 Pro, Perplexity Sonar Large) in detecting and grading the severity of age-related macular degeneration (AMD) from ultrawide field fundus images. Images from 76 patients (136 eyes; mean age 81.1\u00a0years; 69.7% female) seen at the University of California San Diego were graded independently for AMD severity by two junior retinal specialists (and an adjudicating senior retina specialist for disagreements) using the Age-Related Eye Disease Study (AREDS) classification. The cohort included 17 (12.5%) eyes with \u2018No AMD\u2019, 18 (13.2%) with \u2018Early AMD\u2019, 50 (36.8%) with \u2018Intermediate AMD\u2019, and 51 (37.5%) with \u2018Advanced AMD\u2019. Between December 2024 and February 2025, each MLLM was prompted with single images and standardized queries to assess the primary outcomes of accuracy, sensitivity, and specificity in binary disease classification, disease severity grading, open-ended diagnosis, and multiple-choice diagnosis (with distractor diseases). Secondary outcomes included precision, F1 scores, Cohen\u2019s kappa, model performance comparisons, and error analysis. ChatGPT-4o demonstrated the highest accuracy for binary disease classification [mean 0.824 (95% confidence interval (CI)): 0.743, 0.875)], followed by Perplexity Sonar Large [mean 0.815 (95% CI: 0.744, 0.879)], both of which were significantly more accurate (<jats:italic>P<\/jats:italic>\u2009&lt;\u20090.00033) Than Gemini 1.5 Pro [mean 0.669 (95% CI: 0.581, 0.743)] and Claude 3.5 Sonnet [mean 0.301 (95% CI: 0.221, 0.375)]. For severity grading, Perplexity Sonar Large was most accurate [mean 0.463 (95% CI: 0.368, 0.537)], though differences among models were not statistically significant. ChatGPT-4o led in open-ended and multiple-choice diagnostic tasks. In summary, while MLLMs show promise for automated AMD detection and grading from fundus images, their current reliability is insufficient for clinical application, highlighting the need for further model development and validation.<\/jats:p>","DOI":"10.1038\/s41598-025-18306-1","type":"journal-article","created":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T11:31:53Z","timestamp":1758886313000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Evaluating the clinical utility of multimodal large language models for detecting age-related macular degeneration from retinal imaging"],"prefix":"10.1038","volume":"15","author":[{"given":"Jesse A.","family":"Most","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gillian A.","family":"Folk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Evan H.","family":"Walker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ines D.","family":"Nagel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nehal N.","family":"Mehta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elena","family":"Flester","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shyamanga","family":"Borooah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,9,26]]},"reference":[{"issue":"2","key":"18306_CR1","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1167\/tvst.9.2.25","volume":"9","author":"A Bhuiyan","year":"2020","unstructured":"Bhuiyan, A. et al. Artificial intelligence to stratify severity of age-related macular degeneration (AMD) and predict risk of progression to late AMD. Transl. Vis. Sci. Technol. 9(2), 25. https:\/\/doi.org\/10.1167\/tvst.9.2.25 (2020).","journal-title":"Transl. Vis. Sci. Technol."},{"key":"18306_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-030-66014-7_1","volume":"1256","author":"TDL Keenan","year":"2021","unstructured":"Keenan, T. D. L., Cukras, C. A. & Chew, E. Y. Age-related macular degeneration: Epidemiology and clinical aspects. Adv. Exp. Med. Biol. 1256, 1\u201331. https:\/\/doi.org\/10.1007\/978-3-030-66014-7_1 (2021).","journal-title":"Adv. Exp. Med. Biol."},{"issue":"10","key":"18306_CR3","doi-asserted-by":"publisher","first-page":"2828","DOI":"10.1109\/TMI.2022.3172773","volume":"41","author":"H Fang","year":"2022","unstructured":"Fang, H. et al. ADAM challenge: Detecting age-related macular degeneration from fundus images. IEEE Trans. Med. Imaging 41(10), 2828\u20132847. https:\/\/doi.org\/10.1109\/TMI.2022.3172773 (2022).","journal-title":"IEEE Trans. Med. Imaging"},{"issue":"10","key":"18306_CR4","doi-asserted-by":"publisher","first-page":"CD015522","DOI":"10.1002\/14651858.CD015522.pub2","volume":"10","author":"C Kang","year":"2024","unstructured":"Kang, C. et al. Artificial intelligence for diagnosing exudative age-related macular degeneration. Cochrane Database Syst. Rev. 10(10), CD015522. https:\/\/doi.org\/10.1002\/14651858.CD015522.pub2 (2024).","journal-title":"Cochrane Database Syst. Rev."},{"issue":"2","key":"18306_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2024.100667","volume":"5","author":"J Jalili","year":"2025","unstructured":"Jalili, J. et al. Glaucoma detection and feature identification via GPT-4V fundus image analysis. Ophthalmol. Sci. 5(2), 100667. https:\/\/doi.org\/10.1016\/j.xops.2024.100667 (2025).","journal-title":"Ophthalmol. Sci."},{"key":"18306_CR6","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1186\/s12886-024-03381-1","volume":"24","author":"E Crincoli","year":"2024","unstructured":"Crincoli, E., Sacconi, R., Querques, L. & Querques, G. Artificial intelligence in age-related macular degeneration: State of the art and recent updates. BMC Ophthalmol. 24, 121. https:\/\/doi.org\/10.1186\/s12886-024-03381-1 (2024).","journal-title":"BMC Ophthalmol."},{"issue":"6","key":"18306_CR7","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1093\/jamia\/ocaa302","volume":"28","author":"Q Chen","year":"2021","unstructured":"Chen, Q. et al. Multimodal, multitask, multiattention (M3) deep learning detection of reticular pseudodrusen: Toward automated and accessible classification of age-related macular degeneration. J. Am. Med. Inform. Assoc. 28(6), 1135\u20131148. https:\/\/doi.org\/10.1093\/jamia\/ocaa302 (2021).","journal-title":"J. Am. Med. Inform. Assoc."},{"issue":"1","key":"18306_CR8","doi-asserted-by":"publisher","first-page":"813","DOI":"10.21037\/qims-24-1406","volume":"15","author":"Z Yang","year":"2025","unstructured":"Yang, Z. et al. Evolutionary patterns and research frontiers of artificial intelligence in age-related macular degeneration: A bibliometric analysis. Quant. Imaging Med. Surg. 15(1), 813\u2013830. https:\/\/doi.org\/10.21037\/qims-24-1406 (2025).","journal-title":"Quant. Imaging Med. Surg."},{"issue":"10","key":"18306_CR9","doi-asserted-by":"publisher","first-page":"1341","DOI":"10.1136\/bjo-2024-325459","volume":"108","author":"MA Chia","year":"2024","unstructured":"Chia, M. A. et al. Foundation models in ophthalmology. Br. J. Ophthalmol. 108(10), 1341\u20131348. https:\/\/doi.org\/10.1136\/bjo-2024-325459 (2024).","journal-title":"Br. J. Ophthalmol."},{"key":"18306_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2024.100681","author":"AP Agnihotri","year":"2024","unstructured":"Agnihotri, A. P. et al. Large language models in ophthalmology: A review of publications from top ophthalmology journals. Ophthalmol. Sci. https:\/\/doi.org\/10.1016\/j.xops.2024.100681 (2024).","journal-title":"Ophthalmol. Sci."},{"issue":"11","key":"18306_CR11","doi-asserted-by":"publisher","first-page":"1484","DOI":"10.1001\/archopht.123.11.1484","volume":"123","author":"Age-Related Eye Disease Study Research Group","year":"2005","unstructured":"Age-Related Eye Disease Study Research Group. The age-related eye disease study severity scale for age-related macular degeneration: AREDS report No. 17. Arch. Ophthalmol. 123(11), 1484\u20131498. https:\/\/doi.org\/10.1001\/archopht.123.11.1484 (2005).","journal-title":"Arch. Ophthalmol."},{"issue":"1","key":"18306_CR12","doi-asserted-by":"publisher","first-page":"P1","DOI":"10.1016\/j.ophtha.2019.09.024","volume":"127","author":"CJ Flaxel","year":"2020","unstructured":"Flaxel, C. J. et al. Age-related macular degeneration preferred practice pattern\u00ae. Ophthalmology 127(1), P1\u2013P65. https:\/\/doi.org\/10.1016\/j.ophtha.2019.09.024 (2020).","journal-title":"Ophthalmology"},{"issue":"3","key":"18306_CR13","doi-asserted-by":"publisher","first-page":"454","DOI":"10.4103\/tjo.TJO-D-23-00193","volume":"14","author":"JH Wu","year":"2024","unstructured":"Wu, J. H., Nishida, T., Moghimi, S. & Weinreb, R. N. Effects of prompt engineering on large language model performance in response to questions on common ophthalmic conditions. Taiwan J. Ophthalmol. 14(3), 454\u2013457. https:\/\/doi.org\/10.4103\/tjo.TJO-D-23-00193 (2024).","journal-title":"Taiwan J. Ophthalmol."},{"issue":"1","key":"18306_CR14","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1111\/opo.13242","volume":"44","author":"S Ho","year":"2024","unstructured":"Ho, S., Doig, G. S. & Ly, A. Diagnostic accuracy of community optometrists for age-related macular degeneration using colour fundus photographs: A pilot evaluation. Ophthalmic Physiol. Opt. 44(1), 17\u201322. https:\/\/doi.org\/10.1111\/opo.13242 (2024).","journal-title":"Ophthalmic Physiol. Opt."},{"issue":"550","key":"18306_CR15","doi-asserted-by":"publisher","first-page":"309","DOI":"10.3399\/096016408784217395","volume":"58","author":"AL Solebo","year":"2008","unstructured":"Solebo, A. L., Angunawela, R. I., Dasgupta, S. & Marshall, J. Recent advances in the treatment of age-related macular degeneration. Br. J. Gen. Pract. 58(550), 309. https:\/\/doi.org\/10.3399\/096016408784217395 (2008).","journal-title":"Br. J. Gen. Pract."},{"key":"18306_CR16","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.compbiomed.2017.01.018","volume":"82","author":"P Burlina","year":"2017","unstructured":"Burlina, P., Pacheco, K. D., Joshi, N., Freund, D. E. & Bressler, N. M. Comparing humans and deep learning performance for grading AMD: A study in using universal deep features and transfer learning for automated AMD analysis. Comput. Biol. Med. 82, 80\u201386. https:\/\/doi.org\/10.1016\/j.compbiomed.2017.01.018 (2017).","journal-title":"Comput. Biol. Med."},{"issue":"10","key":"18306_CR17","doi-asserted-by":"publisher","first-page":"1221","DOI":"10.3390\/diagnostics15101221","volume":"15","author":"C Jiao","year":"2025","unstructured":"Jiao, C. et al. Diagnostic performance of publicly available large language models in corneal diseases: A comparison with human specialists. Diagnostics (Basel) 15(10), 1221. https:\/\/doi.org\/10.3390\/diagnostics15101221 (2025).","journal-title":"Diagnostics (Basel)"},{"key":"18306_CR18","doi-asserted-by":"publisher","unstructured":"Than JCM, Vong WT, Yong KSC. Comparison of Multi-Modal Large Language Models with Deep Learning Models for Medical Image Classification. In: 2024 IEEE 8th International Conference on Signal and Image Processing Applications (ICSIPA). ; 2024:1\u20135. https:\/\/doi.org\/10.1109\/ICSIPA62061.2024.10687159","DOI":"10.1109\/ICSIPA62061.2024.10687159"},{"issue":"4","key":"18306_CR19","doi-asserted-by":"publisher","first-page":"104468","DOI":"10.1016\/j.jfo.2025.104468","volume":"48","author":"S Demir","year":"2025","unstructured":"Demir, S. Comparison of ChatGPT-4o, Google Gemini 1.5 Pro, Microsoft Copilot Pro, and Ophthalmologists in the management of uveitis and ocular inflammation: A comparative study of large language models. J. Fran\u00e7ais d\u2019Ophtalmologie 48(4), 104468. https:\/\/doi.org\/10.1016\/j.jfo.2025.104468 (2025).","journal-title":"J. Fran\u00e7ais d\u2019Ophtalmologie"},{"key":"18306_CR20","doi-asserted-by":"publisher","unstructured":"Sapkota R, Raza S, Karkee M. 2025 Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models. Published online. https:\/\/doi.org\/10.48550\/arXiv.2502.18505","DOI":"10.48550\/arXiv.2502.18505"},{"key":"18306_CR21","doi-asserted-by":"publisher","unstructured":"Neuman WR, Coleman C, Shah M. Analyzing the Ethical Logic of Six Large Language Models. Published online January 15, 2025. https:\/\/doi.org\/10.48550\/arXiv.2501.08951","DOI":"10.48550\/arXiv.2501.08951"},{"issue":"4","key":"18306_CR22","doi-asserted-by":"publisher","first-page":"e507","DOI":"10.1016\/j.jcjo.2025.01.001","volume":"60","author":"M Silhadi","year":"2025","unstructured":"Silhadi, M., Nassrallah, W. B., Mikhail, D., Milad, D. & Harissi-Dagher, M. Assessing the performance of Microsoft Copilot, GPT-4 and Google Gemini in ophthalmology. Can. J. Ophthalmol. 60(4), e507\u2013e514. https:\/\/doi.org\/10.1016\/j.jcjo.2025.01.001 (2025).","journal-title":"Can. J. Ophthalmol."},{"issue":"1","key":"18306_CR23","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0262838","volume":"17","author":"S Rajaraman","year":"2022","unstructured":"Rajaraman, S., Ganesan, P. & Antani, S. Deep learning model calibration for improving performance in class-imbalanced medical image classification tasks. PLoS ONE 17(1), e0262838. https:\/\/doi.org\/10.1371\/journal.pone.0262838 (2022).","journal-title":"PLoS ONE"},{"issue":"12","key":"18306_CR24","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1001\/jamaophthalmol.2022.4401","volume":"140","author":"DB Rein","year":"2022","unstructured":"Rein, D. B. et al. Prevalence of Age-Related Macular Degeneration in the US in 2019. JAMA Ophthalmol. 140(12), 1202\u20131208. https:\/\/doi.org\/10.1001\/jamaophthalmol.2022.4401 (2022).","journal-title":"JAMA Ophthalmol."},{"issue":"7","key":"18306_CR25","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1016\/j.oret.2024.03.013","volume":"8","author":"A Mihalache","year":"2024","unstructured":"Mihalache, A. et al. Chatbot and academy preferred practice pattern guidelines on retinal diseases. Ophthalmol. Retina. 8(7), 723\u2013725. https:\/\/doi.org\/10.1016\/j.oret.2024.03.013 (2024).","journal-title":"Ophthalmol. Retina."},{"issue":"12","key":"18306_CR26","doi-asserted-by":"publisher","first-page":"1359","DOI":"10.1001\/jamaophthalmol.2018.4118","volume":"136","author":"PM Burlina","year":"2018","unstructured":"Burlina, P. M. et al. Use of deep learning for detailed severity characterization and estimation of 5-year risk among patients with age-related macular degeneration. JAMA Ophthalmol. 136(12), 1359\u20131366. https:\/\/doi.org\/10.1001\/jamaophthalmol.2018.4118 (2018).","journal-title":"JAMA Ophthalmol."},{"issue":"1","key":"18306_CR27","doi-asserted-by":"publisher","first-page":"1108","DOI":"10.21037\/qims-23-892","volume":"14","author":"D Tian","year":"2024","unstructured":"Tian, D., Jiang, S., Zhang, L., Lu, X. & Xu, Y. The role of large language models in medical image processing: A narrative review. Quant. Imaging Med. Surg. 14(1), 1108\u20131121. https:\/\/doi.org\/10.21037\/qims-23-892 (2024).","journal-title":"Quant. Imaging Med. Surg."},{"issue":"6","key":"18306_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2024.100556","volume":"4","author":"A Mihalache","year":"2024","unstructured":"Mihalache, A. et al. Interpretation of clinical retinal images using an artificial intelligence chatbot. Ophthalmol. Sci. 4(6), 100556. https:\/\/doi.org\/10.1016\/j.xops.2024.100556 (2024).","journal-title":"Ophthalmol. Sci."},{"issue":"7","key":"18306_CR29","first-page":"822","volume":"65","author":"N Bommakanti","year":"2024","unstructured":"Bommakanti, N. et al. Accuracy and readability of after visit summaries for retinal conditions generated by a large language model. Invest. Ophthalmol. Vis. Sci. 65(7), 822 (2024).","journal-title":"Invest. Ophthalmol. Vis. Sci."},{"issue":"10","key":"18306_CR30","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1136\/bjo-2023-324533","volume":"108","author":"KX Cheong","year":"2024","unstructured":"Cheong, K. X. et al. Comparing generative and retrieval-based chatbots in answering patient questions regarding age-related macular degeneration and diabetic retinopathy. Br. J. Ophthalmol. 108(10), 1443\u20131449. https:\/\/doi.org\/10.1136\/bjo-2023-324533 (2024).","journal-title":"Br. J. Ophthalmol."},{"issue":"4","key":"18306_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2024.100485","volume":"4","author":"PD Tailor","year":"2024","unstructured":"Tailor, P. D. et al. A comparative study of responses to retina questions from either experts, expert-edited large language models, or expert-edited large language models alone. Ophthalmol. Sci. 4(4), 100485. https:\/\/doi.org\/10.1016\/j.xops.2024.100485 (2024).","journal-title":"Ophthalmol. Sci."},{"issue":"4","key":"18306_CR32","doi-asserted-by":"publisher","DOI":"10.1111\/nan.12997","volume":"50","author":"D Ono","year":"2024","unstructured":"Ono, D., Dickson, D. W. & Koga, S. Evaluating the efficacy of few-shot learning for GPT-4Vision in neurodegenerative disease histopathology: A comparative analysis with convolutional neural network model. Neuropathol. Appl. Neurobiol. 50(4), e12997. https:\/\/doi.org\/10.1111\/nan.12997 (2024).","journal-title":"Neuropathol. Appl. Neurobiol."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-18306-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-18306-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-18306-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T11:31:55Z","timestamp":1758886315000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-18306-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,26]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["18306"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-18306-1","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,26]]},"assertion":[{"value":"7 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 September 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"33214"}}