{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,2]],"date-time":"2026-11-02T18:36:55Z","timestamp":1793644615967,"version":"4.5.1"},"reference-count":22,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T00:00:00Z","timestamp":1743984000000},"content-version":"unspecified","delay-in-days":6,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005057","name":"National Tsing Hua University","doi-asserted-by":"publisher","award":["BX20230189"],"award-info":[{"award-number":["BX20230189"]}],"id":[{"id":"10.13039\/501100005057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001349","name":"National Medical Research Council","doi-asserted-by":"publisher","award":["NMRC\/MOH\/ HCSAINV21nov-000"],"award-info":[{"award-number":["NMRC\/MOH\/ HCSAINV21nov-000"]}],"id":[{"id":"10.13039\/501100001349","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open Ophth"],"accepted":{"date-parts":[[2025,2,14]]},"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The advent of generative artificial intelligence has led to the emergence of multiple vision large language models (VLLMs). This study aimed to evaluate the capabilities of commonly available VLLMs, such as OpenAI\u2019s GPT-4V and Google\u2019s Gemini, in detecting and diagnosing ocular diseases from retinal images.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods and analysis<\/jats:title>\n                    <jats:p>From the Singapore Epidemiology of Eye Diseases (SEED) study, we selected 44 representative retinal photographs, including 10 healthy and 34 representing six eye diseases (age-related macular degeneration, diabetic retinopathy, glaucoma, visually significant cataract, myopic macular degeneration and retinal vein occlusion). OpenAI\u2019s GPT-4V (both default and data analyst modes) and Google Gemini were prompted with each image to determine if the retina was normal or abnormal and to provide diagnostic descriptions if deemed abnormal. The outputs from the VLLMs were evaluated for accuracy by three attending-level ophthalmologists using a three-point scale (poor, borderline, good).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>GPT-4V default mode demonstrated the highest detection rate, correctly identifying 33 out of 34 detected correctly (97.1%), outperforming its data analyst mode (61.8%) and Google Gemini (41.2%). Despite the relatively high detection rates, the quality of diagnostic descriptions was generally suboptimal\u2014with only 21.2% of GPT-4V\u2019s (default) responses, 4.8% of GPT-4V\u2019s (data analyst) responses and 28.6% for Google Gemini\u2019s responses rated as good.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Although GPT-4V default mode showed generally high sensitivity in abnormality detection, all evaluated VLLMs were inadequate in providing accurate diagnoses for ocular diseases. These findings emphasise the need for domain-customised VLLMs and suggest the continued need for human oversight in clinical ophthalmology.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjophth-2024-002076","type":"journal-article","created":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T21:40:28Z","timestamp":1744062028000},"page":"e002076","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":6,"title":["Can off-the-shelf visual large language models detect and diagnose ocular diseases from retinal photographs?"],"prefix":"10.1136","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-9355-6717","authenticated-orcid":false,"given":"Sahana","family":"Srinivasan","sequence":"first","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"},{"name":"Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3657-4666","authenticated-orcid":false,"given":"Hongwei","family":"Ji","sequence":"additional","affiliation":[{"name":"Tsinghua Medicine, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2153-3100","authenticated-orcid":false,"given":"David Ziyou","family":"Chen","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Department of Ophthalmology, National University Hospital, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4514-250X","authenticated-orcid":false,"given":"Wendy","family":"Wong","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Department of Ophthalmology, National University Hospital, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1182-3489","authenticated-orcid":false,"given":"Zhi Da","family":"Soh","sequence":"additional","affiliation":[{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5052-6081","authenticated-orcid":false,"given":"Jocelyn Hui Lin","family":"Goh","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"},{"name":"Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6263-8489","authenticated-orcid":false,"given":"Krithi","family":"Pushpanathan","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3175-0344","authenticated-orcid":false,"given":"Xiaofei","family":"Wang","sequence":"additional","affiliation":[{"name":"Beihang University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5604-7527","authenticated-orcid":false,"given":"Weizhi","family":"Ma","sequence":"additional","affiliation":[{"name":"Institure for AI Industry Research, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8448-1264","authenticated-orcid":false,"given":"Tien Yin","family":"Wong","sequence":"additional","affiliation":[{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"},{"name":"Tsinghua Medicine, Tsinghua University, Beijing, China"},{"name":"Beijing Visual Science and Translational Eye Research Institute (BERI), Eye Center of Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing, China"},{"name":"Beijing Key Laboratory of Intelligent Diagnostic Technology and Devices for Major Blinding Eye Diseases, Tsinghua Medicine, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2749-7793","authenticated-orcid":false,"given":"Ya Xing","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Visual Science and Translational Eye Research Institute (BERI), Eye Center of Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing, China"},{"name":"Beijing Key Laboratory of Intelligent Diagnostic Technology and Devices for Major Blinding Eye Diseases, Tsinghua Medicine, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0655-885X","authenticated-orcid":false,"given":"Ching-Yu","family":"Cheng","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"},{"name":"Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore"},{"name":"Eye Academic Clinical Program (Eye ACP), Duke NUS Medical School, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6752-797X","authenticated-orcid":false,"given":"Yih Chung","family":"Tham","sequence":"additional","affiliation":[{"name":"Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore"},{"name":"Singapore Eye Research Institute, Singapore National Eye Centre, Singapore"},{"name":"Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore"},{"name":"Eye Academic Clinical Program (Eye ACP), Duke NUS Medical School, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2025,4,7]]},"reference":[{"key":"2026010721202718000_10.1.e002076.1","doi-asserted-by":"crossref","first-page":"e1432","DOI":"10.1016\/S2214-109X(23)00323-6","article-title":"Artificial intelligence and digital health in global eye health: opportunities and challenges","volume":"11","author":"Tan","year":"2023","journal-title":"Lancet Glob Health"},{"key":"2026010721202718000_10.1.e002076.2","unstructured":"Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need. In: Guyon I , Luxburg UV , Bengio S , eds. Advances in Neural Information Processing Systems. Curran Associates, Inc, 2017."},{"key":"2026010721202718000_10.1.e002076.3","doi-asserted-by":"crossref","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the Performance of ChatGPT in Ophthalmology: An Analysis of Its Successes and Shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol Sci"},{"key":"2026010721202718000_10.1.e002076.4","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1001\/jamaophthalmol.2024.1165","article-title":"Vision-Language Models for Feature Detection of Macular Diseases on Optical Coherence Tomography","volume":"142","author":"Antaki","year":"2024","journal-title":"JAMA Ophthalmol"},{"key":"2026010721202718000_10.1.e002076.5","unstructured":"Bard . Google bard new features update july 2023. 2023."},{"key":"2026010721202718000_10.1.e002076.6","article-title":"ChatGPT: promise and challenges for deployment in low- and middle-income countries","volume":"41","author":"Wang","year":"2023","journal-title":"Lancet Reg Health West Pac"},{"key":"2026010721202718000_10.1.e002076.7","unstructured":"ChatGPT . ChatGPT \u2014 Release Notes, 2023. Available: https:\/\/help.openai.com\/en\/articles\/6825453-chatgpt-release-notes"},{"key":"2026010721202718000_10.1.e002076.8","doi-asserted-by":"crossref","first-page":"e917","DOI":"10.1016\/S2589-7500(23)00201-7","article-title":"Large language models and their impact in ophthalmology","volume":"5","author":"Betzler","year":"2023","journal-title":"The Lancet Digital Health"},{"key":"2026010721202718000_10.1.e002076.9","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1001\/jamaophthalmol.2023.4508","article-title":"Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral","volume":"141","author":"Korot","year":"2023","journal-title":"JAMA Ophthalmol"},{"key":"2026010721202718000_10.1.e002076.10","unstructured":"OpenAI . GPT-4 system card. 2023."},{"key":"2026010721202718000_10.1.e002076.11","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/dyaa238"},{"key":"2026010721202718000_10.1.e002076.12","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1080\/08820538.2023.2209166","article-title":"ChatGPT and Ophthalmology: Exploring Its Potential with Discharge Summaries and Operative Notes","volume":"38","author":"Singh","year":"2023","journal-title":"Semin Ophthalmol"},{"key":"2026010721202718000_10.1.e002076.13","doi-asserted-by":"crossref","DOI":"10.1016\/j.xops.2024.100600","article-title":"Analysis of ChatGPT Responses to Ophthalmic Cases: Can ChatGPT Think like an Ophthalmologist?","volume":"5","author":"Chen","year":"2025","journal-title":"Ophthalmol Sci"},{"key":"2026010721202718000_10.1.e002076.14","doi-asserted-by":"publisher","DOI":"10.1136\/bjo-2023-325054"},{"key":"2026010721202718000_10.1.e002076.15","doi-asserted-by":"crossref","DOI":"10.2196\/50842","article-title":"Performance of ChatGPT on Ophthalmology-Related Questions Across Various Examination Levels: Observational Study","volume":"10","author":"Haddad","year":"2024","journal-title":"JMIR Med Educ"},{"key":"2026010721202718000_10.1.e002076.16","doi-asserted-by":"crossref","DOI":"10.2196\/60226","article-title":"Performance of ChatGPT in Ophthalmic Registration and Clinical Diagnosis: Cross-Sectional Study","volume":"26","author":"Ming","year":"2024","journal-title":"J Med Internet Res"},{"key":"2026010721202718000_10.1.e002076.17","unstructured":"OpenAI . GPT-4 technical report. 2023."},{"key":"2026010721202718000_10.1.e002076.18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3529755","article-title":"On the Explainability of Natural Language Processing Deep Models","volume":"55","author":"Zini","year":"2023","journal-title":"ACM Comput Surv"},{"key":"2026010721202718000_10.1.e002076.19","unstructured":"Wu C , Zheng Q , Zhao W , et al . Can GPT-4V(ision) Serve Medical Applications Case Studies on GPT-4V for Multimodal Medical Diagnosis. arXiv 2023.:arXiv231009909v2."},{"key":"2026010721202718000_10.1.e002076.20","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"2026010721202718000_10.1.e002076.21","doi-asserted-by":"publisher","DOI":"10.1001\/jamaophthalmol.2024.2513"},{"key":"2026010721202718000_10.1.e002076.22","unstructured":"Chen X , Chen R , Xu P , et al . Visual Question Answering in Ophthalmology: A Progressive and Practical Perspective. arXiv 2024."}],"container-title":["BMJ Open Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjophth-2024-002076","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T05:20:48Z","timestamp":1767849648000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjophth.bmj.com\/lookup\/doi\/10.1136\/bmjophth-2024-002076"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4]]},"references-count":22,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,4,7]]},"published-print":{"date-parts":[[2025,4]]}},"alternative-id":["10.1136\/bmjophth-2024-002076"],"URL":"https:\/\/doi.org\/10.1136\/bmjophth-2024-002076","relation":{},"ISSN":["2397-3269"],"issn-type":[{"value":"2397-3269","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4]]}}}