{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,5]],"date-time":"2026-09-05T11:55:00Z","timestamp":1788609300272,"version":"build-2803163510"},"reference-count":47,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T00:00:00Z","timestamp":1747094400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: This study evaluated the diagnostic accuracy of seven publicly available large language models (LLMs)\u2014GPT-3.5, GPT-4.o Mini, GPT-4.o, Gemini 1.5 Flash, Claude 3.5 Sonnet, Grok3, and DeepSeek R1\u2014in diagnosing corneal diseases, comparing their performance to human specialists. Methods: Twenty corneal disease cases from the University of Iowa\u2019s EyeRounds were presented to each LLM. Diagnostic accuracy was determined by comparing LLM-generated diagnoses to the confirmed case diagnoses. Four human cornea specialists evaluated the same cases to establish a benchmark and assess interobserver agreement. Results: Diagnostic accuracy varied significantly among LLMs (p = 0.001). GPT-4.o achieved the highest accuracy (80.0%), followed by Claude 3.5 Sonnet and Grok3 (70.0%), DeepSeek R1 (65.0%), GPT-3.5 (60.0%), GPT-4.o Mini (55.0%), and Gemini 1.5 Flash (30.0%). Human experts averaged 92.5% accuracy, outperforming all LLMs (p &lt; 0.001, Cohen\u2019s d = \u22121.314). GPT-4.o showed no significant difference from human consensus (p = 0.250, \u03ba = 0.348), while Claude and Grok3 showed fair agreement (\u03ba = 0.219). DeepSeek R1 also performed reasonably (\u03ba = 0.178), although not significantly. Conclusions: Among the evaluated LLMs, GPT-4.o, Claude 3.5 Sonnet, Grok3, and DeepSeek R1 demonstrated promising diagnostic accuracy, with GPT-4.o most closely matching human performance. However, performance remained inconsistent, especially in complex cases. LLMs may offer value as diagnostic support tools, but human expertise remains indispensable for clinical decision-making.<\/jats:p>","DOI":"10.3390\/diagnostics15101221","type":"journal-article","created":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T06:40:23Z","timestamp":1747118423000},"page":"1221","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Diagnostic Performance of Publicly Available Large Language Models in Corneal Diseases: A Comparison with Human Specialists"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4191-4656","authenticated-orcid":false,"given":"Cheng","family":"Jiao","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erik","family":"Rosas","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0744-8337","authenticated-orcid":false,"given":"Hassan","family":"Asadigandomani","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of California San Francisco, San Francisco, CA 94143, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5638-2034","authenticated-orcid":false,"given":"Mohammad","family":"Delsoz","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Hamilton Eye Institute, University of Tennessee Health Science Center, Memphis, TN 38103, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yeganeh","family":"Madadi","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Hamilton Eye Institute, University of Tennessee Health Science Center, Memphis, TN 38103, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hina","family":"Raja","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Hamilton Eye Institute, University of Tennessee Health Science Center, Memphis, TN 38103, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8070-6668","authenticated-orcid":false,"given":"Wuqaas","family":"Munir","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology and Visual Sciences, University of Maryland School of Medicine, Baltimore, MD 21201, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brendan","family":"Tamm","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology and Visual Sciences, University of Maryland School of Medicine, Baltimore, MD 21201, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4249-3304","authenticated-orcid":false,"given":"Shiva","family":"Mehravaran","sequence":"additional","affiliation":[{"name":"Department of Biology, School of Computer, Mathematical, and Natural Sciences, Morgan State University, Baltimore, MD 21251, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Djalilian","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siamak","family":"Yousefi","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Hamilton Eye Institute, University of Tennessee Health Science Center, Memphis, TN 38103, USA"},{"name":"Department of Genetics, Genomics, and Informatics, University of Tennessee Health Science Center, Memphis, TN 38136, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6546-3546","authenticated-orcid":false,"given":"Mohammad","family":"Soleimani","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1016\/j.ophtha.2023.03.012","article-title":"Global trends in blindness and vision impairment resulting from corneal opacity 1984\u20132020: A meta-analysis","volume":"130","author":"Wang","year":"2023","journal-title":"Ophthalmology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.ajo.2024.11.023","article-title":"Socioeconomic and Demographic Disparities in Keratoconus Treatment","volume":"271","author":"Erukulla","year":"2025","journal-title":"Am. J. Ophthalmol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e130","DOI":"10.1016\/S2214-109X(20)30425-3","article-title":"Trends in prevalence of blindness and distance and near vision impairment over 30 years: An analysis for the Global Burden of Disease Study","volume":"9","author":"Bourne","year":"2021","journal-title":"Lancet Glob. Health"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically applicable deep learning for diagnosis and referral in retinal disease","volume":"24","author":"Ledsam","year":"2018","journal-title":"Nat. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1001\/jama.2017.18152","article-title":"Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes","volume":"318","author":"Ting","year":"2017","journal-title":"JAMA"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices","volume":"1","author":"Lavin","year":"2018","journal-title":"NPJ Digit. Med."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Soleimani, M., Cheraqpour, K., Sadeghi, R., Pezeshgi, S., Koganti, R., and Djalilian, A.R. (2023). Artificial intelligence and infectious keratitis: Where are we now?. Life, 13.","DOI":"10.3390\/life13112117"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1167\/tvst.9.2.24","article-title":"Evaluating the performance of various machine learning algorithms to detect subclinical keratoconus","volume":"9","author":"Cao","year":"2020","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1310","DOI":"10.1097\/ICO.0000000000003626","article-title":"Applications of Artificial Intelligence in Diagnosis of Dry Eye Disease: A Systematic Review and Meta-Analysis","volume":"43","author":"Heidari","year":"2024","journal-title":"Cornea"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1167\/tvst.10.13.27","article-title":"Automated image segmentation of the corneal endothelium in patients with Fuchs dystrophy","volume":"10","author":"Shilpashree","year":"2021","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s12348-024-00417-x","article-title":"Clinical characteristics, predisposing factors, and management of moraxella keratitis in a tertiary eye hospital","volume":"14","author":"Soleimani","year":"2024","journal-title":"J. Ophthalmic Inflamm. Infect."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1007\/s10792-024-03106-9","article-title":"Unveiling the landscape of post-keratoplasty keratitis: A comprehensive epidemiological analysis in a tertiary center","volume":"44","author":"Soleimani","year":"2024","journal-title":"Int. Ophthalmol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"110234","DOI":"10.1016\/j.exer.2025.110234","article-title":"A deep dive into radiation keratopathy; going beyond the current frontierss","volume":"251","author":"Soleimani","year":"2025","journal-title":"Exp. Eye Res."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.jtos.2024.07.010","article-title":"A novel artificial intelligence model for diagnosing Acanthamoeba keratitis through confocal microscopy","volume":"34","author":"Shareef","year":"2024","journal-title":"Ocul. Surf."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hayati, A., Abdol Homayuni, M.R., Sadeghi, R., Asadigandomani, H., Dashtkoohi, M., Eslami, S., and Soleimani, M. (2025). Advancing Diabetic Retinopathy Screening: A Systematic Review of Artificial Intelligence and Optical Coherence Tomography Angiography Innovations. Diagnostics, 15.","DOI":"10.3390\/diagnostics15060737"},{"key":"ref_17","first-page":"510","article-title":"Comparison of anterior segment measurements using rotating Scheimpflug imaging and partial coherence interferometry","volume":"6","author":"Soleimani","year":"2013","journal-title":"Int. J. Ophthalmol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat. Med."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"B\u00e9lisle-Pipon, J.-C. (2024). Why we need to be careful with LLMs in medicine. Front. Med., 11.","DOI":"10.3389\/fmed.2024.1495582"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Taloni, A., Borselli, M., Scarsi, V., Rossi, C., Coco, G., Scorcia, V., and Giannaccare, G. (2023). Comparative performance of humans versus GPT-4.0 and GPT-3.5 in the self-assessment program of American Academy of Ophthalmology. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-45837-2"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1097\/ICO.0000000000003492","article-title":"Performance of ChatGPT in diagnosis of corneal eye diseases","volume":"43","author":"Delsoz","year":"2024","journal-title":"Cornea"},{"key":"ref_23","unstructured":"Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., and Millican, K. (2023). Gemini: A family of highly capable multimodal models. arXiv."},{"key":"ref_24","unstructured":"(2024, December 28). GPT-4o vs. Gemini 1.5 Pro vs. Claude 3 Opus: Multimodal AI Model Comparison. Available online: https:\/\/encord.com\/blog\/gpt-4o-vs-gemini-vs-claude-3-opus\/."},{"key":"ref_25","unstructured":"(2024, December 28). Introducing the Next Generation of Claude. Available online: https:\/\/www.anthropic.com\/news\/claude-3-family."},{"key":"ref_26","unstructured":"EyeRounds (2024, December 28). Case Reports, Tutorials, Videos, Images from Univ of Iowa Dept of Ophthalmology. Available online: https:\/\/eyerounds.org\/#gsc.tab=0."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1038\/s41746-023-00879-8","article-title":"The shaky foundations of large language models and foundation models for electronic health records","volume":"6","author":"Wornow","year":"2023","journal-title":"npj Digit. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"100600","DOI":"10.1016\/j.xops.2024.100600","article-title":"Analysis of ChatGPT responses to ophthalmic cases: Can ChatGPT think like an ophthalmologist?","volume":"5","author":"Chen","year":"2025","journal-title":"Ophthalmol. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2530","DOI":"10.1038\/s41433-024-03067-4","article-title":"Google Gemini and Bard artificial intelligence chatbot performance in ophthalmology knowledge assessment","volume":"38","author":"Mihalache","year":"2024","journal-title":"Eye"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.ajo.2023.05.024","article-title":"Performance of Generative Large Language Models on Ophthalmology Board-Style Questions","volume":"254","author":"Cai","year":"2023","journal-title":"Am. J. Ophthalmol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw. Open"},{"key":"ref_32","unstructured":"Holmes, J., Peng, R., Li, Y., Hu, J., Liu, Z., Wu, Z., Zhao, H., Jiang, X., Liu, W., and Wei, H. (2023). Evaluating multiple large language models in pediatric ophthalmology. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1001\/jamaophthalmol.2023.6917","article-title":"Assessment of a Large Language Model\u2019s Responses to Questions and Cases About Glaucoma and Retina Management","volume":"142","author":"Huang","year":"2024","journal-title":"JAMA Ophthalmol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"105886","DOI":"10.1016\/j.ijmedinf.2025.105886","article-title":"Using large language models as decision support tools in emergency ophthalmology","volume":"199","author":"Kreso","year":"2025","journal-title":"Int. J. Med. Inform."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"798","DOI":"10.1001\/jamaophthalmol.2023.2754","article-title":"Performance of an upgraded artificial intelligence chatbot for ophthalmic knowledge assessment","volume":"141","author":"Mihalache","year":"2023","journal-title":"JAMA Ophthalmol."},{"key":"ref_36","first-page":"e45700","article-title":"Evaluating the artificial intelligence performance growth in ophthalmic knowledge","volume":"15","author":"Jiao","year":"2023","journal-title":"Cureus"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Vithanage, D., Deng, C., Wang, L., Yin, M., Alkhalaf, M., Zhang, Z., Zhu, Y., Soewargo, A.C., and Yu, P. (2024). Evaluating approaches of training a generative large language model for multi-label classification of unstructured electronic health records. medRxiv.","DOI":"10.1101\/2024.06.24.24309441"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"6656797","DOI":"10.1155\/2021\/6656797","article-title":"Dual Generative Network with Discriminative Information for Generalized Zero-Shot Learning","volume":"2021","author":"Xu","year":"2021","journal-title":"Complexity"},{"key":"ref_39","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv."},{"key":"ref_40","unstructured":"Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., and Henighan, T. (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv."},{"key":"ref_41","first-page":"e44720","article-title":"A call to address AI \u201challucinations\u201d and how healthcare professionals can mitigate their risks","volume":"15","author":"Hatem","year":"2023","journal-title":"Cureus"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Soleimani, M., Esmaili, K., Rahdar, A., Aminizadeh, M., Cheraqpour, K., Tabatabaei, S.A., Mirshahi, R., Bibak, Z., Mohammadi, S.F., and Koganti, R. (2023). From the diagnosis of infectious keratitis to discriminating fungal subtypes; a deep learning-based study. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-49635-8"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hussain, Z.S., Delsoz, M., Elahi, M., Jerkins, B., Kanner, E., Wright, C., Munir, W.M., Soleimani, M., Djalilian, A., and Lao, P.A. (2025). Performance of DeepSeek, Qwen 2.5 MAX, and ChatGPT Assisting in Diagnosis of Corneal Eye Diseases, Glaucoma, and Neuro-Ophthalmology Diseases Based on Clinical Case Reports. medRxiv.","DOI":"10.1101\/2025.03.14.25323836"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s12348-025-00465-x","article-title":"Diagnosis of microbial keratitis using smartphone-captured images; a deep-learning model","volume":"15","author":"Soleimani","year":"2025","journal-title":"J. Ophthalmic Inflamm. Infect."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Ferrara, E. (2023). Should chatgpt be biased? challenges and risks of bias in large language models. arXiv.","DOI":"10.2139\/ssrn.4627814"},{"key":"ref_46","unstructured":"Rosenfeld, A., and Lazebnik, T. (2024). Whose LLM is it anyway? Linguistic comparison and LLM attribution for GPT-3.5, GPT-4 and bard. arXiv."},{"key":"ref_47","unstructured":"Brown, N.B. (2024). Enhancing trust in llms: Algorithms for comparing and interpreting llms. arXiv."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/10\/1221\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:31:43Z","timestamp":1760031103000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/10\/1221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,13]]},"references-count":47,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["diagnostics15101221"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15101221","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,13]]}}}