{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T19:37:41Z","timestamp":1793907461063,"version":"4.8.0"},"posted":{"date-parts":[[2025,3,17]]},"group-title":"Ophthalmology","reference-count":22,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2025,3,17]],"date-time":"2025-03-17T00:00:00Z","timestamp":1742169600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.medrxiv.org\/about\/FAQ#license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2025,3,17]]},"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n This study evaluates the diagnostic performance of several AI models, including Deepseek, in diagnosing corneal diseases, glaucoma, and neuro\u25a1ophthalmologic disorders.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We retrospectively selected 53 case reports from the Department of Ophthalmology and Visual Sciences at the University of Iowa, comprising 20 corneal disease cases, 11 glaucoma cases, and 22 neuro\u25a1ophthalmology cases. The case descriptions were input into DeepSeek, ChatGPT\u25a14.0, ChatGPT\u25a101, and Qwens 2.5 Max. These responses were compared with diagnoses rendered by human experts (corneal specialists, glaucoma attendings, and neuro\u25a1ophthalmologists). Diagnostic accuracy and interobserver agreement, defined as the percentage difference between each AI model\u2019s performance and the average human expert performance, were determined.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n DeepSeek achieved an overall diagnostic accuracy of 79.2%, with specialty-specific accuracies of 90.0% in corneal diseases, 54.5% in glaucoma, and 81.8% in neuro\u25a1ophthalmology. ChatGPT\u25a101 outperformed the other models with an overall accuracy of 84.9% (85.0% in corneal diseases, 63.6% in glaucoma, and 95.5% in neuro\u25a1ophthalmology), while Qwens exhibited a lower overall accuracy of 64.2% (55.0% in corneal diseases, 54.5% in glaucoma, and 77.3% in neuro\u25a1ophthalmology). Interobserver agreement analysis revealed that in corneal diseases, DeepSeek differed by \u20133.3% (90.0% vs 93.3%), ChatGPT\u25a101 by \u20138.3%, and Qwens by \u201338.3%. In glaucoma, DeepSeek outperformed the human expert average by +3.0% (54.5% vs 51.5%), while ChatGPT\u25a14.0 and ChatGPT\u25a101 exceeded it by +12.1%, and Qwens was +3.0% above the human average. In neuro\u25a1ophthalmology, DeepSeek and ChatGPT\u25a14.0 were 9.1% lower than the human average, ChatGPT\u25a101 exceeded it by +4.6%, and Qwens was 13.6% lower.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n ChatGPT\u25a101 demonstrated the highest overall diagnostic accuracy, especially in neuro\u25a1ophthalmology, while DeepSeek and ChatGPT\u25a14.0 showed comparable performance. Qwens underperformed relative to the other models, especially in corneal diseases. Although these AI models exhibit promising diagnostic capabilities, they currently lag behind human experts in certain areas, underscoring the need for a collaborative integration of clinical judgment.<\/jats:p>\n <\/jats:sec>\n \n Plain Language Summary<\/jats:title>\n This study evaluated how well several artificial intelligence (AI) models diagnose eye diseases compared to human experts. We tested four AI systems across three types of eye conditions: diseases of the cornea, glaucoma, and neuro-ophthalmologic disorders. Overall, one AI model, ChatGPT-01, performed the best, correctly diagnosing about 85% of cases, and it excelled in neuro-ophthalmology by correctly diagnosing 95.5% of cases. Two other models, DeepSeek and ChatGPT-4.0, each achieved an overall accuracy of around 79%, while the Qwens model performed lower, with an overall accuracy of about 64%. When compared with human experts, who achieved very high accuracy in corneal diseases (93.3%) and neuro-ophthalmology (90.9%) but lower in glaucoma (51.5%), the AI models showed mixed results. In glaucoma, for instance, some AI models even outperformed human experts slightly, while in corneal diseases, all AI models were less accurate than the experts. These findings indicate that while AI shows promise as a supportive tool in diagnosing eye conditions, it still needs further improvement. Combining AI with human clinical judgment appears to be the best approach for accurate eye disease diagnosis.<\/jats:p>\n <\/jats:sec>\n \n Key summary points<\/jats:title>\n \n \n \n Why carry out this study?<\/jats:bold>\n With the rising burden of eye diseases and the inherent diagnostic challenges for complex conditions like glaucoma and neuro-ophthalmologic disorders, there is an unmet need for innovative diagnostic tools to support clinical decision-making.\n <\/jats:p>\n <\/jats:list-item>\n \n \n What did the study ask?<\/jats:bold>\n This study evaluated the diagnostic performance of four AI models across three ophthalmologic subspecialties, testing the hypothesis that advanced language models can achieve accuracy levels comparable to human experts.\n <\/jats:p>\n <\/jats:list-item>\n \n \n What was learned from the study?<\/jats:bold>\n Our results showed that ChatGPT-01 achieved the highest overall accuracy (84.9%), excelling in neuro-ophthalmology with a 95.5% accuracy, while DeepSeek and ChatGPT-4.0 each achieved 79.2%, and Qwens reached 64.2%.\n <\/jats:p>\n <\/jats:list-item>\n \n \n What specific outcomes were observed?<\/jats:bold>\n In glaucoma, AI model accuracies ranged from 54.5% to 63.6%, with some models slightly surpassing the human expert average of 51.5%, underscoring the diagnostic difficulty of this condition.\n <\/jats:p>\n <\/jats:list-item>\n \n \n What has been learned and future implications?<\/jats:bold>\n These findings highlight the potential of AI as a valuable adjunct to clinical judgment in ophthalmology, although further research and the integration of multimodal data are essential to optimize these tools for routine clinical practice.\n <\/jats:p>\n <\/jats:list-item>\n <\/jats:list>\n <\/jats:sec>","DOI":"10.1101\/2025.03.14.25323836","type":"posted-content","created":{"date-parts":[[2025,3,20]],"date-time":"2025-03-20T06:38:36Z","timestamp":1742452716000},"source":"Crossref","is-referenced-by-count":16,"title":["Performance of DeepSeek, Qwen 2.5 MAX, and ChatGPT Assisting in Diagnosis of Corneal Eye Diseases, Glaucoma, and Neuro-Ophthalmology Diseases Based on Clinical Case Reports"],"prefix":"10.64898","author":[{"given":"Zain S.","family":"Hussain","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammad","family":"Delsoz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammad","family":"Elahi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian","family":"Jerkins","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elliot","family":"Kanner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claire","family":"Wright","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wuqaas M.","family":"Munir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammad","family":"Soleimani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Djalilian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Priscilla A.","family":"Lao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph W.","family":"Fong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Malik Y.","family":"Kahook","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siamak","family":"Yousefi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2025032100483481000_2025.03.14.25323836v1.1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2014.3192"},{"key":"2025032100483481000_2025.03.14.25323836v1.2","doi-asserted-by":"publisher","DOI":"10.1016\/j.survophthal.2006.10.002"},{"key":"2025032100483481000_2025.03.14.25323836v1.3","doi-asserted-by":"publisher","DOI":"10.1016\/S0002-9394(99)00195-6"},{"key":"2025032100483481000_2025.03.14.25323836v1.4","doi-asserted-by":"publisher","DOI":"10.1097\/WNO.0000000000001214"},{"issue":"1","key":"2025032100483481000_2025.03.14.25323836v1.5","doi-asserted-by":"crossref","first-page":"21829","DOI":"10.1038\/s41598-024-72752-x","article-title":"A comparative evaluation of deep learning approaches for ophthalmology","volume":"14","year":"2024","journal-title":"Scientific Reports"},{"key":"2025032100483481000_2025.03.14.25323836v1.6","doi-asserted-by":"crossref","first-page":"1252295","DOI":"10.3389\/fcomp.2023.1252295","article-title":"Deep learning-based classification of eye diseases using Convolutional Neural Network for OCT images","volume":"5","year":"2024","journal-title":"Frontiers in Computer Science"},{"issue":"9","key":"2025032100483481000_2025.03.14.25323836v1.7","doi-asserted-by":"crossref","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and mitigation of the limitations of large language models in clinical decision-making","volume":"30","year":"2024","journal-title":"Nature medicine"},{"issue":"4","key":"2025032100483481000_2025.03.14.25323836v1.8","doi-asserted-by":"crossref","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings","volume":"3","year":"2023","journal-title":"Ophthalmology science"},{"issue":"8","key":"2025032100483481000_2025.03.14.25323836v1.9","doi-asserted-by":"crossref","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions","volume":"6","year":"2023","journal-title":"JAMA network open"},{"issue":"1","key":"2025032100483481000_2025.03.14.25323836v1.10","doi-asserted-by":"crossref","first-page":"100599","DOI":"10.1016\/j.xops.2024.100599","article-title":"Diagnosing glaucoma based on the ocular hypertension treatment study dataset using chat generative pre-trained transformer as a large language model","volume":"5","year":"2025","journal-title":"Ophthalmology Science"},{"key":"2025032100483481000_2025.03.14.25323836v1.11","doi-asserted-by":"crossref","unstructured":"Huang X , Raja H , Madadi Y , et al. Predicting glaucoma before onset using a large language model chatbot. American Journal of Ophthalmology. 2024;","DOI":"10.1016\/j.ajo.2024.06.035"},{"issue":"5","key":"2025032100483481000_2025.03.14.25323836v1.12","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1097\/ICO.0000000000003492","article-title":"Performance of ChatGPT in diagnosis of corneal eye diseases","volume":"43","year":"2024","journal-title":"Cornea"},{"key":"2025032100483481000_2025.03.14.25323836v1.13","doi-asserted-by":"crossref","unstructured":"Madadi Y , Delsoz M , Lao PA , et al. ChatGPT assisting diagnosis of neuro-ophthalmology diseases based on case reports. Journal of Neuro-Ophthalmology. 2022:10.1097.","DOI":"10.1101\/2023.09.13.23295508"},{"key":"2025032100483481000_2025.03.14.25323836v1.14","unstructured":"Gibney E. China\u2019s cheap, open AI model DeepSeek thrills scientists. Nature."},{"key":"2025032100483481000_2025.03.14.25323836v1.15","unstructured":"Team Q. Qwen2. 5: A party of foundation models. Qwen (Sept 2024) url: https:\/\/qwenlmgithubio\/blog\/qwen2. 2024;5"},{"issue":"6","key":"2025032100483481000_2025.03.14.25323836v1.16","doi-asserted-by":"crossref","first-page":"1817","DOI":"10.1007\/s40123-024-00937-8","article-title":"A Response to: Letter to the Editor Regarding \u201cThe Use of ChatGPT to Assist in Diagnosing Glaucoma Based on Clinical Case Reports.\u201d","volume":"13","year":"2024","journal-title":"Ophthalmology and Therapy"},{"issue":"6","key":"2025032100483481000_2025.03.14.25323836v1.17","doi-asserted-by":"crossref","first-page":"192","DOI":"10.3390\/fi15060192","article-title":"Chatgpt and open-ai models: A preliminary review","volume":"15","year":"2023","journal-title":"Future Internet"},{"issue":"6","key":"2025032100483481000_2025.03.14.25323836v1.18","doi-asserted-by":"crossref","first-page":"3121","DOI":"10.1007\/s40123-023-00805-x","article-title":"The use of ChatGPT to assist in diagnosing glaucoma based on clinical case reports","volume":"12","year":"2023","journal-title":"Ophthalmology and Therapy"},{"key":"2025032100483481000_2025.03.14.25323836v1.19","unstructured":"Singh R , Bengani V , Saini K. HYBRID LEARNING SYSTEMS: INTEGRATING TRADITIONAL MACHINE LEARNING WITH DEEP LEARNING TECHNIQUES."},{"key":"2025032100483481000_2025.03.14.25323836v1.20","doi-asserted-by":"crossref","first-page":"baaa010","DOI":"10.1093\/database\/baaa010","article-title":"Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine","volume":"2020","year":"2020","journal-title":"Database"},{"key":"2025032100483481000_2025.03.14.25323836v1.21","doi-asserted-by":"crossref","unstructured":"Li J , Guan Z , Wang J , et al. Integrated image-based deep learning and language models for primary diabetes care. Nature medicine. 2024:1\u201311.","DOI":"10.3389\/fcvm.2024.1384977"},{"key":"2025032100483481000_2025.03.14.25323836v1.22","doi-asserted-by":"publisher","DOI":"10.1145\/3561048"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1101\/2025.03.14.25323836","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T20:30:56Z","timestamp":1768509056000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2025.03.14.25323836"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,17]]},"references-count":22,"URL":"https:\/\/doi.org\/10.1101\/2025.03.14.25323836","relation":{},"subject":[],"published":{"date-parts":[[2025,3,17]]},"subtype":"preprint"}}