{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T12:25:51Z","timestamp":1777551951523,"version":"3.51.4"},"reference-count":34,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T00:00:00Z","timestamp":1769904000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T00:00:00Z","timestamp":1769904000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["DIGITAL HEALTH"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>The aim of this study was to comparatively evaluate four large language models (LLMs) used for patient education in ophthalmology in terms of accuracy, reliability, and patient safety across different ophthalmic subspecialties.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this cross-sectional evaluation, a total of 50 frequently asked patient questions covering five ophthalmic subspecialties (strabismus\/pediatric ophthalmology, oculoplastics, cataract and refractive surgery, retina, and dry eye) were included. All questions were submitted in a text-only format to ChatGPT o3 Mini High, Gemini 2.0 Pro, Claude-Sonnet 3.7, and LLaMA 3.1 405B. The generated responses were independently evaluated by five blinded ophthalmologists using a 10-point scale assessing accuracy, currency, informativeness\/clarity, and patient safety. Potentially unsafe content was identified and categorized using a predefined structured error taxonomy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Marked differences in performance were observed among the models. Mean scores were 3.44 for Gemini, 2.99 for ChatGPT, 2.48 for Claude, and 1.09 for LLaMA. Gemini demonstrated higher performance across most subspecialties, whereas in the retina subspecialty, ChatGPT and Claude generated comparatively stronger responses. Of the 200 evaluated responses, 19 (9.5%) contained potentially unsafe content, with the lowest proportion observed for Gemini and the highest for LLaMA.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>LLMs can generate useful responses for patient education in ophthalmology, but performance varies by model and subspecialty. Within this 50-question, text-only expert-rating framework, Gemini 2.0 Pro and ChatGPT o3 Mini High provided relatively higher accuracy and reliability in most areas, whereas LLaMA 3.1 405B lagged. Larger and clinically integrated evaluations, including direct assessment of patient understanding and behavior, are needed to define their safe use in practice.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/20552076261433657","type":"journal-article","created":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T14:05:24Z","timestamp":1775657124000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Digital guides in eye care: Comparing AI model accuracy and reliability"],"prefix":"10.1177","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3281-9892","authenticated-orcid":false,"given":"Hakan Veli","family":"Sava\u015f","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, Karako\u00e7an State Hospital, Karako\u00e7an, Elaz\u0131\u011f, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3989-2432","authenticated-orcid":false,"given":"Osman","family":"Altay","sequence":"additional","affiliation":[{"name":"Manisa Celal Bayar University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2026,4,8]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.70572"},{"key":"e_1_3_3_3_2","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2020-037648","article-title":"Eye health and quality of life: an umbrella review protocol","volume":"10","author":"Assi L","year":"2020","unstructured":"Assi L, Rosman L, Chamseddine F, et al. Eye health and quality of life: an umbrella review protocol. BMJ Open. 2020; 10: e037648.","journal-title":"BMJ Open"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780198802907.003.0016"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pec.2018.06.016"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph182111248"},{"key":"e_1_3_3_7_2","doi-asserted-by":"crossref","unstructured":"Worrall AP Connolly MJ O\u2019neill A et al. BMC public health. DOI: 10.1186\/s12889-020-09710-5.","DOI":"10.1186\/s12889-020-09710-5"},{"key":"e_1_3_3_8_2","first-page":"47\u201357.","article-title":"The role of artificial intelligence in the development of the company\u2019s business processes","author":"Kvitka A","year":"2024","unstructured":"Kvitka A, Sosnin D, Kvitka Y, et al. The role of artificial intelligence in the development of the company\u2019s business processes. FU Econ Org 2024; 21: 47\u201357.","journal-title":"FU Econ Org"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1159\/000250399"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1093\/acrefore\/9780190224851.013.434"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.18331\/BRJ2023.10.1.4"},{"key":"e_1_3_3_12_2","doi-asserted-by":"crossref","DOI":"10.4108\/eetpht.9.4713","article-title":"Artificial intelligence in medical filed","volume":"9","author":"Fatima I","year":"2023","unstructured":"Fatima I, Grover V, Khan IR, et al. Artificial intelligence in medical filed. EAI Endorsed Trans Pervasive Health Technol 2023; 9: 1\u20135.","journal-title":"EAI Endorsed Trans Pervasive Health Technol"},{"key":"e_1_3_3_13_2","doi-asserted-by":"crossref","unstructured":"Reinhart WF Statt A. Large language models design sequence-defined macromolecules via evolutionary optimization. Epub ahead of print 4 September 2024. DOI: 10.26434\/CHEMRXIV-2024-9146H.","DOI":"10.26434\/chemrxiv-2024-9146h"},{"key":"e_1_3_3_14_2","article-title":"Application of large language models in healthcare: a bibliometric analysis","volume":"11","author":"Zhang L","year":"2025","unstructured":"Zhang L, Zhao Q, Zhang D, et al. Application of large language models in healthcare: a bibliometric analysis. Digital Health 2025; 11: 20552076251324444.","journal-title":"Digital Health"},{"issue":"1169595","key":"e_1_3_3_15_2","article-title":"ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations","volume":"6","author":"Dave T","year":"2023","unstructured":"Dave T, Athaluri SA , Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell 2023; 6: 1169595.","journal-title":"Front Artif Intell"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.2196\/68560"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.31662\/jmaj.2024-0139"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2025.1530799"},{"key":"e_1_3_3_19_2","article-title":"Evaluating ChatGPT\u2019s accuracy and readability in responding to common ophthalmology questions","volume":"17","author":"Riazi Esfahani P","year":"2025","unstructured":"Riazi Esfahani P, Ward J, Yong A, et al. Evaluating ChatGPT\u2019s accuracy and readability in responding to common ophthalmology questions. Cureus 2025; 17: e87920.","journal-title":"Cureus"},{"key":"e_1_3_3_20_2","article-title":"Accuracy and readability of ChatGPT-4o in answering ophthalmology patient questions","author":"Hamzeh N","unstructured":"Hamzeh N, Lidder AK, Feder RS, et al. Accuracy and readability of ChatGPT-4o in answering ophthalmology patient questions. Ophthalmology Science 2026; 6: 101007.","journal-title":"Ophthalmology Science"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infoh.2025.07.002"},{"key":"e_1_3_3_22_2","article-title":"Disparity and factors associated with internet health information seeking among US adults living with diabetes mellitus: cross-sectional study","volume":"24","author":"Eke R","year":"2022","unstructured":"Eke R, Jannat KT, Yang XT, et al. Disparity and factors associated with internet health information seeking among US adults living with diabetes mellitus: cross-sectional study. J Med Internet Res 2022; 24: e32723.","journal-title":"J Med Internet Res"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0000000000004963"},{"key":"e_1_3_3_24_2","doi-asserted-by":"crossref","DOI":"10.2196\/41138","article-title":"Artificial intelligence in medicine: text mining of health care workers\u2019 opinions","volume":"25","author":"Niti\u00e9ma P","year":"2023","unstructured":"Niti\u00e9ma P. Artificial intelligence in medicine: text mining of health care workers\u2019 opinions. J Med Internet Res 2023; 25:e41138.","journal-title":"J Med Internet Res"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.30320"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.5005\/jp-journals-10078-1448"},{"key":"e_1_3_3_28_2","article-title":"Assessing the competence of artificial intelligence programs in pediatric ophthalmology and strabismus and comparing their relative advantages","volume":"67","author":"Sensoy E","year":"2023","unstructured":"Sensoy E, Citirik M. Assessing the competence of artificial intelligence programs in pediatric ophthalmology and strabismus and comparing their relative advantages. Rom J Ophthalmol 2023; 67: 394\u2013397.","journal-title":"Rom J Ophthalmol"},{"key":"e_1_3_3_29_2","doi-asserted-by":"crossref","unstructured":"Chiu EK-Y Sridhar S Wong SS-Y et al. Generative artificial intelligence models in clinical infectious disease consultations: a cross-sectional analysis among specialists and resident trainees. Epub ahead of print 19 August 2024. DOI: 10.1101\/2024.08.15.24312054.","DOI":"10.3390\/healthcare13070744"},{"key":"e_1_3_3_30_2","article-title":"Gemini AI vs. ChatGPT: a comprehensive examination alongside ophthalmology residents in medical knowledge","volume":"263","author":"Bahir D","year":"2025","unstructured":"Bahir D, Zur O, Attal L, et al. Gemini AI vs. ChatGPT: a comprehensive examination alongside ophthalmology residents in medical knowledge. Graefes Arch Clin Exp Ophthalmol 2025; 263: 527\u2013536.","journal-title":"Graefes Arch Clin Exp Ophthalmol"},{"key":"e_1_3_3_31_2","first-page":"1412","article-title":"Google DeepMind\u2019s Gemini AI versus ChatGPT: a comparative analysis in ophthalmology","volume":"38","author":"Masalkhi M","year":"2024","unstructured":"Masalkhi M, Ong J, Waisberg E, et al. Google DeepMind\u2019s Gemini AI versus ChatGPT: a comparative analysis in ophthalmology. Eye (Basingstoke) 2024; 38: 1412\u20131417.","journal-title":"Eye (Basingstoke)"},{"key":"e_1_3_3_32_2","doi-asserted-by":"crossref","unstructured":"Casagrande D Gobira M. Evaluating the accuracy of Gemini 2.0 advanced and ChatGPT 4o in cataract knowledge: a performance analysis using Brazilian Council of Ophthalmology Board exam questions. Epub ahead of print 2025. DOI: 10.7759\/cureus.79565.","DOI":"10.7759\/cureus.79565"},{"key":"e_1_3_3_33_2","article-title":"Application of large language models in healthcare: a bibliometric analysis","author":"Zhang L","unstructured":"Zhang L, Zhao Q, Zhang D, et al. Application of large language models in healthcare: a bibliometric analysis. Digit Health 2025; 11: 20552076251324444.","journal-title":"Digit Health"},{"key":"e_1_3_3_34_2","first-page":"2530","article-title":"Google gemini and bard artificial intelligence chatbot performance in ophthalmology knowledge assessment","volume":"38","author":"Mihalache A","year":"2024","unstructured":"Mihalache A, Grad J, Patil NS, et al. Google gemini and bard artificial intelligence chatbot performance in ophthalmology knowledge assessment. Eye (Basingstoke) 2024; 38: 2530\u20132535.","journal-title":"Eye (Basingstoke)"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamaophthalmol.2023.1144"}],"container-title":["DIGITAL HEALTH"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20552076261433657","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20552076261433657","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20552076261433657","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T17:56:25Z","timestamp":1777398985000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20552076261433657"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2]]},"references-count":34,"alternative-id":["10.1177\/20552076261433657"],"URL":"https:\/\/doi.org\/10.1177\/20552076261433657","relation":{},"ISSN":["2055-2076","2055-2076"],"issn-type":[{"value":"2055-2076","type":"print"},{"value":"2055-2076","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2]]},"article-number":"20552076261433657"}}