{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:52:00Z","timestamp":1791525120025,"version":"4.3.4"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T00:00:00Z","timestamp":1791504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Patients increasingly use large language models (LLMs) to seek medical information, but their performance in answering diabetic retinopathy (DR) questions remains insufficiently characterized.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To evaluate the safety, accuracy, patient-centered communication, information quality, transparency, and readability of diabetic retinopathy information generated by widely accessible large language models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Fifty standardized DR patient questions were submitted to ChatGPT, Gemini, DeepSeek, Doubao, Copilot, and Perplexity. Responses were independently evaluated by five raters using guideline-based reference answer elements. Rater-based outcomes included Safety, Accuracy, Empathy, DISCERN, EQIP, JAMA benchmark criteria, and Global Quality Score (GQS). Readability was assessed using the Automated Readability Index, Coleman\u2013Liau Index, Flesch\u2013Kincaid Grade Level, Gunning Fog Index, SMOG, and Flesch Reading Ease Score. Inter-rater agreement and between-model differences were analyzed using appropriate agreement statistics and paired nonparametric tests.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Inter-rater agreement was moderate for Safety (Fleiss\u2019 \u03ba\u202f=\u202f0.581) and good for non-binary outcomes, with ICC (2,1) values ranging from 0.806 to 0.870. Most responses were classified as safe, with safe response rates ranging from 86.0 to 94.0%, and no significant between-model difference was observed. Accuracy scores were generally high across models, although between-model differences were statistically significant with a small effect size. Empathy varied more noticeably, with Copilot scoring highest and Doubao and Perplexity scoring lower. Perplexity achieved the highest DISCERN and JAMA scores, while Copilot performed best on EQIP. GQS differences were small. Readability differed significantly across models, with Gemini generally producing the most readable responses; however, all models exceeded the recommended sixth-grade reading level, and all Flesch Reading Ease Scores were below the recommended threshold.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>No single LLM performed best across all domains. Although the evaluated models generally provided safe and accurate responses to DR patient questions, limitations remained in safety escalation, empathy, transparency, and readability. LLM-generated DR information may support general patient education but should not replace professional ophthalmic consultation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1945318","type":"journal-article","created":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:33:27Z","timestamp":1791524007000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Large language models as sources of diabetic retinopathy patient education: a comparative evaluation of safety, health information quality, and readability"],"prefix":"10.3389","volume":"14","author":[{"given":"Zhiqi","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wen","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tian","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qing","family":"Bao","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pingting","family":"Ji","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongmei","family":"Yao","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"He","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Zhongnan Hospital of Wuhan University","place":["Wuhan, Hubei, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,9]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"1580","DOI":"10.1016\/j.ophtha.2021.04.027","article-title":"Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis","volume":"128","author":"Teo","year":"2021","journal-title":"Ophthalmology"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"S261","DOI":"10.2337\/dc26-S012","article-title":"12. Retinopathy, neuropathy, and foot care: standards of care in diabetes\u20142026","volume":"49","year":"2026","journal-title":"Diabetes Care"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"P75","DOI":"10.1016\/j.ophtha.2024.12.020","article-title":"Diabetic retinopathy preferred practice pattern\u00ae","volume":"132","author":"Lim","year":"2025","journal-title":"Ophthalmology"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1740","DOI":"10.3390\/healthcare9121740","article-title":"Online health information seeking behavior: a systematic review","volume":"9","author":"Jia","year":"2021","journal-title":"Healthcare (Basel)"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1016\/j.oret.2022.03.015","article-title":"Readability and accountability of online patient education materials for common retinal diseases","volume":"6","author":"Cohen","year":"2022","journal-title":"Ophthalmol Retina"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: a 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J Patient Exp"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"pkad015","DOI":"10.1093\/jncics\/pkad015","article-title":"Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information","volume":"7","author":"Johnson","year":"2023","journal-title":"JNCI Cancer Spectr"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"e244630","DOI":"10.1001\/jamanetworkopen.2024.4630","article-title":"Quality of large language model responses to radiation oncology patient care questions","volume":"7","author":"Yalamanchili","year":"2024","journal-title":"JAMA Netw Open"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"e58041","DOI":"10.2196\/58041","article-title":"Enhancement of the performance of large language models in diabetes education through retrieval-augmented generation: comparative study","volume":"26","author":"Wang","year":"2024","journal-title":"J Med Internet Res"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"104770","DOI":"10.1016\/j.ebiom.2023.104770","article-title":"Benchmarking large language models\u2019 performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google bard","volume":"95","author":"Lim","year":"2023","journal-title":"EBioMedicine"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","article-title":"Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases","volume":"7","author":"Momenaei","year":"2023","journal-title":"Ophthalmol Retina"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"20552076241277021","DOI":"10.1177\/20552076241277021","article-title":"Evaluation of the quality and readability of ChatGPT responses to frequently asked questions about myopia in traditional Chinese language","volume":"10","author":"Chang","year":"2024","journal-title":"Digit Health"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1136\/bjo-2024-325861","article-title":"Enhancing diabetic retinopathy query responses: assessing large language model in ophthalmology","volume":"109","author":"Wu","year":"2025","journal-title":"Br J Ophthalmol"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e2530220","DOI":"10.1001\/jamanetworkopen.2025.30220","article-title":"Reporting guideline for chatbot health advice studies: the CHART statement","volume":"8","year":"2025","journal-title":"JAMA Netw Open"},{"key":"ref16","year":"2024"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"923","DOI":"10.1016\/j.eprac.2022.08.002","article-title":"American Association of Clinical Endocrinology Clinical Practice Guideline: developing a diabetes mellitus comprehensive care plan\u20142022 update","volume":"28","author":"Blonde","year":"2022","journal-title":"Endocr Pract"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"99","DOI":"10.3760\/cma.j.cn511434-20230110-00018","article-title":"Evidence-based guidelines for diagnosis and treatment of diabetic retinopathy in China (2022)","volume":"39","year":"2023","journal-title":"Chin J Ocul Fundus Dis"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"101406","DOI":"10.1016\/j.preteyeres.2025.101406","article-title":"International consensuses and controversies on causes, diagnosis and management of diabetic macular edema (DME)","volume":"109","author":"Ng","year":"2025","journal-title":"Prog Retin Eye Res"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"100262","DOI":"10.1016\/j.apjo.2025.100262","article-title":"International consensuses and guidelines on management of proliferative diabetic retinopathy (PDR) by the Asia-Pacific Vitreo-retina society (APVRS), the academy of Asia-Pacific professors of ophthalmology (AAPPO), and the academia retina Internationalis (ARI)","volume":"14","author":"Lam","year":"2025","journal-title":"Asia Pac J Ophthalmol"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1177\/2474126419893829","article-title":"American Society of Retina Specialists clinical practice guidelines: management of nonproliferative and proliferative diabetic retinopathy without diabetic macular edema","volume":"4","author":"Yonekawa","year":"2020","journal-title":"J Vitreoretin Dis"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1089\/tmj.2020.0006","article-title":"Practice guidelines for ocular telehealth-diabetic retinopathy, third edition","volume":"26","author":"Horton","year":"2020","journal-title":"Telemed J E Health"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"1608","DOI":"10.1016\/j.ophtha.2018.04.007","article-title":"Guidelines on diabetic eye care: the International Council of Ophthalmology recommendations for screening, follow-up, referral, and treatment based on resource settings","volume":"125","author":"Wong","year":"2018","journal-title":"Ophthalmology"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1796","DOI":"10.1001\/archopht.1985.01050120030015","article-title":"Photocoagulation for diabetic macular edema: early treatment diabetic retinopathy study report number 1","volume":"103","year":"1985","journal-title":"Arch Ophthalmol"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1016\/S0161-6420(81)34978-1","article-title":"Photocoagulation treatment of proliferative diabetic retinopathy: clinical application of diabetic retinopathy study findings, DRS report number 8","volume":"88","year":"1981","journal-title":"Ophthalmology"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"1193","DOI":"10.1056\/NEJMoa1414264","article-title":"Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema","volume":"372","author":"Wells","year":"2015","journal-title":"N Engl J Med"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.1016\/j.ophtha.2016.02.022","article-title":"Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema: two-year results from a comparative effectiveness randomized clinical trial","volume":"123","author":"Wells","year":"2016","journal-title":"Ophthalmology"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"2137","DOI":"10.1001\/jama.2015.15217","article-title":"Panretinal photocoagulation vs intravitreous ranibizumab for proliferative diabetic retinopathy: a randomized clinical trial","volume":"314","author":"Gross","year":"2015","journal-title":"JAMA"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1138","DOI":"10.1001\/jamaophthalmol.2018.3255","article-title":"Five-year outcomes of panretinal photocoagulation vs intravitreous ranibizumab for proliferative diabetic retinopathy: a randomized clinical trial","volume":"136","author":"Gross","year":"2018","journal-title":"JAMA Ophthalmol"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1001\/jamaophthalmol.2021.2809","article-title":"Evaluation of intravitreal aflibercept for the treatment of severe nonproliferative diabetic retinopathy: results from the PANORAMA randomized clinical trial","volume":"139","author":"Brown","year":"2021","journal-title":"JAMA Ophthalmol"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1016\/j.ophtha.2011.12.039","article-title":"Ranibizumab for diabetic macular edema: results from 2 phase III randomized trials: RISE and RIDE","volume":"119","author":"Nguyen","year":"2012","journal-title":"Ophthalmology"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"2247","DOI":"10.1016\/j.ophtha.2014.05.006","article-title":"Intravitreal aflibercept for diabetic macular edema","volume":"121","author":"Korobelnik","year":"2014","journal-title":"Ophthalmology"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices","volume":"1","author":"Abr\u00e0moff","year":"2018","journal-title":"NPJ Digit Med"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.2337\/dci23-0032","article-title":"Artificial intelligence and diabetic retinopathy: AI framework, prospective studies, head-to-head validation, and cost-effectiveness","volume":"46","author":"Rajesh","year":"2023","journal-title":"Diabetes Care"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor\u2014let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the world wide web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol"},{"key":"ref39","year":""},{"key":"ref40","volume-title":"Health Literacy and patient Safety: help Patients Understand. Manual for Clinicians","author":"Weiss","year":"2007"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"e18161","DOI":"10.5812\/atr.18161","article-title":"Readability of patient education materials on the American Association for Surgery of trauma website","volume":"3","author":"Eltorai","year":"2014","journal-title":"Arch Trauma Res"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"e76947","DOI":"10.2196\/76947","article-title":"Evaluating large language models in ophthalmology: systematic review","volume":"27","author":"Zhang","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref43","doi-asserted-by":"publisher","first-page":"1828810","DOI":"10.3389\/fcvm.2026.1828810","article-title":"Cross-platform evaluation of LLM-generated educational texts on cardiac myxoma: quality, readability, and actionability using network analysis and latent profile analysis","volume":"13","author":"Deng","year":"2026","journal-title":"Front Cardiovasc Med"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"1801829","DOI":"10.3389\/fpubh.2026.1801829","article-title":"Challenges of using generative AI for patient education in chronic heart failure: an evaluation of content quality, readability, and actionability in cross-platform LLM-generated texts","volume":"14","author":"Wang","year":"2026","journal-title":"Front Public Health"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"20552076261464724","DOI":"10.1177\/20552076261464724","article-title":"Challenges of patient-facing generative artificial intelligence in hypertension care: a cross-platform evaluation of the quality, readability, and actionability of LLM-generated patient education materials","volume":"12","author":"Hu","year":"2026","journal-title":"Digit Health"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"20556","DOI":"10.1038\/s41598-025-06404-z","article-title":"Large language models provide discordant information compared to ophthalmology guidelines","volume":"15","author":"Taloni","year":"2025","journal-title":"Sci Rep"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1038\/s41746-026-02428-5","article-title":"Large language models provide unsafe answers to patient-posed medical questions","volume":"9","author":"Draelos","year":"2026","journal-title":"NPJ Digit Med"},{"key":"ref48","doi-asserted-by":"publisher","first-page":"2511.15943","DOI":"10.48550\/arXiv.2511.15943","article-title":"Boosting medical visual understanding from multi-granular language learning","author":"Li","year":"2025","journal-title":"arXiv"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"11848","DOI":"10.1109\/TPAMI.2025.3598734","article-title":"VisionUnite: a vision-language foundation model for ophthalmology enhanced with clinical knowledge","volume":"47","author":"Li","year":"2025","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"ref50","doi-asserted-by":"publisher","first-page":"e112695","DOI":"10.1136\/bmjopen-2025-112695","article-title":"Generative artificial intelligence-driven chatbots and medical misinformation: an accuracy, referencing and readability audit","volume":"16","author":"Tiller","year":"2026","journal-title":"BMJ Open"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1001\/jamaophthalmol.2019.3116","article-title":"Assessment of the quality, content, and readability of freely available online information for patients regarding diabetic retinopathy","volume":"137","author":"Kloosterboer","year":"2019","journal-title":"JAMA Ophthalmol"},{"key":"ref52","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"Bedi","year":"2025","journal-title":"JAMA"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1945318\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:33:29Z","timestamp":1791524009000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1945318\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,9]]},"references-count":52,"alternative-id":["10.3389\/fpubh.2026.1945318"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1945318","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,9]]},"article-number":"1945318"}}