{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:22Z","timestamp":1785545002808,"version":"3.56.0"},"reference-count":31,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Artificial intelligence\u2013generated health information is increasingly used by patients, but its reliability, visible transparency indicators, and readability remain uncertain in specialized ophthalmic conditions such as age-related macular degeneration (AMD).<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate and compare the informational reliability, visible transparency indicators, overall quality, and readability of responses generated by 6 publicly accessible large language models (LLMs) to AMD-related patient-facing prompts under a zero-shot, single-turn prompting scenario.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Thirty English-language AMD-related prompts were curated from Google Trends, the 2023 Chinese AMD guideline, and the 2025 American Academy of Ophthalmology Preferred Practice Pattern. Chinese guideline\u2013derived prompts were translated and reviewed before model querying. Each finalized prompt was entered verbatim into ChatGPT-5.1-auto, DeepSeek-v3.2, Gemini-2.5-Flash-Thinking, Grok 4, Claude-Sonnet 4.5, and Qwen3-Max between October 10 and November 25, 2025. Two senior ophthalmologists (ZL and XM) blinded to model identity independently scored all responses using DISCERN, Ensuring Quality Information for Patients (EQIP), Global Quality Scale, and Journal of the American Medical Association benchmark criteria, with adjudication for disagreements. Readability was assessed using 6 standard formulas against a sixth-grade benchmark. Between-model differences were analyzed using Friedman tests with Holm-adjusted pairwise comparisons.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n A total of 180 responses were analyzed. Interrater agreement was substantial to near-perfect across reliability instruments (\u03ba=0.72\u20100.97). No model met the recommended sixth-grade readability target. Grok 4 achieved the highest scores on reliability-related instruments, including DISCERN (mean 46.40, SD 7.43) and EQIP (mean 74.33, SD 9.07), whereas DeepSeek-v3.2 generated the most readable responses, with the highest Flesch Reading Ease Score (mean 48.23, SD 9.16) and lowest Flesch-Kincaid Grade Level (mean 9.95, SD 1.87). Significant between-model differences were observed across all reliability and readability metrics (all\n P<\/jats:italic>\n <.001).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Under zero-shot, single-turn prompting conditions, the evaluated public LLMs showed substantial model-dependent differences in AMD-related patient education quality and readability. No model met the sixth-grade readability benchmark, including those with comparatively stronger reliability performance. These findings support clinician oversight, readability optimization, and further evaluation before LLM-generated AMD information is used directly in patient-facing settings.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/91016","type":"journal-article","created":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T14:35:07Z","timestamp":1781793307000},"page":"e91016-e91016","source":"Crossref","is-referenced-by-count":0,"title":["Mapping the Reliability-Readability Gap in the Education of Patients With Age-Related Macular Degeneration Across 6 Large Language Models: Comparative Evaluation Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-9723-5752","authenticated-orcid":false,"given":"Zhili","family":"Lu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9143-589X","authenticated-orcid":false,"given":"Haixing","family":"Cao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7094-0178","authenticated-orcid":false,"given":"Cong","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5837-529X","authenticated-orcid":false,"given":"Jin","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-4901-4294","authenticated-orcid":false,"given":"Xiang","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,28]]},"reference":[{"issue":"4","key":"R1","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1016\/j.mayocpiqo.2024.05.003","article-title":"Recent updates on the diagnosis and management of age-related macular degeneration","volume":"8","author":"Boopathiraj","journal-title":"Mayo Clin Proc Innov Qual Outcomes"},{"issue":"2","key":"R2","doi-asserted-by":"publisher","first-page":"e106","DOI":"10.1016\/S2214-109X(13)70145-1","article-title":"Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis","volume":"2","author":"Wong","journal-title":"Lancet Glob Health"},{"issue":"10","key":"R3","doi-asserted-by":"publisher","first-page":"2023","DOI":"10.1038\/s41433-025-03788-0","article-title":"ChatGPT-4 for addressing patient-centred frequently asked questions in age-related macular degeneration clinical practice","volume":"39","author":"Wang","journal-title":"Eye (Lond)"},{"issue":"19","key":"R4","doi-asserted-by":"publisher","DOI":"10.3390\/jcm14196968","article-title":"Enhancing patient education with AI: a readability analysis of AI-generated versus American Academy of ophthalmology online patient education materials","volume":"14","author":"Kufta","journal-title":"J Clin Med"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","DOI":"10.1186\/s12886-020-01664-x","article-title":"Assessment of online patient education materials designed for people with age-related macular degeneration","volume":"20","author":"Fortuna","journal-title":"BMC Ophthalmol"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/69955","article-title":"Enhancing the readability of online patient education materials using large language models: cross-sectional study","volume":"27","author":"Will","journal-title":"J Med Internet Res"},{"issue":"2","key":"R7","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1097\/IOP.0000000000002549","article-title":"Optimizing ophthalmology patient education via ChatBot-generated materials: readability analysis of AI-generated patient education materials and the American Society of Ophthalmic Plastic and Reconstructive Surgery patient brochures","volume":"40","author":"Eid","journal-title":"Ophthalmic Plast Reconstr Surg"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","journal-title":"Front Med (Lausanne)"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1001\/jamaophthalmol.2014.6104","article-title":"Assessment of online patient education materials from major ophthalmologic associations","volume":"133","author":"Huang","journal-title":"JAMA Ophthalmol"},{"issue":"8","key":"R10","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1001\/jamadermatol.2016.1135","article-title":"Patient education materials in dermatology: addressing the health literacy needs of patients","volume":"152","author":"Prabhu","journal-title":"JAMA Dermatol"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1080\/01658107.2024.2301728","article-title":"Assessment of the quality, accountability, and readability of online patient education materials for optic neuritis","volume":"48","author":"Patel","journal-title":"Neuroophthalmology"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/s12886-016-0315-0","article-title":"Readability of patient education materials in ophthalmology: a single-institution study and systematic review","volume":"16","author":"Williams","journal-title":"BMC Ophthalmol"},{"issue":"5","key":"R13","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1016\/j.ogla.2022.03.004","article-title":"Readability and suitability of online patient education materials for glaucoma","volume":"5","author":"Martin","journal-title":"Ophthalmol Glaucoma"},{"issue":"8","key":"R14","doi-asserted-by":"publisher","first-page":"2546","DOI":"10.1038\/s41591-025-03727-2","article-title":"Benchmark evaluation of DeepSeek large language models in clinical decision-making","volume":"31","author":"Sandmann","journal-title":"Nat Med"},{"issue":"5","key":"R15","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooaf109","article-title":"Leveraging open-source large language models for clinical information extraction in resource-constrained settings","volume":"8","author":"Builtjes","journal-title":"JAMIA Open"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/55318","article-title":"An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: algorithm development and validation study","volume":"12","author":"Sivarajkumar","journal-title":"JMIR Med Inform"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.3760\/cma.j.cn112142-20221222-00649","article-title":"Evidence-based guidelines for diagnosis and treatment of age-related macular degeneration in China (2023)","volume":"59","author":"Chinese Vitreo-Retina Society of Chinese Medical Association; Fundus Disease Group of Chinese Ophthalmologist Association","journal-title":"Zhonghua Yan Ke Za Zhi"},{"issue":"4","key":"R18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ophtha.2024.12.018","article-title":"Age-related macular degeneration preferred practice pattern\u00ae","volume":"132","author":"Vemulakonda","journal-title":"Ophthalmology"},{"issue":"2","key":"R19","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","journal-title":"J Epidemiol Community Health"},{"issue":"1-2","key":"R20","doi-asserted-by":"publisher","first-page":"e31","DOI":"10.1016\/j.wneu.2012.11.003","article-title":"Evaluation of the quality of information on the Internet available to patients undergoing cervical spine surgery","volume":"82","author":"Weil","journal-title":"World Neurosurg"},{"issue":"2","key":"R21","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","journal-title":"Health Expect"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1046\/j.1369-6513.2002.00189.x","article-title":"Improving the quality of health information: the contribution of C-H-i-Q","volume":"5","author":"Hain","journal-title":"Health Expect"},{"issue":"15","key":"R23","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","volume":"277","author":"Silberg","journal-title":"JAMA"},{"key":"R24","author":"Smith","journal-title":"AMRL TR"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-025-02178-z","article-title":"AI Chatbots as sources of STD information: a study on reliability and readability","volume":"49","author":"Y\u0131ld\u0131z","journal-title":"J Med Syst"},{"issue":"2","key":"R26","doi-asserted-by":"publisher","DOI":"10.5812\/atr.18161","article-title":"Readability of patient education materials on the American Association for Surgery of Trauma website","volume":"3","author":"Eltorai","journal-title":"Arch Trauma Res"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","first-page":"1051","DOI":"10.1111\/jgs.16960","article-title":"Assessment of the readability of online patient education material from major geriatric associations","volume":"69","author":"van Ballegooie","journal-title":"J Am Geriatr Soc"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1111\/imj.16607","article-title":"Can artificial intelligence improve patient educational material readability? A systematic review and narrative synthesis","volume":"55","author":"Nasra","journal-title":"Intern Med J"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/73469","article-title":"Evaluating the performance of DeepSeek-R1 and DeepSeek-V3 versus OpenAI models in the Chinese National Medical Licensing Examination: cross-sectional comparative study","volume":"11","author":"Wang","journal-title":"JMIR Med Educ"},{"issue":"9","key":"R30","doi-asserted-by":"publisher","DOI":"10.3390\/clinpract15090169","article-title":"Artificial intelligence in clinical medicine: challenges across diagnostic imaging, clinical decision support, surgery, pathology, and drug discovery","volume":"15","author":"Ogut","journal-title":"Clin Pract"},{"issue":"6","key":"R31","doi-asserted-by":"publisher","first-page":"1436","DOI":"10.1093\/jamia\/ocad258","article-title":"Efficient healthcare with large language models: optimizing clinical workflow and enhancing patient care","volume":"31","author":"Tripathi","journal-title":"J Am Med Inform Assoc"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e91016","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:24:15Z","timestamp":1785536655000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e91016"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,28]]},"references-count":31,"URL":"https:\/\/doi.org\/10.2196\/91016","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,28]]},"article-number":"v14i4e91016"}}