{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T14:55:26Z","timestamp":1782917726525,"version":"3.54.5"},"reference-count":19,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T00:00:00Z","timestamp":1782864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"\n Purpose<\/jats:title>\n To evaluate how the quality of retina-specific ophthalmology counseling provided by an early publicly available large language model (LLM) differs when advising patients with varying clinical characteristics and risk factors.<\/jats:p>\n <\/jats:sec>\n \n Design<\/jats:title>\n Prospective, cross-sectional study.<\/jats:p>\n <\/jats:sec>\n \n Subjects<\/jats:title>\n 18 ophthalmologists.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Six patient vignettes were constructed with high- and low-urgency clinical scenarios for diabetic retinopathy (DR), retinal detachment (RD), and age-related macular degeneration (AMD). Based on these vignettes, an LLM (ChatGPT-3.5) was asked to provide written medical counseling in February 2024. Each response was rated on several metrics via 5-point Likert scale by 18 independent reviewers. Notably, readability was assessed both qualitatively via survey and quantitatively via Readable (an online readability tool that incorporates 5 different metrics).<\/jats:p>\n <\/jats:sec>\n \n Main outcome measures<\/jats:title>\n Counseling generated by the LLM was graded on accuracy, appropriate communication of urgency and empathy, readability, and potential for clinically significant harm.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Counseling accuracy differed across levels of clinical urgency (\n p<\/jats:italic>\n \u2009=\u20090.002) but remained consistent between high- and low-urgency vignettes of AMD (\n p<\/jats:italic>\n \u2009=\u20090.081) and DR (\n p<\/jats:italic>\n \u2009=\u20090.5), albeit not for RD (\n P<\/jats:italic>\n \u2009&lt;\u20090.001). Counseling urgency did not differ significantly from clinical urgency of all vignettes, except for the high-urgency AMD (\n p<\/jats:italic>\n \u2009=\u20090.013) and high-urgency RD (\n p<\/jats:italic>\n \u2009&lt;\u20090.001). While counseling urgency did not significantly differ between high- and low-urgency vignettes of AMD (\n p<\/jats:italic>\n \u2009=\u20090.055) and RD (\n p<\/jats:italic>\n \u2009=\u20090.3), it did differ for the DR vignettes (\n p<\/jats:italic>\n \u2009&lt;\u20090.001). Counseling empathy did not differ across clinical urgency (\n p<\/jats:italic>\n \u2009=\u20090.2). Four readability indices (e.g., Flesch Kincaid Grade Level) consistently indicated that college graduation would be required to understand every counseling output. Across all vignettes, the most common reasons for potential difficulty in understanding the counseling were having too much medical (49%) and non-medical (45%) terminology.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The evaluated LLM-generated counseling outputs were largely similar across the sampled retinal vignettes with differing clinical urgency. Future studies should investigate the optimization of LLM prompting needed to garner counseling of consistent\/appropriate accuracy, readability, empathy, and communication of urgency for specific conditions.<\/jats:p>\n <\/jats:sec>","DOI":"10.3389\/fdgth.2026.1849883","type":"journal-article","created":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T14:23:10Z","timestamp":1782915790000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Assessing retina-specific ophthalmic counseling generated by an early public large language model across different levels of clinical urgency"],"prefix":"10.3389","volume":"8","author":[{"given":"Dominic M.","family":"Choo","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, University of Texas Southwestern Medical Center","place":["Dallas, TX, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tyler A.","family":"Durham","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of Texas Southwestern Medical Center","place":["Dallas, TX, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kishan G.","family":"Patel","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of Texas Southwestern Medical Center","place":["Dallas, TX, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,7,1]]},"reference":[{"key":"B1","year":"2024"},{"key":"B2","author":"Fox","year":"2013"},{"key":"B3","doi-asserted-by":"publisher","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol Sci"},{"key":"B4","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1001\/jamaophthalmol.2023.6917","article-title":"Assessment of a large language Model's responses to questions and cases about glaucoma and retina management","volume":"142","author":"Huang","year":"2024","journal-title":"JAMA Ophthalmol"},{"key":"B5","year":""},{"key":"B6","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","article-title":"Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases","volume":"7","author":"Momenaei","year":"2023","journal-title":"Ophthalmol Retina"},{"key":"B7","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1038\/s41598-023-50884-w","article-title":"Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy","volume":"14","author":"Onder","year":"2024","journal-title":"Sci Rep"},{"key":"B8","doi-asserted-by":"publisher","first-page":"17234","DOI":"10.1038\/s41598-024-67293-2","article-title":"Readability analysis of ChatGPT's Responses on lung cancer","volume":"14","author":"Gencer","year":"2024","journal-title":"Sci Rep"},{"key":"B9","doi-asserted-by":"publisher","first-page":"570","DOI":"10.32614\/RJ-2021-053","article-title":"Reproducible summary tables with the gtsummary package","volume":"13","author":"Sjoberg","year":"2021","journal-title":"R J"},{"key":"B10","author":"Gamer","year":"2019"},{"key":"B11","author":"Revelle","year":"2024"},{"key":"B12","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"},{"key":"B13","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3121","DOI":"10.1007\/s40123-023-00805-x","article-title":"The use of ChatGPT to assist in diagnosing glaucoma based on clinical case reports","volume":"12","author":"Delsoz","year":"2023","journal-title":"Ophthalmol Ther"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1526","DOI":"10.1080\/09273948.2023.2253471","article-title":"Evaluating the diagnostic accuracy and management recommendations of ChatGPT in uveitis","volume":"32","author":"Rojas-Carabali","year":"2024","journal-title":"Ocul Immunol Inflamm"},{"key":"B16","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1016\/j.ajem.2024.02.008","article-title":"Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage?","volume":"79","author":"Zaboli","year":"2024","journal-title":"Am J Emerg Med"},{"key":"B17","doi-asserted-by":"publisher","first-page":"104144","DOI":"10.1016\/j.amjoto.2023.104144","article-title":"Risk stratification of thyroid nodules: assessing the suitability of ChatGPT for text-based analysis","volume":"45","author":"Sievert","year":"2024","journal-title":"Am J Otolaryngol"},{"key":"B18","doi-asserted-by":"publisher","first-page":"e0301854","DOI":"10.1371\/journal.pone.0301854","article-title":"ChatGPT provides inconsistent risk-stratification of patients with atraumatic chest pain","volume":"19","author":"Heston","year":"2024","journal-title":"PLoS One"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/09286586.2026.2651197","article-title":"Evaluating three large language models in ophthalmology education: a comparative study of accuracy and readability across 442 questions","volume":"33","author":"\u00c7akmak","year":"2026","journal-title":"Ophthalmic Epidemiol"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1849883\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T14:23:11Z","timestamp":1782915791000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1849883\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,1]]},"references-count":19,"alternative-id":["10.3389\/fdgth.2026.1849883"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1849883","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,1]]},"article-number":"1849883"}}