{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T17:21:31Z","timestamp":1791566491144,"version":"4.3.4"},"reference-count":28,"publisher":"BMJ","issue":"11","license":[{"start":{"date-parts":[[2025,6,30]],"date-time":"2025-06-30T00:00:00Z","timestamp":1751241600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Natural Science Foundation","award":["U20A20386"],"award-info":[{"award-number":["U20A20386"]}]},{"name":"Clinical Medical Research Center for Eye Diseases of Zhejiang Province","award":["2021E50007"],"award-info":[{"award-number":["2021E50007"]}]},{"name":"Research and Development Program of Zhejiang Province","award":["2024C03204"],"award-info":[{"award-number":["2024C03204"]}]},{"name":"Key Program of the National Natural Science Foundation of China","award":["82330032"],"award-info":[{"award-number":["82330032"]}]},{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["82201195"],"award-info":[{"award-number":["82201195"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["Br J Ophthalmol"],"accepted":{"date-parts":[[2025,4,24]]},"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Diabetic retinopathy (DR) is a leading cause of blindness, with an increasing reliance on large language models (LLMs) for health-related information. The specificity of LLM-generated responses to DR queries is yet to be established, prompting an investigation into their suitability for ophthalmological contexts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A cross-sectional study involving six LLMs was conducted to ascertain the accuracy and comprehensiveness of responses to 42 DR-related questions from 1 February 2024 to 31 March 2024. Three consultant-level ophthalmologists independently assessed the responses, grading them on accuracy and comprehensiveness. Additionally, the self-correction capability and readability of the responses were analysed statistically.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>An analysis of 252 responses from six LLMs showed an average word count ranging from 155.3 to 304.3 and an average character count ranging from 975.3 to 2043.5. The readability scores showed significant variability, with ChatGPT-3.5 displaying the lowest readability level. The accuracy of the responses was high, with ChatGPT-4.0 receiving 97.6% good ratings and no \u2018poor\u2019 grades for the top three models. After introducing a self-correction prompt, the average accuracy score demonstrated a significant improvement, increasing from 6.4 to 7.5.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs have the potential to provide accurate and comprehensive responses to DR-related questions, making them advantageous for ophthalmology applications. However, before clinical integration, further refinement is needed to address readability, and continuous validation assessments are imperative to ensure reliability.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bjo-2024-325861","type":"journal-article","created":{"date-parts":[[2025,6,30]],"date-time":"2025-06-30T13:20:20Z","timestamp":1751289620000},"page":"1272-1278","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":11,"title":["Enhancing diabetic retinopathy query responses: assessing large language model in ophthalmology"],"prefix":"10.1136","volume":"109","author":[{"given":"Hongkang","family":"Wu","sequence":"first","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zichang","family":"Su","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangji","family":"Pan","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"An","family":"Shao","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufeng","family":"Xu","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yao","family":"Wang","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4369-2417","authenticated-orcid":false,"given":"Kai","family":"Jin","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1948-2500","authenticated-orcid":false,"given":"Juan","family":"Ye","sequence":"additional","affiliation":[{"name":"Eye Center, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2025,6,30]]},"reference":[{"key":"2026012223102370000_109.11.1272.1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsr2214184"},{"key":"2026012223102370000_109.11.1272.2","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"2026012223102370000_109.11.1272.3","doi-asserted-by":"crossref","DOI":"10.1097\/MD.0000000000034068","article-title":"Assessing ChatGPT\u2019s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis","volume":"102","author":"Kao","year":"2023","journal-title":"Medicine (Baltimore)"},{"key":"2026012223102370000_109.11.1272.4","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2022.13735"},{"key":"2026012223102370000_109.11.1272.5","doi-asserted-by":"crossref","DOI":"10.1148\/radiol.230970","article-title":"A Context-based Chatbot Surpasses Trained Radiologists and Generic ChatGPT in Following the ACR Appropriateness Guidelines","volume":"308","author":"Rau","year":"2023","journal-title":"Radiology"},{"key":"2026012223102370000_109.11.1272.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajo.2023.05.024"},{"key":"2026012223102370000_109.11.1272.7","doi-asserted-by":"crossref","DOI":"10.2196\/47184","article-title":"Investigating the Impact of User Trust on the Adoption and Use of ChatGPT: Survey Analysis","volume":"25","author":"Choudhury","year":"2023","journal-title":"J Med Internet Res"},{"key":"2026012223102370000_109.11.1272.8","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-00288-7"},{"key":"2026012223102370000_109.11.1272.9","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2021.04.027"},{"key":"2026012223102370000_109.11.1272.10","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.39335.541782.AD"},{"key":"2026012223102370000_109.11.1272.11","unstructured":"Chinese Journal of Ophthalmology . Chinese expert consensus on telemedicine for diabetic retinopathy. 2024. Available: https:\/\/zhykzz.yiigle.com\/"},{"key":"2026012223102370000_109.11.1272.12","unstructured":"National Eye Institute . At a glance: diabetic retinopathy. 2024. Available: https:\/\/www.nei.nih.gov\/learn-about-eye-health\/eye-conditions-and-diseases\/diabetic-retinopathy"},{"key":"2026012223102370000_109.11.1272.13","unstructured":"American Academy of Ophthalmology . Diabetic retinopathy: causes, symptoms, treatment. 2024. Available: https:\/\/www.aao.org\/eye-health\/diseases\/what-is-diabetic-retinopathy"},{"key":"2026012223102370000_109.11.1272.14","doi-asserted-by":"publisher","DOI":"10.1037\/h0057532"},{"key":"2026012223102370000_109.11.1272.15","doi-asserted-by":"crossref","unstructured":"Kincaid JP , Fishburne Jr RP , Rogers RL , et al . Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975.","DOI":"10.21236\/ADA006655"},{"key":"2026012223102370000_109.11.1272.16","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06221-2"},{"key":"2026012223102370000_109.11.1272.17","doi-asserted-by":"crossref","DOI":"10.3389\/fmed.2023.1291404","article-title":"Exploring large language model for next generation of artificial intelligence in ophthalmology","volume":"10","author":"Jin","year":"2023","journal-title":"Front Med (Lausanne)"},{"key":"2026012223102370000_109.11.1272.18","doi-asserted-by":"crossref","DOI":"10.1016\/j.aopr.2022.100078","article-title":"Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives","volume":"2","author":"Jin","year":"2022","journal-title":"Adv Ophthalmol Pract Res"},{"key":"2026012223102370000_109.11.1272.19","doi-asserted-by":"crossref","DOI":"10.1016\/j.ebiom.2023.104770","article-title":"Benchmarking large language models\u2019 performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard","volume":"95","author":"Lim","year":"2023","journal-title":"EBioMedicine"},{"key":"2026012223102370000_109.11.1272.20","doi-asserted-by":"publisher","DOI":"10.1001\/jamaophthalmol.2023.6917"},{"key":"2026012223102370000_109.11.1272.21","doi-asserted-by":"publisher","DOI":"10.1080\/09273948.2024.2317417"},{"key":"2026012223102370000_109.11.1272.22","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw Open"},{"key":"2026012223102370000_109.11.1272.23","unstructured":"OpenAI RJA . GPT-4 technical report. 2023."},{"key":"2026012223102370000_109.11.1272.24","unstructured":"National Institutes of Health . Clear & simple. 2024. Available: https:\/\/www.nih.gov\/institutes-nih\/nih-office-director\/office-communications-public-liaison\/clear-communication\/clear-simple"},{"key":"2026012223102370000_109.11.1272.25","doi-asserted-by":"crossref","DOI":"10.1016\/j.isci.2023.108163","article-title":"Popular large language model chatbots\u2019 accuracy, comprehensiveness, and self-awareness in answering ocular symptom queries","volume":"26","author":"Pushpanathan","year":"2023","journal-title":"iScience"},{"key":"2026012223102370000_109.11.1272.26","unstructured":"Bai J , Bai S , Chu Y , et al . Qwen technical report. 2023."},{"key":"2026012223102370000_109.11.1272.27","unstructured":"Touvron H , Martin L , Stone K , et al . Llama 2: open foundation and fine-tuned chat models. 2023."},{"key":"2026012223102370000_109.11.1272.28","unstructured":"Zhao H , Ling Q , Pan Y , et al . Ophtha-llama2: a large language model for ophthalmology. 2023."}],"container-title":["British Journal of Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bjo-2024-325861","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T07:11:07Z","timestamp":1769152267000},"score":1,"resource":{"primary":{"URL":"https:\/\/bjo.bmj.com\/lookup\/doi\/10.1136\/bjo-2024-325861"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":28,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,10,23]]},"published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1136\/bjo-2024-325861"],"URL":"https:\/\/doi.org\/10.1136\/bjo-2024-325861","relation":{},"ISSN":["0007-1161","1468-2079"],"issn-type":[{"value":"0007-1161","type":"print"},{"value":"1468-2079","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,30]]}}}