{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T08:03:00Z","timestamp":1792569780301,"version":"4.5.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T00:00:00Z","timestamp":1761868800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T00:00:00Z","timestamp":1761868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Retin Vitr"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Chatbots powered by large language models have shown promising results when addressing medical queries, thus transforming access to medical information. Although these chatbots produce detailed and accurate responses, it is unclear how they perform when handling real, unedited patient questions, particularly in non-English languages. This study aimed to assess the readability, accuracy, and comprehensiveness of responses to retinal disease queries provided by four chatbots (ChatGPT 4.0, ConsensusGPT, Gemini, and Claude 3) compared to responses from retina specialists.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      In this cross-sectional, comparative, and blinded study, twenty unedited questions about retinal diseases were randomly selected from a popular online video channel in Portuguese. The questions were submitted to the four selected chatbots and retina specialists with fellowship training. Two independent retinal experts evaluated the responses using standardized Likert scales for accuracy and completeness. Readability was assessed using the Flesch Reading Ease Score and the Flesch-Kincaid Grade Level tests. Additional metrics, including word count and response generation time, were analyzed. Data were compared among groups using non-parametric statistical tests, including the Kruskal-Wallis test with Dunn\u2019s pairwise comparisons and chi-squared tests, with a two-sided\n                      <jats:italic>p<\/jats:italic>\n                      -value threshold of 0.05 for statistical significance.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Retinal specialists and the Gemini chatbot produced responses with higher readability, indicating lower educational levels were needed for comprehension. In contrast, ChatGPT 4.0, ConsensusGPT, and Claude 3 delivered more detailed and accurate answers but required a higher reading level. ChatGPT 4.0 and ConsensusGPT achieved superior quality and comprehensiveness ratings compared to human experts and the other chatbots. Additionally, all AI systems generated responses significantly faster than the human specialists. Evaluators could correctly distinguish between human-generated and AI-generated responses in most cases.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Artificial intelligence chatbots demonstrate considerable promise for rapidly disseminating accurate medical information directly to the end-user, not only in English. However, optimizing the simplicity of their language is essential to ensure that detailed responses remain accessible to a broad audience. Future research should aim to replicate our findings with larger datasets of questions with the goal of refining these systems to balance comprehensive content with user-friendly language.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s40942-025-00737-7","type":"journal-article","created":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T09:37:53Z","timestamp":1761903473000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Chatbots versus retina specialists in answering real-world retina questions"],"prefix":"10.1186","volume":"11","author":[{"given":"Mario Cesar","family":"Bulla","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Caio Augusto","family":"Scocco","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Viviane Souto","family":"Spadoni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gustavo Matias","family":"H\u00fcning","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandre","family":"Grandinetti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andre Moraes","family":"Freitas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandre Antonio Marques","family":"Rosa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro Carlos","family":"Carricondo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,10,31]]},"reference":[{"key":"737_CR1","doi-asserted-by":"publisher","first-page":"e2336483","DOI":"10.1001\/jamanetworkopen.2023.36483","volume":"6","author":"RS Goodman","year":"2023","unstructured":"Goodman RS, Patrinely JR, Stone CA Jr., Zimmerman E, Donald RR, Chang SS, et al. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open. 2023;6:e2336483. https:\/\/doi.org\/10.1001\/jamanetworkopen.2023.36483.","journal-title":"JAMA Netw Open"},{"key":"737_CR2","doi-asserted-by":"publisher","first-page":"104770","DOI":"10.1016\/j.ebiom.2023.104770","volume":"95","author":"ZW Lim","year":"2023","unstructured":"Lim ZW, Pushpanathan K, Yew SME, Lai Y, Sun CH, Lam JSH, et al. Benchmarking large Language models\u2019 performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google bard. EBioMedicine. 2023;95:104770. https:\/\/doi.org\/10.1016\/j.ebiom.2023.104770.","journal-title":"EBioMedicine"},{"key":"737_CR3","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","volume":"7","author":"B Momenaei","year":"2023","unstructured":"Momenaei B, Wakabayashi T, Shahlaee A, Durrani AF, Pandit SA, Wang K, et al. Appropriateness and readability of ChatGPT-4-Generated responses for surgical treatment of retinal diseases. Ophthalmol Retina. 2023;7:862\u20138. https:\/\/doi.org\/10.1016\/j.oret.2023.05.022.","journal-title":"Ophthalmol Retina"},{"key":"737_CR4","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1001\/jamaophthalmol.2023.6917","volume":"142","author":"AS Huang","year":"2024","unstructured":"Huang AS, Hirabayashi K, Barna L, Parikh D, Pasquale LR. Assessment of a large Language model\u2019s responses to questions and cases about glaucoma and retina management. JAMA Ophthalmol. 2024;142:371\u20135. https:\/\/doi.org\/10.1001\/jamaophthalmol.2023.6917.","journal-title":"JAMA Ophthalmol"},{"key":"737_CR5","doi-asserted-by":"publisher","first-page":"100485","DOI":"10.1016\/j.xops.2024.100485","volume":"4","author":"PD Tailor","year":"2024","unstructured":"Tailor PD, Dalvin LA, Chen JJ, Iezzi R, Olsen TW, Scruggs BA, et al. A comparative study of responses to retina questions from either Experts, Expert-Edited large Language models, or Expert-Edited large Language models alone. Ophthalmol Sci. 2024;4:100485. https:\/\/doi.org\/10.1016\/j.xops.2024.100485.","journal-title":"Ophthalmol Sci"},{"key":"737_CR6","doi-asserted-by":"publisher","first-page":"e379","DOI":"10.1016\/S2589-7500(24)00060-8","volume":"6","author":"S Chen","year":"2024","unstructured":"Chen S, Guevara M, Moningi S, Hoebers F, Elhalawani H, Kann BH, et al. The effect of using a large Language model to respond to patient messages. Lancet Digit Health. 2024;6:e379\u201381. https:\/\/doi.org\/10.1016\/S2589-7500(24)00060-8.","journal-title":"Lancet Digit Health"},{"key":"737_CR7","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1186\/s40942-024-00579-9","volume":"10","author":"P Strzalkowski","year":"2024","unstructured":"Strzalkowski P, Strzalkowska A, Chhablani J, Pfau K, Errera MH, Roth M, et al. Evaluation of the accuracy and readability of ChatGPT-4 and Google gemini in providing information on retinal detachment: a multicenter expert comparative study. Int J Retina Vitreous. 2024;10:61. https:\/\/doi.org\/10.1186\/s40942-024-00579-9.","journal-title":"Int J Retina Vitreous"},{"key":"737_CR8","doi-asserted-by":"publisher","first-page":"e44720","DOI":"10.7759\/cureus.44720","volume":"15","author":"R Hatem","year":"2023","unstructured":"Hatem R, Simmons B, Thornton JE. A call to address AI hallucinations and how healthcare professionals can mitigate their risks. Cureus. 2023;15:e44720. https:\/\/doi.org\/10.7759\/cureus.44720.","journal-title":"Cureus"},{"key":"737_CR9","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1515\/dx-2018-0009","volume":"5","author":"ML Millenson","year":"2018","unstructured":"Millenson ML, Baldwin JL, Zipperer L, Singh H. Beyond Dr. Google: the evidence on consumer-facing digital tools for diagnosis. Diagnosis (Berl). 2018;5:95\u2013105. https:\/\/doi.org\/10.1515\/dx-2018-0009.","journal-title":"Diagnosis (Berl)"},{"key":"737_CR10","doi-asserted-by":"publisher","first-page":"e1138","DOI":"10.1016\/j.wneu.2023.11.062","volume":"181","author":"MH Temel","year":"2024","unstructured":"Temel MH, Erden Y, Bagcier F. Information quality and readability: chatgpt\u2019s responses to the most common questions about spinal cord injury. World Neurosurg. 2024;181:e1138\u201344. https:\/\/doi.org\/10.1016\/j.wneu.2023.11.062.","journal-title":"World Neurosurg"},{"key":"737_CR11","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","volume":"183","author":"JW Ayers","year":"2023","unstructured":"Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183:589\u201396. https:\/\/doi.org\/10.1001\/jamainternmed.2023.1838.","journal-title":"JAMA Intern Med"},{"key":"737_CR12","doi-asserted-by":"publisher","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","volume":"6","author":"IA Bernstein","year":"2023","unstructured":"Bernstein IA, Zhang YV, Govil D, Majid I, Chang RT, Sun Y, et al. Comparison of ophthalmologist and large Language model chatbot responses to online patient eye care questions. JAMA Netw Open. 2023;6:e2330320. https:\/\/doi.org\/10.1001\/jamanetworkopen.2023.30320.","journal-title":"JAMA Netw Open"},{"key":"737_CR13","doi-asserted-by":"publisher","first-page":"101027","DOI":"10.1016\/j.asmr.2024.101027","volume":"7","author":"RS Marder","year":"2025","unstructured":"Marder RS, Abdelmalek G, Richards SM, Nadeau NJ, Garcia DJ, Attia PJ, et al. ChatGPT-3.5 and \u2013\u20094.0 do not reliably create readable patient education materials for common orthopaedic Upper- and Lower-Extremity conditions. Arthrosc Sports Med Rehabil. 2025;7:101027. https:\/\/doi.org\/10.1016\/j.asmr.2024.101027.","journal-title":"Arthrosc Sports Med Rehabil"},{"key":"737_CR14","doi-asserted-by":"publisher","first-page":"e61067","DOI":"10.7759\/cureus.61067","volume":"16","author":"TJ Lee","year":"2024","unstructured":"Lee TJ, Rao AK, Campbell DJ, Radfar N, Dayal M, Khrais A. Evaluating ChatGPT-3.5 and ChatGPT-4.0 responses on hyperlipidemia for patient education. Cureus. 2024;16:e61067. https:\/\/doi.org\/10.7759\/cureus.61067.","journal-title":"Cureus"},{"key":"737_CR15","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1016\/j.jacr.2023.09.011","volume":"21","author":"EB Gordon","year":"2024","unstructured":"Gordon EB, Towbin AJ, Wingrove P, Shafique U, Haas B, Kitts AB, et al. Enhancing patient communication with Chat-GPT in radiology: evaluating the efficacy and readability of answers to common Imaging-Related questions. J Am Coll Radiol. 2024;21:353\u20139. https:\/\/doi.org\/10.1016\/j.jacr.2023.09.011.","journal-title":"J Am Coll Radiol"},{"key":"737_CR16","doi-asserted-by":"publisher","first-page":"e35179","DOI":"10.7759\/cureus.35179","volume":"15","author":"H Alkaissi","year":"2023","unstructured":"Alkaissi H, McFarlane SI. Artificial hallucinations in chatgpt: implications in scientific writing. Cureus. 2023;15:e35179. https:\/\/doi.org\/10.7759\/cureus.35179.","journal-title":"Cureus"},{"key":"737_CR17","unstructured":"Goldim JR. Consentimento e informa\u00e7\u00e3o: a import\u00e2ncia Da qualidade do Texto utilizado. Clin Biomedical Res. 2006;26."},{"key":"737_CR18","doi-asserted-by":"publisher","first-page":"1093","DOI":"10.1007\/s12553-021-00574-2","volume":"11","author":"NE Wrigley Kelly","year":"2021","unstructured":"Wrigley Kelly NE, Murray KE, McCarthy C, O\u2019Shea DB. An objective analysis of quality and readability of online information on COVID-19. Health Technol (Berl). 2021;11:1093\u20139. https:\/\/doi.org\/10.1007\/s12553-021-00574-2.","journal-title":"Health Technol (Berl)"},{"key":"737_CR19","unstructured":"Minist\u00e9rio da Educa\u00e7\u00e3o - Universidade Federal de Campina Grande. Total m\u00e9dio de anos de estudo cresce no Brasil, diz pesquisa do IBGE. n.d. [cited 10 Feb 2025]. Available from: https:\/\/portal.ufcg.edu.br\/em-dia\/916-total-medio-de-anos-de-estudo-cresce-no-brasil-diz-pesquisa-do-ibge.html"},{"key":"737_CR20","doi-asserted-by":"publisher","first-page":"2522","DOI":"10.1007\/s00464-024-10720-2","volume":"38","author":"N Srinivasan","year":"2024","unstructured":"Srinivasan N, Samaan JS, Rajeev ND, Kanu MU, Yeo YH, Samakar K. Large Language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources. Surg Endosc. 2024;38:2522\u201332. https:\/\/doi.org\/10.1007\/s00464-024-10720-2.","journal-title":"Surg Endosc"},{"key":"737_CR21","doi-asserted-by":"publisher","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","volume":"3","author":"F Antaki","year":"2023","unstructured":"Antaki F, Touma S, Milad D, El-Khoury J, Duval R. Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings. Ophthalmol Sci. 2023;3:100324. https:\/\/doi.org\/10.1016\/j.xops.2023.100324.","journal-title":"Ophthalmol Sci"},{"key":"737_CR22","doi-asserted-by":"publisher","first-page":"100600","DOI":"10.1016\/j.xops.2024.100600","volume":"5","author":"JS Chen","year":"2025","unstructured":"Chen JS, Reddy AJ, Al-Sharif E, Shoji MK, Kalaw FGP, Eslani M, et al. Analysis of ChatGPT responses to ophthalmic cases: can ChatGPT think like an ophthalmologist? Ophthalmol Sci. 2025;5:100600. https:\/\/doi.org\/10.1016\/j.xops.2024.100600.","journal-title":"Ophthalmol Sci"},{"key":"737_CR23","doi-asserted-by":"publisher","unstructured":"Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the Dangers of Stochastic Parrots. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency; Virtual Event, Canada: Association for Computing Machinery. 2021. pp. 610\u2009\u2013\u200923. https:\/\/doi.org\/10.1145\/3442188.3445922.","DOI":"10.1145\/3442188.3445922"},{"key":"737_CR24","doi-asserted-by":"publisher","first-page":"1368","DOI":"10.1038\/s41433-023-02906-0","volume":"38","author":"F Cappellani","year":"2024","unstructured":"Cappellani F, Card KR, Shields CL, Pulido JS, Haller JA. Reliability and accuracy of artificial intelligence ChatGPT in providing information on ophthalmic diseases and management to patients. Eye (Lond). 2024;38:1368\u201373. https:\/\/doi.org\/10.1038\/s41433-023-02906-0.","journal-title":"Eye (Lond)"},{"key":"737_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-319-99289-1_1","volume-title":"The internet and health in Brazil","author":"A Pereira Neto","year":"2019","unstructured":"Pereira Neto A, Flynn MB. The internet and health in Brazil: Trends and Challenges. In: Pereira Neto A, Flynn MB, editors. The internet and health in Brazil. Cham: Springer International Publishing; 2019. pp. 1\u201311. https:\/\/doi.org\/10.1007\/978-3-319-99289-1_1."}],"container-title":["International Journal of Retina and Vitreous"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40942-025-00737-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40942-025-00737-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40942-025-00737-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T09:37:54Z","timestamp":1761903474000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalretinavitreous.biomedcentral.com\/articles\/10.1186\/s40942-025-00737-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,31]]},"references-count":25,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["737"],"URL":"https:\/\/doi.org\/10.1186\/s40942-025-00737-7","relation":{},"ISSN":["2056-9920"],"issn-type":[{"value":"2056-9920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,31]]},"assertion":[{"value":"10 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 October 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was approved by the institutional review board Universidade do Vale do Rio dos Sinos (Project number: 79501724.8.0000.5344).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"120"}}