{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T15:02:21Z","timestamp":1784732541645,"version":"3.55.0"},"reference-count":39,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T00:00:00Z","timestamp":1782864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T00:00:00Z","timestamp":1782864000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Health Informatics J"],"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>To assess the accuracy, comprehensiveness, and reliability of Large Language Model chatbots in answering frequently asked questions about refractive errors.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Forty-four questions about refractive errors were posed to four chatbots, including Copilot, Perplexity, Gemini, and ChatGPT. Responses to each question were independently evaluated by three experts using a three-point accuracy scale. The readability of the chatbots\u2019 responses was evaluated using several indices. Similarity was assessed using Sentence-Bidirectional Encoder Representations from Transformers (SBERT). Inter-rater agreement among the graders was evaluated using the Gwet Agreement Coefficient 1 (AC1) statistic.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The overall agreement among the graders for all chatbot responses was almost perfect (Gwet AC1: 0.87). All chatbots received scores above 85% in every category, and there was no significant difference in chatbot accuracy scores (\n                      <jats:italic toggle=\"yes\">P= 0.168<\/jats:italic>\n                      ). The highest mean comprehensiveness score was observed for Perplexity (8.13 \u00b1 0.95,\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = 0.028). The similarity scores of the chatbots were very close to each other. All readability scores showed significant differences (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      &lt;0.001) between the chatbots.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>All four chatbots showed comparable accuracy in answering questions about refractive errors. Readability levels of all chatbots exceeded public health recommended thresholds; however, ChatGPT produced relatively more accessible output.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/14604582261470624","type":"journal-article","created":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T13:31:08Z","timestamp":1784554268000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluation of the large language models Chatbot\u2019s responses to frequently asked queries on refractive errors"],"prefix":"10.1177","volume":"32","author":[{"given":"Sare","family":"Safi","sequence":"first","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"},{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4078-4120","authenticated-orcid":false,"given":"Amir","family":"Golmakani","sequence":"additional","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saeed","family":"Rahmani","sequence":"additional","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikoo","family":"Bayan","sequence":"additional","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seyed Mohamadmehdi","family":"Moshtaghion","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Harvard Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hadis","family":"Najafimehr","sequence":"additional","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haleh","family":"Kangari","sequence":"additional","affiliation":[{"name":"Shahid Beheshti University of Medical Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,7,20]]},"reference":[{"key":"e_1_3_7_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aopr.2024.11.001"},{"key":"e_1_3_7_3_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12889-025-22440-w"},{"key":"e_1_3_7_4_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12889-023-16484-z"},{"key":"e_1_3_7_5_2","doi-asserted-by":"publisher","DOI":"10.1136\/bjo-2024-325458"},{"key":"e_1_3_7_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-023-00370-1"},{"key":"e_1_3_7_7_2","doi-asserted-by":"publisher","DOI":"10.2196\/56764"},{"key":"e_1_3_7_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2023.104770"},{"key":"e_1_3_7_9_2","doi-asserted-by":"publisher","DOI":"10.5534\/wjmh.230033"},{"issue":"7","key":"e_1_3_7_10_2","first-page":"e88980","article-title":"Evaluating the Accuracy, Completeness, and Readability of Chatbot Responses to Refractive Surgery-Related Patient Questions: A Comparative Analysis of ChatGPT and Google Gemini","volume":"17","author":"Arslan S","year":"2025","unstructured":"Arslan S, K\u00fc\u00e7\u00fckbezirci GU. Evaluating the Accuracy, Completeness, and Readability of Chatbot Responses to Refractive Surgery-Related Patient Questions: A Comparative Analysis of ChatGPT and Google Gemini. Cureus 2025; 17(7): e88980.","journal-title":"Cureus"},{"key":"e_1_3_7_11_2","first-page":"1","article-title":"Large Language Models: Pioneering New Educational Frontiers in Childhood Myopia","volume":"21","author":"Delsoz M","year":"2025","unstructured":"Delsoz M, Hassan A, Nabavi A, et al. Large Language Models: Pioneering New Educational Frontiers in Childhood Myopia. Ophthalmology and Therapy 2025; 21: 1\u20135. https:\/\/doi.org\/10.1007\/s40123-025-01142-x","journal-title":"Ophthalmology and Therapy"},{"key":"e_1_3_7_12_2","volume-title":"Sentence-bert: Sentence embeddings using siamese bert-networks","author":"Reimers N","year":"2019","unstructured":"Reimers N, Gurevych I. Sentence-bert: Sentence embeddings using siamese bert-networks, 2019. arXiv preprint arXiv:1908.10084."},{"key":"e_1_3_7_13_2","doi-asserted-by":"publisher","DOI":"10.1177\/1536867X1801800408"},{"key":"e_1_3_7_14_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjophth-2024-001824"},{"key":"e_1_3_7_15_2","doi-asserted-by":"publisher","DOI":"10.1002\/jso.27966"},{"issue":"7","key":"e_1_3_7_16_2","first-page":"348","article-title":"Comparative Assessment of AI-Driven Chatbots for Ophthalmic Patient Information: Inaccuracies in Chatbots\u2019 Answers to Patient Questions","volume":"65","author":"Oca M","year":"2024","unstructured":"Oca M, Parikh A, Conger J, et al. Comparative Assessment of AI-Driven Chatbots for Ophthalmic Patient Information: Inaccuracies in Chatbots\u2019 Answers to Patient Questions. Investigative Ophthalmology & Visual Science 2024; 65(7): 348.","journal-title":"Investigative Ophthalmology & Visual Science"},{"key":"e_1_3_7_17_2","doi-asserted-by":"publisher","DOI":"10.1136\/bjo-2023-324533"},{"key":"e_1_3_7_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clae.2024.102130"},{"key":"e_1_3_7_19_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0326351"},{"key":"e_1_3_7_20_2","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000038352"},{"key":"e_1_3_7_21_2","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000039305"},{"key":"e_1_3_7_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-024-02056-0"},{"key":"e_1_3_7_23_2","doi-asserted-by":"publisher","DOI":"10.3390\/jpm14121165"},{"key":"e_1_3_7_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-025-02178-z"},{"issue":"3","key":"e_1_3_7_25_2","article-title":"Assessing the capability of large language model chatbots in generating plain language summaries","volume":"17","author":"Mondal H","year":"2025","unstructured":"Mondal H, Gupta G, Sarangi PK, et al. Assessing the capability of large language model chatbots in generating plain language summaries. Cureus 2025; 17(3). https:\/\/doi.org\/10.7759\/cureus.80976","journal-title":"Cureus"},{"issue":"3","key":"e_1_3_7_26_2","first-page":"e56766","article-title":"Performance of artificial intelligence chatbots on glaucoma questions adapted from patient brochures","volume":"16","author":"Yalla GR","year":"2024","unstructured":"Yalla GR, Hyman N, Hock LE, et al. Performance of artificial intelligence chatbots on glaucoma questions adapted from patient brochures. Cureus 2024; 16(3): e56766. https:\/\/doi.org\/10.7759\/cureus.56766","journal-title":"Cureus"},{"key":"e_1_3_7_27_2","doi-asserted-by":"publisher","DOI":"10.2147\/OPTH.S483024"},{"key":"e_1_3_7_28_2","unstructured":"Weiss BD. Health literacy. American Medical Association 2003 vol 253 p. 358."},{"key":"e_1_3_7_29_2","doi-asserted-by":"publisher","DOI":"10.1002\/epi4.70022"},{"key":"e_1_3_7_30_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.13686"},{"key":"e_1_3_7_31_2","doi-asserted-by":"publisher","DOI":"10.1177\/1062860617751639"},{"key":"e_1_3_7_32_2","article-title":"Dated data: Tracing knowledge cutoffs in large language models","author":"Cheng J","year":"2024","unstructured":"Cheng J, Marone M, Weller O, et al. Dated data: Tracing knowledge cutoffs in large language models. arXiv preprint arXiv:240312958 2024.","journal-title":"arXiv preprint arXiv:240312958"},{"key":"e_1_3_7_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-023-08319-9"},{"issue":"1","key":"e_1_3_7_34_2","first-page":"41","article-title":"A Battle of the Chatbots: An Exploratory Comparison of ChatGPT, Gemini, Copilot, Claude, Perplexity and HuggingChat","volume":"15","author":"Okamoto K","year":"2025","unstructured":"Okamoto K. A Battle of the Chatbots: An Exploratory Comparison of ChatGPT, Gemini, Copilot, Claude, Perplexity and HuggingChat. Practical Academic Librarianship: The International Journal of the SLA Academic Division 2025; 15(1): 41\u201372.","journal-title":"Practical Academic Librarianship: The International Journal of the SLA Academic Division"},{"key":"e_1_3_7_35_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.36483"},{"key":"e_1_3_7_36_2","doi-asserted-by":"publisher","DOI":"10.2196\/60291"},{"key":"e_1_3_7_37_2","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1359073"},{"key":"e_1_3_7_38_2","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231224603"},{"key":"e_1_3_7_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00417-023-06078-1"},{"key":"e_1_3_7_40_2","doi-asserted-by":"publisher","DOI":"10.1097\/WNO.0000000000002145"}],"container-title":["Health Informatics Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14604582261470624","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/14604582261470624","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14604582261470624","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T14:16:49Z","timestamp":1784729809000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/14604582261470624"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,7]]}},"alternative-id":["10.1177\/14604582261470624"],"URL":"https:\/\/doi.org\/10.1177\/14604582261470624","relation":{},"ISSN":["1460-4582","1741-2811"],"issn-type":[{"value":"1460-4582","type":"print"},{"value":"1741-2811","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7]]},"article-number":"14604582261470624"}}