{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T22:13:33Z","timestamp":1792102413129,"version":"4.4.0"},"reference-count":35,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T00:00:00Z","timestamp":1750809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Large language models (LLMs) show promise as clinical consultation tools and may assist optic neuritis patients, though research on their performance in this area is limited. Our study aims to assess and compare the performance of four commonly used LLM-Chatbots\u2014Claude-2, ChatGPT-3.5, ChatGPT-4.0, and Google Bard\u2014in addressing questions related to optic neuritis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We curated 24 optic neuritis-related questions and had three ophthalmologists rate the responses on two three-point scales for accuracy and comprehensiveness. We also assessed readability using four scales. The final results showed performance differences among the four LLM-Chatbots.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The average total accuracy scores (out of 9): ChatGPT-4.0 (7.62 \u00b1 0.86), Google Bard (7.42 \u00b1 1.20), ChatGPT-3.5 (7.21 \u00b1 0.70), Claude-2 (6.44 \u00b1 1.07). ChatGPT-4.0 (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.0006) and Google Bard (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.0015) were significantly more accurate than Claude-2. Also, 62.5% of ChatGPT-4.0\u2019s responses were rated \u201cExcellent,\u201d followed by 58.3% for Google Bard, both higher than Claude-2\u2019s 29.2% (all\n                      <jats:italic>p<\/jats:italic>\n                      \u2264 0.042) and ChatGPT-3.5\u2019s 41.7%. Both Claude-2 and Google Bard had 8.3% \u201cDeficient\u201d responses. The comprehensiveness scores were similar among the four LLMs (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.1531). Note that all responses require at least a university-level reading proficiency.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Large language models-Chatbots hold immense potential as clinical consultation tools for optic neuritis, but they require further refinement and proper evaluation strategies before deployment to ensure reliable and accurate performance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2025.1516442","type":"journal-article","created":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T01:24:12Z","timestamp":1750814652000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Evaluation and comparison of large language models\u2019 responses to questions related optic neuritis"],"prefix":"10.3389","volume":"12","author":[{"given":"Han-Jie","family":"He","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fang-Fang","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia-Jian","family":"Liang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yun","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qian-Qian","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongjie","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingyun","family":"Cen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feifei","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tai-Ping","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanchi","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian-Feng","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lan","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carol Y.","family":"Cheung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yih-Chung","family":"Tham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ling-Ping","family":"Cen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,6,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"101095","DOI":"10.1016\/j.xcrm.2023.101095","article-title":"Artificial intelligence in ophthalmology: The path to the real-world clinic.","volume":"4","author":"Li","year":"2023","journal-title":"Cell Rep Med."},{"key":"B2","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","article-title":"A guide to deep learning in healthcare.","volume":"25","author":"Esteva","year":"2019","journal-title":"Nat Med."},{"key":"B3","year":"2024","journal-title":"Introducing ChatGPT."},{"key":"B4","doi-asserted-by":"publisher","first-page":"3701","DOI":"10.1097\/JS9.0000000000001312","article-title":"ChatGPT in medicine: Prospects and challenges: A review article.","volume":"110","author":"Tan","year":"2024","journal-title":"Int J Surg."},{"key":"B5","doi-asserted-by":"publisher","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings.","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol Sci."},{"key":"B6","doi-asserted-by":"publisher","first-page":"1371","DOI":"10.1136\/bjo-2023-324438","article-title":"Capabilities of GPT-4 in ophthalmology: An analysis of model entropy and progress towards human-level medical question answering.","volume":"108","author":"Antaki","year":"2023","journal-title":"Br J Ophthalmol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"e184","DOI":"10.1055\/s-0043-1774399","article-title":"Improved performance of ChatGPT-4 on the OKAP examination: A comparative study with ChatGPT-3.5.","volume":"15","author":"Teebagy","year":"2023","journal-title":"J Acad Ophthalmol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"20552076241237678","DOI":"10.1177\/20552076241237678","article-title":"Exploring the proficiency of ChatGPT-4: An evaluation of its performance in the Taiwan advanced medical licensing examination.","volume":"10","author":"Lin","year":"2024","journal-title":"Digit Health."},{"key":"B9","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1097\/ICU.0000000000001036","article-title":"ChatGPT enters the room: What it means for patient counseling, physician education, academics, and disease management.","volume":"35","author":"Momenaei","year":"2024","journal-title":"Curr Opin Ophthalmol."},{"key":"B10","doi-asserted-by":"publisher","first-page":"120","DOI":"10.3390\/bioengineering11020120","article-title":"Exploring diagnostic precision and triage proficiency: A comparative study of GPT-4 and bard in addressing common ophthalmic complaints.","volume":"11","author":"Zandi","year":"2024","journal-title":"Bioengineering."},{"key":"B11","doi-asserted-by":"publisher","first-page":"3121","DOI":"10.1007\/s40123-023-00805-x","article-title":"The use of ChatGPT to assist in diagnosing glaucoma based on clinical case reports.","volume":"12","author":"Delsoz","year":"2023","journal-title":"Ophthalmol Ther."},{"key":"B12","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1001\/jamaophthalmol.2024.0017","article-title":"Accuracy of an artificial intelligence Chatbot\u2019s interpretation of clinical ophthalmic images.","volume":"142","author":"Mihalache","year":"2024","journal-title":"JAMA Ophthalmol."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1679","DOI":"10.3390\/jpm13121679","article-title":"Personalized care in eye health: Exploring opportunities, challenges, and the road ahead for Chatbots.","volume":"13","author":"Ittarat","year":"2023","journal-title":"J Pers Med."},{"key":"B14","doi-asserted-by":"publisher","first-page":"1368","DOI":"10.1038\/s41433-023-02906-0","article-title":"Reliability and accuracy of artificial intelligence ChatGPT in providing information on ophthalmic diseases and management to patients.","volume":"38","author":"Cappellani","year":"2024","journal-title":"Eye."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions.","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw Open."},{"key":"B16","doi-asserted-by":"publisher","first-page":"2192","DOI":"10.3390\/genes14122192","article-title":"Uncovering the genetics and physiology behind optic neuritis.","volume":"14","author":"Del Negro","year":"2023","journal-title":"Genes."},{"key":"B17","doi-asserted-by":"publisher","first-page":"1121290","DOI":"10.3389\/fpubh.2023.1121290","article-title":"Switching from offline to online health consultation in the post-pandemic era: The role of perceived pandemic risk.","volume":"11","author":"Pan","year":"2023","journal-title":"Front Public Health."},{"key":"B18","year":"2024","journal-title":"American Academy of Ophthalmology [Internet]."},{"key":"B19","year":"2024","journal-title":"Search Results for \u201cOptic Neuritis\u201d."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2522","DOI":"10.1007\/s00464-024-10720-2","article-title":"Large language models and bariatric surgery patient education: A comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources.","volume":"38","author":"Srinivasan","year":"2024","journal-title":"Surg Endosc."},{"key":"B21","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1016\/j.urology.2021.03.009","article-title":"An evaluation of the readability and content-quality of pelvic organ prolapse youtube transcripts.","volume":"154","author":"Herbert","year":"2021","journal-title":"Urology."},{"key":"B22","doi-asserted-by":"publisher","first-page":"e2336997","DOI":"10.1001\/jamanetworkopen.2023.36997","article-title":"Large language model-based chatbot vs surgeon-generated informed consent documentation for common procedures.","volume":"6","author":"Decker","year":"2023","journal-title":"JAMA Netw Open."},{"key":"B23","year":"2024","journal-title":"Free Readability Assessment Tools [Internet]."},{"key":"B24","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1097\/WNO.0000000000002074","article-title":"Utility of ChatGPT for automated creation of patient education handouts: An application in neuro-ophthalmology.","volume":"44","author":"Tao","year":"2024","journal-title":"J Neuroophthalmol."},{"key":"B25","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1097\/WNO.0000000000002145","article-title":"A comparative study of large language models, human experts, and expert-edited large language models to neuro-ophthalmology questions.","volume":"45","author":"Tailor","year":"2024","journal-title":"J Neuroophthalmol."},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1097\/WNO.0000000000002274","article-title":"ChatGPT assisting diagnosis of neuro-ophthalmology diseases based on case reports.","author":"Madadi","year":"2023","journal-title":"J Neuroophthalmol."},{"key":"B27","author":"Weiss","year":"2007","journal-title":"Health Literacy and Patient Safety: Help Patients Understand."},{"key":"B28","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1001\/jamaophthalmol.2014.6104","article-title":"Assessment of online patient education materials from major ophthalmologic associations.","volume":"133","author":"Huang","year":"2015","journal-title":"JAMA Ophthalmol."},{"key":"B29","doi-asserted-by":"publisher","first-page":"3530","DOI":"10.1038\/s41433-023-02563-3","article-title":"Comparative analysis of large language models in the Royal College of Ophthalmologists fellowship exams.","volume":"37","author":"Raimondi","year":"2023","journal-title":"Eye."},{"key":"B30","author":"OpenAI","year":"2024","journal-title":"GPT-4 Technical Report [Internet]."},{"key":"B31","year":"2024","journal-title":"OpenAI Chatbot [Internet]."},{"key":"B32","first-page":"e43313","article-title":"Artificial hallucinations by Google bard: Think before you leap.","volume":"15","author":"Kumar","year":"2023","journal-title":"Cureus."},{"key":"B33","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.jocn.2024.03.021","article-title":"Evaluation of the safety, accuracy, and helpfulness of the GPT-4.0 large language model in neurosurgery.","volume":"123","author":"Huang","year":"2024","journal-title":"J Clin Neurosci."},{"key":"B34","doi-asserted-by":"publisher","first-page":"104770","DOI":"10.1016\/j.ebiom.2023.104770","article-title":"Benchmarking large language models\u2019 performances for myopia care: A comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard.","volume":"95","author":"Lim","year":"2023","journal-title":"EBioMedicine."},{"key":"B35","doi-asserted-by":"publisher","first-page":"e37458","DOI":"10.1097\/MD.0000000000037458","article-title":"Evaluation of ChatGPT in providing appropriate fracture prevention recommendations and medical science question responses: A quantitative research.","volume":"103","author":"Meng","year":"2024","journal-title":"Medicine."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2025.1516442\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T01:24:17Z","timestamp":1750814657000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2025.1516442\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,25]]},"references-count":35,"alternative-id":["10.3389\/fmed.2025.1516442"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2025.1516442","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,25]]},"article-number":"1516442"}}