{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T18:04:05Z","timestamp":1767204245564,"version":"build-2238731810"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,7,7]],"date-time":"2025-07-07T00:00:00Z","timestamp":1751846400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Cell Dev. Biol."],"abstract":"<jats:p>The avoidance of mistakes by humans is achieved through continuous learning, error correction, and experience accumulation. This process is known to be both time-consuming and laborious, often involving numerous detours. In order to assist humans in their learning endeavors, ChatGPT (Generative Pre-trained Transformer) has been developed as a collection of large language models (LLMs) capable of generating responses that resemble human-like answers to a wide range of problems. In this study, we sought to assess the potential of LLMs as assistants in addressing queries related to orbital diseases. To accomplish this, we gathered a dataset consisting of 100 orbital questions, along with their corresponding answers, sourced from examinations administered to ophthalmologist residents and medical students. Five language models (LLMs) were utilized for testing and comparison purposes, namely, GPT-4, GPT-3.5, PaLM2, Claude 2, and SenseNova. Subsequently, the LLM exhibiting the most exemplary performance was selected for comparison against ophthalmologists and medical students. Notably, GPT-4 and PaLM2 demonstrated a superior average correlation when compared to the other LLMs. Furthermore, GPT-4 exhibited a broader spectrum of accurate responses and attained the highest average score among all the LLMs. Additionally, GPT-4 demonstrated the highest level of confidence during the test. The performance of GPT-4 surpassed that of medical students, albeit falling short of that exhibited by ophthalmologists. In contrast, the findings of the study indicate that GPT-4 exhibited superior performance within the orbital domain of ophthalmology. Given further refinement through training, LLMs possess considerable potential to be utilized as comprehensive instruments alongside medical students and ophthalmologists.<\/jats:p>","DOI":"10.3389\/fcell.2025.1574378","type":"journal-article","created":{"date-parts":[[2025,7,7]],"date-time":"2025-07-07T01:26:56Z","timestamp":1751851616000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating multiple large language models on orbital diseases"],"prefix":"10.3389","volume":"13","author":[{"given":"Qi-Chen","family":"Yang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan-Mei","family":"Zeng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hong","family":"Wei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cheng","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qian","family":"Ling","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao-Yu","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi","family":"Shao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,7,7]]},"reference":[{"key":"B39","first-page":"e48291","article-title":"Large language models in medical education: opportunities, challenges, and future directions","volume-title":"JMIR Med. Educ.","author":"Abd-alrazaq","year":"2023"},{"key":"B1","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1002\/9781119792550.ch14","article-title":"Big data in healthcare: management, analysis, and future prospects","author":"Akilal","year":"2022","journal-title":"Handb. Intelligent Healthc. Anal. Knowl. Eng. Big Data Anal."},{"key":"B2","doi-asserted-by":"publisher","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol. Sci."},{"key":"B3","doi-asserted-by":"publisher","first-page":"52","DOI":"10.61969\/jai.1337500","article-title":"Education in the era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning","volume":"7","author":"Baidoo-Anu","year":"2023","journal-title":"J. AI"},{"key":"B4","article-title":"A neural probabilistic language model","volume":"13","author":"Bengio","year":"2000","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B5","doi-asserted-by":"publisher","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of ophthalmologist and large Language Model chatbot responses to online patient eye care questions","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw. Open"},{"key":"B6","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.ajo.2023.05.024","article-title":"Performance of generative large language models on ophthalmology board-style questions","volume":"254","author":"Cai","year":"2023","journal-title":"Am. J. Ophthalmol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"694","DOI":"10.1097\/IAE.0000000000004351","article-title":"Retinal imaging analysis performed by chatgpt-4o and gemini advanced: the turning point of the revolution?","volume":"45","author":"Carl\u00e0","year":"2025","journal-title":"Retina"},{"key":"B8","doi-asserted-by":"publisher","first-page":"2945","DOI":"10.1007\/s00417-024-06470-5","article-title":"Large language models as assistance for glaucoma surgical cases: a ChatGPT vs. Google Gemini comparison","volume":"262","author":"Carl\u00e0","year":"","journal-title":"Graefes Arch. Clin. Exp. Ophthalmol."},{"key":"B9","doi-asserted-by":"publisher","first-page":"1457","DOI":"10.1136\/bjo-2023-325143","article-title":"Exploring AI-chatbots' capability to suggest surgical planning in ophthalmology: ChatGPT versus Google Gemini analysis of retinal detachment cases","volume":"108","author":"Carl\u00e0","year":"","journal-title":"Br. J. Ophthalmol."},{"key":"B10","doi-asserted-by":"publisher","first-page":"e13930","DOI":"10.2196\/13930","article-title":"Applications and challenges of implementing artificial intelligence in medical education: integrative review","volume":"5","author":"Chan","year":"2019","journal-title":"JMIR Med. Educ."},{"key":"B11","first-page":"22243","article-title":"Big self-supervised models are strong semi-supervised learners","volume":"33","author":"Chen","year":"2020","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B12","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1007\/978-81-322-3972-7_19","article-title":"Natural language processing","volume-title":"Fundamentals of artificial intelligence","author":"Chowdhary","year":"2020"},{"key":"B13","article-title":"Multimodal-GPT: a vision and language model for dialogue with humans","author":"Gong","year":"2023"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e160","DOI":"10.1016\/j.wneu.2023.08.042","article-title":"GPT-4 artificial intelligence model outperforms ChatGPT, medical students, and Neurosurgery residents on Neurosurgery written board-like questions","volume":"179","author":"Guerra","year":"2023","journal-title":"World Neurosurg."},{"key":"B15","doi-asserted-by":"crossref","DOI":"10.3389\/fonc.2023.1219326","article-title":"Evaluating large language models on a highly-specialized topic, radiation oncology physics","author":"Holmes","year":"2023"},{"key":"B16","article-title":"Self-planning code generation with large language model","author":"Jiang","year":"2023"},{"key":"B17","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1016\/j.oret.2023.09.008","article-title":"The use of large language models to generate education materials about uveitis","volume":"8","author":"Kianian","year":"2023","journal-title":"Ophthalmol. Retina"},{"key":"B18","article-title":"Exploring the landscape of large language models in medical question answering: observations and open questions","author":"Korgul","year":"2023"},{"key":"B19","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digit. Health"},{"key":"B20","article-title":"Can large language models reason about medical questions?","author":"Li\u00e9vin","year":"2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"104770","DOI":"10.1016\/j.ebiom.2023.104770","article-title":"Benchmarking large language models' performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard","volume":"95","author":"Lim","year":"2023","journal-title":"EBioMedicine"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.1016\/j.metrad.2023.100017","volume-title":"Summary of ChatGPT-related research and perspective towards the future of large language models","author":"Liu","year":""},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2307.13693","article-title":"Evaluating large language models for radiology natural language processing","author":"Liu","year":""},{"key":"B24","doi-asserted-by":"publisher","first-page":"19","DOI":"10.3389\/fdata.2020.00019","article-title":"Reducing annotation burden through multimodal learning","volume":"3","author":"Lopez","year":"2020","journal-title":"Front. Big Data"},{"key":"B25","doi-asserted-by":"publisher","first-page":"e16048","DOI":"10.2196\/16048","article-title":"Introducing artificial intelligence training in medical education","volume":"5","author":"Paranjape","year":"2019","journal-title":"JMIR Med. Educ."},{"key":"B26","article-title":"A review on large language models: architectures, applications, taxonomies, open issues and challenges","author":"Raiaan","year":"2023"},{"key":"B27","doi-asserted-by":"publisher","first-page":"3530","DOI":"10.1038\/s41433-023-02563-3","article-title":"Comparative analysis of large language models in the Royal College of Ophthalmologists fellowship exams","volume":"37","author":"Raimondi","year":"2023","journal-title":"Eye"},{"key":"B28","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-023-46995-z","article-title":"Evaluation of the performance of GPT-3.5 and GPT-4 on the medical final examination","author":"Roso\u0142","year":"2023"},{"key":"B29","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1007\/s10916-023-01957-w","article-title":"Revolutionizing medical education: can ChatGPT boost subjective learning and expression?","volume":"47","author":"Seetharaman","year":"2023","journal-title":"J. Med. Syst."},{"key":"B30","article-title":"Large language models encode clinical knowledge","author":"Singhal","year":"2022"},{"key":"B31","doi-asserted-by":"publisher","first-page":"33","DOI":"10.33461\/uybisbbd.1244777","article-title":"The role of artificial intelligence in higher education: ChatGPT assessment for anatomy course","volume":"7","author":"Talan","year":"2023","journal-title":"Uluslar. Y\u00f6netim Bili\u015fim Sist. ve Bilgisayar Bilim. Derg."},{"key":"B32","first-page":"1","article-title":"The potential role of large language models in uveitis care: perspectives after ChatGPT and bard launch","author":"Tan","year":"2023","journal-title":"Ocul. Immunol. Inflamm."},{"key":"B33","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1177\/01410768231173123","article-title":"Large language models will not replace healthcare professionals: curbing popular fears and hype","volume":"116","author":"Thirunavukarasu","year":"2023","journal-title":"J. R. Soc. Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"342","DOI":"10.2307\/3090183","article-title":"Evidence-based medicine, clinical uncertainty, and learning to doctor","volume":"42","author":"Timmermans","year":"2001","journal-title":"J. health Soc. Behav."},{"key":"B35","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B36","doi-asserted-by":"publisher","first-page":"59","DOI":"10.7326\/m19-2548","article-title":"Should health care demand interpretable artificial intelligence or accept \u201cblack box\u201d medicine?","volume":"172","author":"Wang","year":"2020","journal-title":"Ann. Intern Med."},{"key":"B37","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1080\/23752696.2021.1883458","article-title":"Towards flexible personalized learning and the future educational system in the fourth industrial revolution in the wake of Covid-19","volume":"6","author":"Whalley","year":"2021","journal-title":"High. Educ. Pedagog."},{"key":"B38","article-title":"A comparative study of open-source large language models, GPT-4 and Claude 2: multiple-choice test taking in Nephrology","author":"Wu","year":"2023","journal-title":"arXiv Prepr. arXiv:2308.04709"}],"updated-by":[{"DOI":"10.3389\/fcell.2025.1664098","type":"corrigendum","label":"Corrigendum","source":"publisher","updated":{"date-parts":[[2025,8,13]],"date-time":"2025-08-13T00:00:00Z","timestamp":1755043200000}}],"container-title":["Frontiers in Cell and Developmental Biology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcell.2025.1574378\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T15:19:44Z","timestamp":1755184784000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcell.2025.1574378\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,7]]},"references-count":39,"alternative-id":["10.3389\/fcell.2025.1574378"],"URL":"https:\/\/doi.org\/10.3389\/fcell.2025.1574378","relation":{},"ISSN":["2296-634X"],"issn-type":[{"value":"2296-634X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,7]]},"article-number":"1574378"}}