{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T04:26:15Z","timestamp":1787199975967,"version":"build-2736575974"},"reference-count":31,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T00:00:00Z","timestamp":1787184000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Contemporary multimodal large language models (LLMs) can process both clinical text and medical images, but their reliability in retinal imaging-based question answering remains uncertain. This study evaluated the performance of six contemporary multimodal LLMs on retinal multiple-choice questions derived from OCTCases.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This cross-sectional benchmark study included 226 multiple-choice questions from 78 OCTCases retinal cases, comprising 151 image-based and 75 nonimage-based questions. Six multimodal LLMs were evaluated: ChatGPT-5.5 Instant, ChatGPT-5.5 Thinking, Grok 4, Gemini 3, DeepSeek V4, and Kimi 2.5. Model-selected answers were compared with the OCTCases answer key, which was reviewed by three retina specialists. Accuracy was assessed overall and by question subset. Pairwise model comparisons, inter-model agreement, voting-based group-answer performance, item difficulty distribution, and interface latency were analyzed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Overall accuracy ranged from 59.3 to 77.4%, with the highest accuracy observed for ChatGPT-5.5 Thinking, followed by Gemini 3 and ChatGPT-5.5 Instant. DeepSeek V4 showed the lowest overall accuracy. All models performed better on nonimage-based questions than on image-based questions. In the image-based subset, Gemini 3 achieved the highest accuracy, whereas DeepSeek V4 showed the lowest accuracy. Under the majority-vote rule, group answers were generated for 182 of 226 questions and achieved an accuracy of 89.6% among questions with a determinate group answer. The plurality-vote rule generated group answers for more questions but with lower accuracy. Inter-model agreement was generally higher for nonimage-based than image-based questions, and all questions answered incorrectly by all six models were image-based. Interface latency varied across models and showed no consistent association with response correctness.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Contemporary multimodal LLMs showed promising but uneven performance on OCTCases retinal questions. Their performance was consistently better on nonimage-based than image-based questions, indicating that retinal image interpretation remains a major limitation. Voting-based group answers may provide a useful reliability signal when model agreement is present, whereas model disagreement may help identify difficult or visually ambiguous questions requiring specialist review. These findings support continued benchmarking and cautious, specialist-supervised use of multimodal LLMs in retinal education and structured imaging question answering.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1934968","type":"journal-article","created":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T04:16:06Z","timestamp":1787199366000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Comparative performance of contemporary multimodal large language models in retinal imaging question answering"],"prefix":"10.3389","volume":"13","author":[{"given":"Xiaochen","family":"Gu","sequence":"first","affiliation":[{"name":"National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University","place":["Wenzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yizhou","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Eye, Ear, Nose, and Throat Hospital of Fudan University","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuanqiao","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Eye, Ear, Nose, and Throat Hospital of Fudan University","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Chen","sequence":"additional","affiliation":[{"name":"National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University","place":["Wenzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,20]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","year":"2023","journal-title":"Commun Med"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"201","DOI":"10.3928\/23258160-20160229-01","article-title":"Multimodal imaging in retinal disease: a consensus definition","volume":"47","author":"Novais","year":"2016","journal-title":"Ophthalmic Surg Lasers Imaging Retina"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1111\/opo.13284","article-title":"Utility of artificial intelligence-based large language models in ophthalmic care","volume":"44","author":"Biswas","year":"2024","journal-title":"Ophthalmic Physiol Opt"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1016\/j.survophthal.2025.02.009","article-title":"Advancing ophthalmology with large language models: applications, challenges, and future directions","volume":"70","author":"Zhang","year":"2025","journal-title":"Surv Ophthalmol"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"1741888","DOI":"10.3389\/fmed.2025.1741888","article-title":"Making chatbots more human: deep reasoning large language models in ophthalmology","volume":"12","author":"Lin","year":"2025","journal-title":"Front Med"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"e76947","DOI":"10.2196\/76947","article-title":"Evaluating large language models in ophthalmology: systematic review","volume":"27","author":"Zhang","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref7","article-title":"A novel ophthalmic benchmark for evaluating multimodal large language models with fundus photographs and OCT images","author":"Liang","year":"2025","journal-title":"arXiv"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"P75","DOI":"10.1016\/j.ophtha.2024.12.020","article-title":"Diabetic retinopathy preferred practice pattern\u00ae","volume":"132","author":"Lim","year":"2025","journal-title":"Ophthalmology"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"1548385","DOI":"10.3389\/frai.2025.1548385","article-title":"Online platform vs. doctors: a comparative exploration of congenital cataract patient education from virtual to reality","volume":"8","author":"Lin","year":"2025","journal-title":"Front Artif Intell"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"42778","DOI":"10.1038\/s41598-025-27020-x","article-title":"Assessing the quality and educational applicability of AI-generated anterior segment images in ophthalmology","volume":"15","author":"Yang","year":"2025","journal-title":"Sci Rep"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"4999","DOI":"10.1038\/s41598-025-88948-8","article-title":"Integrated visual and text-based analysis of ophthalmology clinical cases using a large language model","volume":"15","author":"Sorin","year":"2025","journal-title":"Sci Rep"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"25158414251340569","DOI":"10.1177\/25158414251340569","article-title":"Multimodal LLMs for retinal disease diagnosis via OCT: few-shot versus single-shot learning","volume":"17","author":"Agbareia","year":"2025","journal-title":"Ther Adv Ophthalmol"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"20552076261444617","DOI":"10.1177\/20552076261444617","article-title":"Comparison of three large language models in recognizing ophthalmological examination and supporting preoperative toric IOL planning","volume":"12","author":"Lin","year":"2026","journal-title":"Digit Health"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"101158","DOI":"10.1016\/j.xops.2026.101158","article-title":"Comparative evaluation of deep-reasoning large language models for ophthalmic emergencies","volume":"6","author":"Lin","year":"2026","journal-title":"Ophthalmol Sci"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e70080","DOI":"10.2196\/70080","article-title":"Large language model synergy for ensemble learning in medical question answering: design and evaluation study","volume":"27","author":"Yang","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1001\/jamaophthalmol.2023.6917","article-title":"Assessment of a large language model's responses to questions and cases about Glaucoma and retina management","volume":"142","author":"Huang","year":"2024","journal-title":"JAMA Ophthalmol"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"e60063","DOI":"10.2196\/60063","article-title":"EyeGPT for patient inquiries and medical education: development and validation of an ophthalmology large language model","volume":"26","author":"Chen","year":"2024","journal-title":"J Med Internet Res"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"48","DOI":"10.21037\/jmai-24-269","article-title":"GPT-4o vs. residency exams in ophthalmology\u2014a performance analysis","volume":"8","author":"Shvartz","year":"2025","journal-title":"J Med Artif Intell"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"20","DOI":"10.4103\/ijo.IJO_902_17","article-title":"Optical coherence tomography: a guide to interpretation of common macular diseases","volume":"66","author":"Bhende","year":"2018","journal-title":"Indian J Ophthalmol"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"P197","DOI":"10.1016\/j.ophtha.2024.12.019","article-title":"Idiopathic epiretinal membrane and vitreomacular traction preferred practice pattern\u00ae","volume":"132","author":"Bailey","year":"2025","journal-title":"Ophthalmology"},{"key":"ref21","article-title":"Boosting medical visual understanding from multi-granular language learning","author":"Li","year":"2025","journal-title":"arXiv"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2025.3598734","article-title":"VisionUnite: a vision-language foundation model for ophthalmology enhanced with clinical knowledge","author":"Li","year":"2025","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"4219","DOI":"10.1097\/ico.0000000000004219","article-title":"Comparative accuracy and safety of 4 large language models on cornea and external disease multiple-choice questions","author":"Argon","year":"2026","journal-title":"Cornea"},{"key":"ref24","first-page":"220311171","article-title":"Self-consistency improves chain of thought reasoning in language models","author":"Wang","year":"2022","journal-title":"arXiv"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"AIcs2400502","DOI":"10.1056\/AIcs2400502","article-title":"Combining multiple large language models improves diagnostic accuracy","volume":"1","author":"Barabucci","year":"2024","journal-title":"NEJM AI"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"e66917","DOI":"10.2196\/66917","article-title":"Benchmarking the confidence of large language models in answering clinical questions: cross-sectional evaluation study","volume":"13","author":"Omar","year":"2025","journal-title":"JMIR Med Inform"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.1093\/jamia\/ocae131","article-title":"Reasoning with large language models for medical question answering","volume":"31","author":"Lucas","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically applicable deep learning for diagnosis and referral in retinal disease","volume":"24","author":"De Fauw","year":"2018","journal-title":"Nat Med"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.aopr.2025.10.006","article-title":"Benchmarking large multimodal models for ophthalmic visual question answering with OphthalWeChat","volume":"6","author":"Xu","year":"2026","journal-title":"Adv Ophthalmol Pract Res"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"292","DOI":"10.3390\/jcm15010292","article-title":"Diagnostic performance of ChatGPT-4o in classifying idiopathic epiretinal membrane based on optical coherence tomography","volume":"15","author":"Sato","year":"2025","journal-title":"J Clin Med"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1093\/jamia\/ocae254","article-title":"Large language model uncertainty proxies: discrimination and calibration for medical diagnosis and treatment","volume":"32","author":"Savage","year":"2025","journal-title":"J Am Med Inform Assoc"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1934968\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T04:16:07Z","timestamp":1787199367000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1934968\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,20]]},"references-count":31,"alternative-id":["10.3389\/fmed.2026.1934968"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1934968","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,20]]},"article-number":"1934968"}}