{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T15:04:56Z","timestamp":1790262296012,"version":"4.1.0"},"reference-count":20,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Symptom-based localization of brainstem ischemia is challenging because of the anatomical complexity of the brainstem and the nonspecific overlap of clinical syndromes. Whether large language models (LLMs) can meaningfully assist in this task remains uncertain.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to compare the performance of several OpenAI LLMs and neurologists in localizing diffusion-weighted imaging (DWI)\u2013confirmed brainstem ischemic lesions based on symptom descriptions alone.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this retrospective single-center study, 109 patients with DWI-confirmed acute brainstem ischemia were included. Three neurologists and 6 LLMs (GPT-5, GPT-4, GPT-4.1, GPT-4o, o3, and o3-pro) predicted lesion localization using a combined anatomical-lateral end point (left or right midbrain, pons, and medulla) based on symptom descriptions alone. Overall and regional accuracy, the Cohen \u03ba, 6-class confusion matrices, and point-biserial correlations between symptom count and correct prediction were assessed. Because all raters evaluated the same cases, paired McNemar tests with Benjamini-Hochberg correction were used for pairwise performance comparisons.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>GPT-4 and GPT-4o achieved the highest overall accuracy (61\/109, 56%; 95% CI 46.1%\u201065.5%). Agreement with the DWI reference standard remained limited across all raters, with the Cohen \u03ba reaching a maximum of 0.291 for GPT-4o. Confusion matrices showed that higher performance was driven mainly by pontine cases, whereas misclassification remained frequent in mesencephalic and medullary lesions. Regional analyses outside the pons were imprecise because mesencephalic and medullary subgroups each contained only 16 cases. A higher number of documented symptoms was associated with correct prediction for GPT-4, GPT-5, GPT-o3, and 1 neurologist.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Although some LLMs showed higher relative accuracy than the participating neurologists, absolute performance remained limited and clinically insufficient. These findings are best interpreted as an exploratory benchmark under constrained conditions: absolute performance remained modest, agreement beyond chance was limited, and performance outside pontine lesions was inconclusive.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/87163","type":"journal-article","created":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T06:30:06Z","timestamp":1775197806000},"page":"e87163-e87163","source":"Crossref","is-referenced-by-count":1,"title":["Symptom-Only Localization of Brainstem Ischemia Using Large Language Models Versus Neurologists in Diffusion-Weighted Imaging\u2013Positive Cases: Retrospective Single-Center Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5420-1199","authenticated-orcid":false,"given":"Nedim","family":"Beste","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4014-7763","authenticated-orcid":false,"given":"Thomas","family":"Dratsch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0475-1472","authenticated-orcid":false,"given":"Jonathan","family":"Kottlors","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5453-1059","authenticated-orcid":false,"given":"Pia","family":"Flo\u00dfdorf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7296-4349","authenticated-orcid":false,"given":"Agni-Maria","family":"Konitsioti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0161-654X","authenticated-orcid":false,"given":"Lukas J","family":"Volz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7543-8555","authenticated-orcid":false,"given":"Uta","family":"Hanning","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4785-6394","authenticated-orcid":false,"given":"Daniel","family":"Pinto dos Santos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2620-7611","authenticated-orcid":false,"given":"Lukas","family":"Goertz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9978-7453","authenticated-orcid":false,"given":"David","family":"Zopfs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3712-2258","authenticated-orcid":false,"given":"Christoph","family":"Kabbasch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7734-611X","authenticated-orcid":false,"given":"Marc","family":"Schlamann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5600-5914","authenticated-orcid":false,"given":"Kai Roman","family":"Laukamp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2096-3448","authenticated-orcid":false,"given":"Michael","family":"Sch\u00f6nfeld","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,8]]},"reference":[{"key":"R1","unstructured":"Gowda SN Munakomi S . Brainstem stroke. In: StatPearls [Internet]. StatPearls Publishing; 2026. URL: https:\/\/pubmed.ncbi.nlm.nih.gov\/32809731\/ [Accessed 12-05-2026] [Medline: 32809731]"},{"issue":"5","key":"R2","doi-asserted-by":"publisher","first-page":"1843","DOI":"10.1161\/STROKEAHA.120.032457","article-title":"Improved detectability of brain stem ischemia by combining axial and coronal diffusion-weighted imaging","volume":"52","author":"Steffen","journal-title":"Stroke"},{"issue":"7","key":"R3","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0200092","article-title":"Improved detectability of acute and subacute brainstem infarctions by combining standard axial and thin-sliced sagittal DWI","volume":"13","author":"Sch\u00f6nfeld","journal-title":"PLoS One"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","DOI":"10.1038\/s44387-025-00011-z","article-title":"Large language models for disease diagnosis: a scoping review","volume":"1","author":"Zhou","journal-title":"NPJ Artif Intell"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.2196\/72062","article-title":"Large language models in medical diagnostics: scoping review with bibliometric analysis","volume":"27","author":"Su","journal-title":"J Med Internet Res"},{"issue":"10","key":"R6","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning: a randomized clinical trial","volume":"7","author":"Goh","journal-title":"JAMA Netw Open"},{"issue":"3","key":"R7","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooaf055","article-title":"Comparative analysis of large language models in clinical diagnosis: performance evaluation across common and complex medical cases","volume":"8","author":"Dinc","journal-title":"JAMIA Open"},{"issue":"8","key":"R8","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.25000","article-title":"Use of GPT-4 to analyze medical records of patients with extensive investigations and delayed diagnosis","volume":"6","author":"Shea","journal-title":"JAMA Netw Open"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.2196\/49082","article-title":"Beyond the hype-the actual role and risks of AI in today\u2019s medical practice: comparative-approach study","volume":"3","author":"Hansen","journal-title":"JMIR AI"},{"issue":"11","key":"R10","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-012799","article-title":"STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration","volume":"6","author":"Cohen","journal-title":"BMJ Open"},{"key":"R11","unstructured":"OpenAIAchiam J Adler S et al. GPT-4 technical report. arXiv. Preprint posted online on Mar 15, 2023. [doi: 10.48550\/arXiv.2303.08774]"},{"key":"R12","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. In: Koyejo S Mohamed S Agarwal A Belgrave D Cho K Oh A , editors. NIPS \u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems. Curran Associates Inc; 2022:1-43."},{"key":"R13","unstructured":"Sahoo P Singh AK Saha S Jain V Mondal S Chadha A . A systematic survey of prompt engineering in large language models: techniques and applications. arXiv. Preprint posted online on Feb 5, 2025. [doi: 10.48550\/arXiv.2402.07927]"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"114","DOI":"10.4258\/hir.2025.31.2.114","article-title":"Large language models in medicine: clinical applications, technical challenges, and ethical considerations","volume":"31","author":"Jung","journal-title":"Healthc Inform Res"},{"issue":"7","key":"R15","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1093\/qjmed\/hct057","article-title":"Stroke syndromes and clinical management","volume":"106","author":"Balami","journal-title":"QJM"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Bogousslavsky J . Stroke Syndromes. Cambridge University Press; 2001. ISBN: 9780521771429","DOI":"10.1017\/CBO9780511586521"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1176\/jnp.2010.22.1.iv","article-title":"The brainstem: anatomy, assessment, and clinical syndromes","volume":"22","author":"Hurley","journal-title":"J Neuropsychiatry Clin Neurosci"},{"issue":"10","key":"R18","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ad387d","article-title":"Advancing medical imaging with language models: featuring a spotlight on ChatGPT","volume":"69","author":"Hu","journal-title":"Phys Med Biol"},{"issue":"2","key":"R19","doi-asserted-by":"publisher","first-page":"135","DOI":"10.56977\/jicce.2025.23.2.135","article-title":"Integrating large language models with deep learning for medical imaging modality classification","volume":"23","author":"Ali","journal-title":"JICCE"},{"issue":"12","key":"R20","doi-asserted-by":"publisher","first-page":"1265","DOI":"10.1080\/14737140.2023.2286001","article-title":"Medical imaging and multimodal artificial intelligence models for streamlining and enhancing cancer care: opportunities and challenges","volume":"23","author":"Pierre","journal-title":"Expert Rev Anticancer Ther"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e87163","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:18Z","timestamp":1785536718000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e87163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,8]]},"references-count":20,"URL":"https:\/\/doi.org\/10.2196\/87163","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,8]]},"article-number":"v10i5e87163"}}