{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T18:11:21Z","timestamp":1793211081304,"version":"4.5.0"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T00:00:00Z","timestamp":1760486400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T00:00:00Z","timestamp":1760486400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012990","name":"Universit\u00e0 degli Studi di Siena","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012990","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Neurooncol"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background and purpose<\/jats:title>\n                    <jats:p>Large language models (LLMs) such as ChatGPT-4 have shown potential for medical decision support, but their reliability in specialized fields remains uncertain. This study aimed to evaluate ChatGPT-4\u2019s performance as a clinical decision support tool in neuro-oncology radiotherapy by comparing its treatment recommendations for patients with central nervous system tumors against a multidisciplinary tumor board\u2019s decisions, an independent specialist\u2019s opinion, and published guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>We prospectively collected 101 neuro-oncology cases (May 2024\u2013May 2025) presented at a tertiary-care tumor board. Key case details were entered into ChatGPT-4 with a standardized query asking whether to recommend radiotherapy and, if so, the target volumes and dose. The AI\u2019s recommendations were recorded and compared to the tumor board\u2019s consensus, a blinded radiation oncologist\u2019s recommendation, and ESMO guideline indications when applicable. Concordance rates (percentage agreement) and Cohen\u2019s kappa were calculated. Sensitivity and specificity were assessed using the reference decisions as ground truth. McNemar\u2019s test was used to evaluate any bias in discordant recommendations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      ChatGPT-4 matched the tumor board\u2019s radiotherapy recommendations in 76% of cases (\u03ba\u2009=\u20090.61). Agreement with the independent specialist was 79% (\u03ba\u2009=\u20090.58). In 61 low-complexity cases with clear guidelines, ChatGPT-4 concurred with guideline-based indications in 76.7% of cases, missing some recommended treatments (sensitivity 73%, specificity 100%). In intermediate-complexity scenarios, concordance with the tumor board was 70.8%, with most discrepancies due to the AI recommending treatment that experts did not (sensitivity 85.7%, specificity 64.7%). In high-complexity cases, agreement was 90.9% (sensitivity 100%, specificity 83.3%). Overall, ChatGPT-4 showed an\n                      <jats:italic>overtreatment bias<\/jats:italic>\n                      , more often recommending radiotherapy when the human experts chose observation (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.05 for AI vs. tumor board discordances). Its overall agreement (76%) was lower than that of the human specialist (90%).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>ChatGPT-4 can reproduce many expert radiotherapy decisions in neuro-oncology, reflecting substantial absorption of standard clinical practice. However, it cannot substitute for human judgment: the AI omitted some indicated treatments in straightforward cases and suggested unnecessary therapy in some borderline cases, indicating a lack of nuanced clinical reasoning. Careful human oversight is essential if such models are to be used for clinical decision support.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s11060-025-05254-z","type":"journal-article","created":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T14:21:30Z","timestamp":1760538090000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Assessing ChatGPT-4 as a clinical decision support tool in neuro-oncology radiotherapy: a prospective comparative study"],"prefix":"10.1007","volume":"176","author":[{"given":"Paolo","family":"Tini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Federica","family":"Novi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Flavio","family":"Donnini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Armando","family":"Perrella","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giulio","family":"Bagnacci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria Antonietta","family":"Mazzei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giuseppe","family":"Minniti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,10,15]]},"reference":[{"issue":"2","key":"5254_CR1","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepa\u00f1o C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V (2023) Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large Language models. PLOS Digit Health 2(2):e0000198. https:\/\/doi.org\/10.1371\/journal.pdig.0000198PMID: 36812645; PMCID: PMC9931230","journal-title":"PLOS Digit Health"},{"key":"5254_CR2","doi-asserted-by":"crossref","unstructured":"Wang P, Holmes J, Liu Z, Chen D, Liu T, Shen J, Liu W. A recent evaluation on the performance of LLMs on radiation oncology physics using questions of randomly shuffled options. Front Oncol; 15:1557064","DOI":"10.3389\/fonc.2025.1557064"},{"key":"5254_CR3","doi-asserted-by":"crossref","unstructured":"Schramm F, Haggenm\u00fcller S, Kather JN, Hetz MJ, Wies C, Michel MS, Wessels F, Brinker TJ (2025) Large language model use in clinical oncology. NPJ Precis Oncol. 2024;8(1):240. doi: 10.1038\/s41698-024-00733-4. PMID: 39443582; PMCID: PMC11499929","DOI":"10.1038\/s41698-024-00733-4"},{"key":"5254_CR4","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1038\/s41746-025-01824-7","volume":"8","author":"Y Hao","year":"2025","unstructured":"Hao Y, Qiu Z, Holmes J et al (2025) Large Language model integrations in cancer decision-making: a systematic review and meta-analysis. Npj Digit Med 8:450. https:\/\/doi.org\/10.1038\/s41746-025-01824-7","journal-title":"Npj Digit Med"},{"key":"5254_CR5","doi-asserted-by":"publisher","first-page":"110645","DOI":"10.1016\/j.radonc.2024.110645","volume":"202","author":"S Ramadan","year":"2025","unstructured":"Ramadan S, Mutsaers A, Chen PC, Bauman G, Velker V, Ahmad B, Arifin AJ, Nguyen TK, Palma D, Goodman CD (2025) Evaluating chatgpt\u2019s competency in radiation oncology: A comprehensive assessment across clinical scenarios. Radiother Oncol 202:110645 Epub 2024 Nov 19. PMID: 39571686","journal-title":"Radiother Oncol"},{"key":"5254_CR6","doi-asserted-by":"publisher","unstructured":"Vinod SK, Merie R, Harden S (2025) Quality of decision making in radiation oncology. Clin oncol (R coll Radiol). 38:103523. https:\/\/doi.org\/10.1016\/j.clon.2024.02.001. Epub 2024 Feb 6. PMID: 38342658","DOI":"10.1016\/j.clon.2024.02.001"},{"issue":"2","key":"5254_CR7","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/j.ijrobp.2015.01.046","volume":"92","author":"KL Moore","year":"2015","unstructured":"Moore KL, Schmidt R, Moiseenko V, Olsen LA, Tan J, Xiao Y, Galvin J, Pugh S, Seider MJ, Dicker AP, Bosch W, Michalski J, Mutic S (2015) Quantifying unnecessary normal tissue complication risks due to suboptimal planning: A secondary study of RTOG 0126. Int J Radiat Oncol Biol Phys 92(2):228\u2013235 Epub 2015 Apr 3. PMID: 25847605; PMCID: PMC4431941","journal-title":"Int J Radiat Oncol Biol Phys"},{"issue":"6","key":"5254_CR8","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1093\/jnci\/djt001","volume":"105","author":"N Ohri","year":"2013","unstructured":"Ohri N, Shen X, Dicker AP, Doyle LA, Harrison AS, Showalter TN (2013) Radiotherapy protocol deviations and clinical outcomes: a meta-analysis of cooperative group clinical trials. J Natl Cancer Inst 105(6):387\u2013393. https:\/\/doi.org\/10.1093\/jnci\/djt001Epub 2013 Mar 6. PMID: 23468460; PMCID: PMC3601950","journal-title":"J Natl Cancer Inst"},{"key":"5254_CR9","doi-asserted-by":"publisher","unstructured":"Chuang WK, Kao YS, Liu YT, Lee CY Assessing ChatGPT for Clinical Decision-Making in Radiation Oncology, With Open-Ended Questions and Images. Pract Radiat Oncol. 2025 Apr 29:S1879-8500(25)00115-8. https:\/\/doi.org\/10.1016\/j.prro.2025.04.009. Epub ahead of print. PMID: 40311921","DOI":"10.1016\/j.prro.2025.04.009"},{"key":"5254_CR10","doi-asserted-by":"crossref","unstructured":"Deng J, Zubair A, Park YJ, Affan E, Zuo QK (2024) The use of large language models in medicine: proceeding with caution. Curr Med Res Opin. ;40(2):151\u2013153. doi: 10.1080\/03007995.2023.2295411. Epub 2024 Jan 24. PMID: 38093584","DOI":"10.1080\/03007995.2023.2295411"},{"issue":"1","key":"5254_CR11","doi-asserted-by":"publisher","first-page":"642","DOI":"10.1038\/s41467-024-55628-6","volume":"16","author":"M Griot","year":"2025","unstructured":"Griot M, Hemptinne C, Vanderdonckt J, Yuksel D (2025) Large Language models lack essential metacognition for reliable medical reasoning. Nat Commun 16(1):642. https:\/\/doi.org\/10.1038\/s41467-024-55628-6PMID: 39809759; PMCID: PMC11733150","journal-title":"Nat Commun"},{"issue":"11","key":"5254_CR12","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.1016\/j.annonc.2021.07.016","volume":"32","author":"E Le Rhun","year":"2021","unstructured":"Le Rhun E, Guckenberger M, Smits M, Dummer R, Bachelot T, Sahm F, Galldiks N, de Azambuja E, Berghoff AS, Metellus P, Peters S, Hong YK, Winkler F, Schadendorf D, van den Bent M, Seoane J, Stahel R, Minniti G, Wesseling P, Weller M, Preusser M, EANO Executive Board and ESMO Guidelines Committee (2021) Electronic address: clinicalguidelines@esmo.org. EANO-ESMO clinical practice guidelines for diagnosis, treatment and follow-up of patients with brain metastasis from solid tumours. Ann Oncol 32(11):1332\u20131347. https:\/\/doi.org\/10.1016\/j.annonc.2021.07.016Epub 2021 Aug 6. PMID: 34364998","journal-title":"Ann Oncol"},{"issue":"Suppl 3","key":"5254_CR13","doi-asserted-by":"publisher","first-page":"iii93","DOI":"10.1093\/annonc\/mdu050","volume":"25","author":"R Stupp","year":"2014","unstructured":"Stupp R, Brada M, van den Bent MJ, Tonn JC, Pentheroudakis G, ESMO Guidelines Working Group (2014) High-grade glioma: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol 25(Suppl 3):iii93\u2013101. https:\/\/doi.org\/10.1093\/annonc\/mdu050Epub 2014 Apr 29. PMID: 24782454","journal-title":"Ann Oncol"},{"issue":"5","key":"5254_CR14","doi-asserted-by":"publisher","first-page":"101624","DOI":"10.1016\/j.esmoop.2023.101624","volume":"8","author":"E Le Rhun","year":"2023","unstructured":"Le Rhun E, Weller M, van den Bent M, Brandsma D, Furtner J, Rud\u00e0 R, Schadendorf D, Seoane J, Tonn JC, Wesseling P, Wick W, Minniti G, Peters S, Curigliano G, Preusser M, EANO Guidelines Committee and ESMO Guidelines Committee (2023) Electronic address: clinicalguidelines@esmo.org. Leptomeningeal metastasis from solid tumours: EANO-ESMO clinical practice guideline for diagnosis, treatment and follow-up. ESMO Open 8(5):101624. https:\/\/doi.org\/10.1016\/j.esmoop.2023.101624Epub 2023 Sep 19. PMID: 37863528; PMCID: PMC10619142","journal-title":"ESMO Open"},{"key":"5254_CR15","doi-asserted-by":"publisher","first-page":"109663","DOI":"10.1016\/j.radonc.2023.109663","volume":"184","author":"M Niyazi","year":"2023","unstructured":"Niyazi M, Andratschke N, Bendszus M, Chalmers AJ, Erridge SC, Galldiks N, Lagerwaard FJ, Navarria P, Munck Af Rosensch\u00f6ld P, Ricardi U, van den Bent MJ, Weller M, Belka C, Minniti G (2023) ESTRO-EANO guideline on target delineation and radiotherapy details for glioblastoma. Radiother Oncol 184:109663 Epub 2023 Apr 13. PMID: 37059335","journal-title":"Radiother Oncol"},{"issue":"1","key":"5254_CR16","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1186\/s12911-025-02954-4","volume":"25","author":"S Shool","year":"2025","unstructured":"Shool S, Adimi S, Saboori Amleshi R, Bitaraf E, Golpira R, Tara M (2025) A systematic review of large Language model (LLM) evaluations in clinical medicine. BMC Med Inf Decis Mak 25(1):117. https:\/\/doi.org\/10.1186\/s12911-025-02954-4PMID: 40055694; PMCID: PMC11889796","journal-title":"BMC Med Inf Decis Mak"},{"issue":"3","key":"5254_CR17","doi-asserted-by":"publisher","first-page":"100045","DOI":"10.1016\/j.metrad.2023.100045","volume":"1","author":"C Liu","year":"2023","unstructured":"Liu C, Liu Z, Holmes J, Zhang L, Zhang L, Ding Y, Shu P, Wu Z, Dai H, Li Y, Shen D, Liu N, Li Q, Li X, Zhu D, Liu T, Liu W (2023) Artificial general intelligence for radiation oncology. Meta Radiol 1(3):100045 Epub 2023 Nov 24. PMID: 38344271; PMCID: PMC10857824","journal-title":"Meta Radiol"},{"key":"5254_CR18","doi-asserted-by":"publisher","first-page":"108013","DOI":"10.1016\/j.cmpb.2024.108013","volume":"245","author":"J Li","year":"2024","unstructured":"Li J, Dada A, Puladi B, Kleesiek J, Egger J (2024) ChatGPT in healthcare: A taxonomy and systematic review. Comput Methods Programs Biomed 245:108013. https:\/\/doi.org\/10.1016\/j.cmpb.2024.108013Epub 2024 Jan 15. PMID: 38262126","journal-title":"Comput Methods Programs Biomed"},{"issue":"9","key":"5254_CR19","doi-asserted-by":"publisher","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","volume":"30","author":"P Hager","year":"2024","unstructured":"Hager P, Jungmann F, Holland R, Bhagat K, Hubrecht I, Knauer M, Vielhauer J, Makowski M, Braren R, Kaissis G, Rueckert D (2024) Evaluation and mitigation of the limitations of large Language models in clinical decision-making. Nat Med 30(9):2613\u20132622. https:\/\/doi.org\/10.1038\/s41591-024-03097-1Epub 2024 Jul 4. PMID: 38965432; PMCID: PMC11405275","journal-title":"Nat Med"},{"issue":"1","key":"5254_CR20","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1111\/jep.12840","volume":"24","author":"S Salloch","year":"2018","unstructured":"Salloch S, Otte I, Reinacher-Schick A, Vollmann J (2018) What does physicians\u2019 clinical expertise contribute to oncologic decision-making? A qualitative interview study. J Eval Clin Pract 24(1):180\u2013186. https:\/\/doi.org\/10.1111\/jep.12840Epub 2017 Oct 27. PMID: 29076629","journal-title":"J Eval Clin Pract"},{"issue":"6","key":"5254_CR21","doi-asserted-by":"publisher","first-page":"1873","DOI":"10.1038\/s41591-025-03626-6","volume":"31","author":"M Omar","year":"2025","unstructured":"Omar M, Soffer S, Agbareia R, Bragazzi NL, Apakama DU, Horowitz CR, Charney AW, Freeman R, Kummer B, Glicksberg BS, Nadkarni GN, Klang E (2025) Sociodemographic biases in medical decision making by large Language models. Nat Med 31(6):1873\u20131881. https:\/\/doi.org\/10.1038\/s41591-025-03626-6Epub 2025 Apr 7. PMID: 40195448","journal-title":"Nat Med"},{"issue":"1","key":"5254_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.imed.2025.01.001","volume":"5","author":"J Li","year":"2025","unstructured":"Li J, Zhou Z, Lyu H, Wang Z (2025) Large Language models-powered clinical decision support: enhancing or replacing human expertise? Intell Med 5(1):1\u20134. https:\/\/doi.org\/10.1016\/j.imed.2025.01.001","journal-title":"Intell Med"},{"key":"5254_CR23","doi-asserted-by":"publisher","first-page":"1219326","DOI":"10.3389\/fonc.2023.1219326","volume":"13","author":"J Holmes","year":"2023","unstructured":"Holmes J, Liu Z, Zhang L, Ding Y, Sio TT, McGee LA, Ashman JB, Li X, Liu T, Shen J, Liu W (2023) Evaluating large Language models on a highly-specialized topic, radiation oncology physics. Front Oncol 13:1219326 PMID: 37529688; PMCID: PMC10388568","journal-title":"Front Oncol"}],"container-title":["Journal of Neuro-Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11060-025-05254-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11060-025-05254-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11060-025-05254-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T12:28:25Z","timestamp":1781612905000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11060-025-05254-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,15]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["5254"],"URL":"https:\/\/doi.org\/10.1007\/s11060-025-05254-z","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-7461621\/v1","asserted-by":"object"}]},"ISSN":["0167-594X","1573-7373"],"issn-type":[{"value":"0167-594X","type":"print"},{"value":"1573-7373","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,15]]},"assertion":[{"value":"26 August 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 October 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"No clinical decisions were made by the AI \u2013 its outputs were recorded for analysis only and did not influence patient care. All patient information was de-identified. The institutional review board approved the study protocol. The approval covered the use of AI for analysis of fully de-identified patient data, in compliance with ethical standards for AI research in healthcare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"13"}}