{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T20:58:29Z","timestamp":1792184309656,"version":"4.4.0"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T00:00:00Z","timestamp":1765411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oncol."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Large language models (LLM) have shown great potential in clinical decision support and medical education. GPT-5 is a novel LLM system that has been specifically marketed towards oncology use. This study comprehensively benchmarks GPT-5 for the field of radiation oncology.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Performance was assessed using two complementary benchmarks: (i) the American College of Radiology Radiation Oncology In-Training Examination (TXIT, 2021), comprising 300 multiple-choice items, and (ii) a curated set of 60 authentic radiation oncologic vignettes representing diverse disease sites and treatment indications. For the vignette evaluation, GPT-5 was instructed to generate structured therapeutic plans and concise two-line summaries. Four board-certified radiation oncologists independently rated outputs for correctness, comprehensiveness, and hallucinations. Inter-rater reliability was quantified using Fleiss\u2019\n                      <jats:italic>\u03ba<\/jats:italic>\n                      . GPT-5\u201314 results were compared to published GPT-3.5 and GPT-4 baselines.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      On the TXIT benchmark, GPT-5 achieved a mean accuracy of 92.8%, outperforming GPT-4 (78.8%) and GPT-3.5 (62.1%). Domain-specific gains were most pronounced in dose specification and diagnosis. In the vignette evaluation, GPT-5\u2019s treatment recommendations were rated highly for correctness (mean 3.24\/4, 95% CI: 3.11\u20133.38) and comprehensiveness (3.59\/4, 95% CI: 3.49\u20133.69). Hallucinations were rare, flagged in 10.0% of all individual reviewer assessments (24 of 240), and no patient case reached majority consensus for their presence. Inter-rater agreement was low (Fleiss\u2019\n                      <jats:italic>\u03ba<\/jats:italic>\n                      0.083 for correctness), reflecting inherent variability in clinical judgment. Errors clustered in complex scenarios requiring precise trial knowledge or detailed clinical adaptation.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>GPT-5 clearly outperformed prior model variants on the radiation oncology multiple-choice benchmark. Although GPT-5 exhibited favorable performance in generating real-world radiation oncology treatment recommendations, correctness ratings indicate room for further improvement. While hallucinations were infrequent, the presence of substantive errors underscores that GPT-5-generated recommendations require rigorous expert oversight before clinical implementation. In addition, considerable inter-rater variability highlights the challenge of achieving consistent expert evaluation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fonc.2025.1695468","type":"journal-article","created":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T05:19:49Z","timestamp":1765430389000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Benchmarking GPT-5 in radiation oncology: measurable gains, but persistent need for expert oversight"],"prefix":"10.3389","volume":"15","author":[{"given":"Ugur","family":"Din\u00e7","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jibak","family":"Sarkar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philipp","family":"Schubert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sabine","family":"Semrau","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Weissmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andre","family":"Karius","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johann","family":"Brand","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernd-Niklas","family":"Axer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed","family":"Gomaa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pluvio","family":"Stephan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ishita","family":"Sheth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sogand","family":"Beirami","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Annette","family":"Schwarz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Udo","family":"Gaipl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Frey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christoph","family":"Bert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefanie","family":"Corradini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rainer","family":"Fietkau","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florian","family":"Putz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,12,11]]},"reference":[{"key":"B1","article-title":"Scaling laws for neural language models","author":"Kaplan","year":"2020","journal-title":"arXiv"},{"key":"B2","article-title":"Deep reinforcement learning from human preferences","volume":"30","author":"Christiano","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"B3","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"B4","article-title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer","author":"Shazeer","year":"2017","journal-title":"arXiv"},{"key":"B5","first-page":"1","article-title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity","volume":"23","author":"Fedus","year":"2022","journal-title":"J Mach Learn Res"},{"key":"B6","first-page":"1","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"B7","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"B8","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btz682","article-title":"Biobert: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"B10","first-page":"1","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2021","journal-title":"ACM Trans Computing Healthcare (HEALTH)"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-19","article-title":"Publicly available clinical bert embeddings","author":"Alsentzer","year":"2019","journal-title":"arXiv"},{"key":"B12","first-page":"1","article-title":"Gpt-4 technical report","year":"2023","journal-title":"arXiv"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of chatgpt on usmle: potential for ai-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PloS digital Health"},{"key":"B15","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijrobp.2023.03.075","article-title":"Chatgpt: can a natural language processing tool be trusted for radiation oncology use","volume":"116","author":"Ebrahimi","year":"2023","journal-title":"Int J Radiat Oncology Biology Phys"},{"key":"B16","doi-asserted-by":"publisher","first-page":"e244630","DOI":"10.1001\/jamanetworkopen.2024.4630","article-title":"Quality of large language model responses to radiation oncology patient care questions","volume":"7","author":"Yalamanchili","year":"2024","journal-title":"JAMA network Open"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1136\/bmjonc-2025-000759","article-title":"Large language models in oncology: a review","volume":"4","author":"Chen","year":"2025","journal-title":"BMJ Oncol"},{"key":"B18","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1038\/s41746-025-01824-7","article-title":"Large language model integrations in cancer decision-making: a systematic review and meta-analysis","volume":"8","author":"Hao","year":"2025","journal-title":"NPJ Digital Med"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1007\/s00066-024-02272-0","article-title":"Principles of artificial intelligence in radiooncology","volume":"201","author":"Huang","year":"2025","journal-title":"Strahlentherapie und Onkologie"},{"key":"B20","doi-asserted-by":"publisher","first-page":"17133","DOI":"10.1038\/s41598-025-02026-7","article-title":"A self-supervised multimodal deep learning approach to differentiate post-radiotherapy progression from pseudoprogression in glioblastoma","volume":"15","author":"Gomaa","year":"2025","journal-title":"Sci Rep"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1002\/mp.15863","article-title":"Deep learning for brain metastasis detection and segmentation in longitudinal mri data","volume":"49","author":"Huang","year":"2022","journal-title":"Med Phys"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.1115258","article-title":"Deep learning for automatic head and neck lymph node level delineation provides expert-level accuracy","volume":"13","author":"Weissmann","year":"2023","journal-title":"Front Oncol"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/s00066-022-02039-5","article-title":"Improving cbct image quality to the ct level using reggan in esophageal cancer adaptive radiotherapy","volume":"199","author":"Wang","year":"2023","journal-title":"Strahlentherapie und Onkologie"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1002\/mp.13953","article-title":"A feasibility study on deep learning-based radiotherapy dose calculation","volume":"47","author":"Xing","year":"2020","journal-title":"Med Phys"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/s00066-024-02262-2","article-title":"Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives","volume":"201","author":"Erdur","year":"2025","journal-title":"Strahlentherapie und Onkologie"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2024.1493716","article-title":"Fine-tuning a local llama-3 large language model for automated privacy-preserving physician letter generation in radiation oncology","volume":"7","author":"Hou","year":"2025","journal-title":"Front Artif Intell"},{"key":"B27","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1038\/s44172-024-00271-8","article-title":"Interactive computer-aided diagnosis on medical image using large language models","volume":"3","author":"Wang","year":"2024","journal-title":"Commun Eng"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/adf02c","article-title":"Automated radiotherapy treatment planning guided by gpt-4vision","author":"Liu","year":"2024","journal-title":"Phys Med Biol"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.1219326","article-title":"Evaluating large language models on a highly-specialized topic, radiation oncology physics","volume":"13","author":"Holmes","year":"2023","journal-title":"Front Oncol"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2025.1557064","article-title":"A recent evaluation on the performance of llms on radiation oncology physics using questions of randomly shuffled options","volume":"15","author":"Wang","year":"2025","journal-title":"Front Oncol"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.2196\/69313","article-title":"Role of artificial intelligence in surgical training by assessing gpt-4 and gpt-4o on the Japan surgical board examination with text-only and image-accompanied questions: Performance evaluation study","volume":"11","author":"Maruyama","year":"2025","journal-title":"JMIR Med Educ"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2025.1441747","article-title":"Gpt-4\u2019s capabilities for formative and summative assessments in norwegian medicine exams\u2014an intrinsic case study in the early phase of intervention","volume":"12","author":"Krumsvik","year":"2025","journal-title":"Front Med"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1056\/AIcs2500025","article-title":"Exploring large language models for specialist-level oncology care","author":"Palepu","year":"2024","journal-title":"arXiv"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat Med"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2023.11.062","article-title":"Exploring the capabilities and limitations of large language models for radiation oncology decision support","volume":"118","author":"Putz","year":"2024","journal-title":"Int J Radiat Oncology Biology Phys"},{"key":"B36","article-title":"Constitutional ai: Harmlessness from ai feedback","author":"Bai","year":"2022","journal-title":"arXiv"},{"key":"B37","unstructured":"OpenAI\n          \n          Gpt-5 system card"},{"key":"B38","unstructured":"OpenAI\n          \n          Gpt-5"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.1265024","article-title":"Benchmarking chatgpt-4 on a radiation oncology in-training exam and red journal gray zone cases: potentials and challenges for ai-assisted medical education and decision making in radiation oncology","volume":"13","author":"Huang","year":"2023","journal-title":"Front Oncol"},{"key":"B40","doi-asserted-by":"publisher","first-page":"41","DOI":"10.37549\/ARO1287","article-title":"Analysis of the radiation oncology in-training examination content using a clinical care path conceptual framework","volume":"10","author":"Rogacki","year":"2021","journal-title":"Appl Radiat Oncol"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/S1470-2045(19)30395-X","article-title":"Adjuvant chemoradiotherapy versus radiotherapy alone in women with high-risk endometrial cancer (portec-3): patterns of recurrence and post-hoc survival analysis of a randomised phase 3 trial","volume":"20","author":"de Boer","year":"2019","journal-title":"Lancet Oncol"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2020.0147","article-title":"Outcomes of observation vs stereotactic ablative radiation for oligometastatic prostate cancer: the oriole phase 2 randomized clinical trial","volume":"6","author":"Phillips","year":"2020","journal-title":"JAMA Oncol"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21388","article-title":"The eighth edition ajcc cancer staging manual: continuing to build a bridge from a population-based to a more \u201cpersonalized\u201d approach to cancer staging","volume":"67","author":"Amin","year":"2017","journal-title":"CA: Cancer J Clin"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1007\/s00066-024-02342-3","article-title":"Patient-and clinician-based evaluation of large language models for patient education in prostate cancer radiotherapy","volume":"201","author":"Trapp","year":"2025","journal-title":"Strahlentherapie und Onkologie"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02412-6","article-title":"Large language model ai chatbots require approval as medical devices","volume":"29","author":"Gilbert","year":"2023","journal-title":"Nat Med"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocad072","article-title":"Using ai-generated suggestions from chatgpt to optimize clinical decision support","volume":"30","author":"Liu","year":"2023","journal-title":"J Am Med Inf Assoc"}],"updated-by":[{"DOI":"10.3389\/fonc.2025.1766619","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T00:00:00Z","timestamp":1767657600000}}],"container-title":["Frontiers in Oncology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2025.1695468\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T14:58:03Z","timestamp":1767711483000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2025.1695468\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,11]]},"references-count":46,"alternative-id":["10.3389\/fonc.2025.1695468"],"URL":"https:\/\/doi.org\/10.3389\/fonc.2025.1695468","relation":{},"ISSN":["2234-943X"],"issn-type":[{"value":"2234-943X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,11]]},"article-number":"1695468"}}