{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T06:10:00Z","timestamp":1790143800164,"version":"4.0.1"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oncol."],"abstract":"<jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>This study evaluated GPT-4o, GPT-5.4, and two DeepSeek platform configurations using 1, 053 examination-preparation questions from a publicly and commercially available 2025 exercise collection for the Chinese National Radiation Oncology Technology Qualification Examination (Intermediate Level). We assessed accuracy rates, identical-response coverage, accuracy consensus, and model-reported latency estimates.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Four configurations\u2014GPT-4o, GPT-5.4, DS-Fast, and DS-Expert\u2014were tested using an identical zero-shot Chinese prompt. Accuracy rates were summarized with Wilson 95% confidence intervals. Pairwise differences in the primary Chinese-language analysis were assessed using exact McNemar\u2019s tests with Holm adjustment. A descriptive language-controlled analysis evaluated English translations of the same questions using an equivalent English prompt.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Overall accuracy rates were 56.7% (95% CI, 53.7\u201359.7) for GPT-4o, 66.1% (63.2\u201368.9) for GPT-5.4, 98.4% (97.4\u201399.0) for DS-Fast, and 99.8% (99.3\u201399.9) for DS-Expert. All six overall pairwise comparisons remained significant after Holm adjustment. DS-Fast and DS-Expert produced identical responses for 1, 034 of 1, 053 questions (98.2%), all of which were correct. GPT-5.4 and DS-Expert showed 100% conditional accuracy among shared responses, but their identical-response coverage was only 65.9% (694\/1, 053). Model-reported latency estimates were lowest for DS-Fast (0.68 \u00b1 1.00 seconds) and highest for DS-Expert (3.69 \u00b1 2.24 seconds). Under the English-translated condition, overall accuracy rates were 62.6%, 61.5%, 66.5%, and 76.9%, respectively. DS-Expert remained the most accurate configuration, although its advantage over the GPT models was reduced.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>DeepSeek configurations outperformed the GPT models on this Chinese-language examination-preparation question set, with DS-Expert achieving the highest accuracy. However, performance varied substantially by question language. These findings support further evaluation for answer verification and practice-question review, but they do not establish educational effectiveness.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fonc.2026.1946351","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T05:49:10Z","timestamp":1790142550000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking large language models on a Chinese radiation oncology technology examination-preparation question set: accuracy, consensus, and efficiency for AI assisted education"],"prefix":"10.3389","volume":"16","author":[{"given":"Heling","family":"Zhu","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongguang","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyu","family":"Long","sequence":"additional","affiliation":[{"name":"Huayi Boao (Beijing) Quantum Technology Co., Ltd.","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanchen","family":"Liu","sequence":"additional","affiliation":[{"name":"Huayi Boao (Beijing) Quantum Technology Co., Ltd.","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tingtian","family":"Pang","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenbo","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bo","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Yao","sequence":"additional","affiliation":[{"name":"Huayi Boao (Beijing) Quantum Technology Co., Ltd.","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1002\/hcs2.61","article-title":"Large language models in health care: Development, applications, and challenges","volume":"2","author":"Yang","year":"2023","journal-title":"Health Care Sci"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.2196\/60807","article-title":"Performance of ChatGPT across different versions in medical licensing examinations worldwide: Systematic review and meta-analysis","volume":"26","author":"Liu","year":"2024","journal-title":"J Med Internet Res"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.2196\/63430","article-title":"ChatGPT-4 Omni performance in USMLE disciplines and clinical skills: Comparative analysis","volume":"10","author":"Bicknell","year":"2024","journal-title":"JMIR Med Educ"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PloS Digital Health"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1372","DOI":"10.1186\/s12909-024-06309-x","article-title":"Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: A comparison study","volume":"24","author":"Chen","year":"2024","journal-title":"BMC Med Educ"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13187-025-02652-9","article-title":"A comparative analysis of GPT-4o and ERNIE Bot in a Chinese radiation oncology exam","volume":"41","author":"Wang","year":"2025","journal-title":"J Cancer Educ"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1080\/10872981.2025.2592430","article-title":"When AI models take the exam: Large language models vs medical students on multiple-choice course exams","volume":"30","author":"Ros-Arlanz\u00f3n","year":"2025","journal-title":"Med Educ Online"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1371","DOI":"10.1038\/s41598-025-31251-3","article-title":"Comparative performance of ChatGPT-4o, ChatGPT-5, and gemini 2.5 flash on Persian internal medicine subspecialty board exams","volume":"6","author":"Sheikhalishahi","year":"2025","journal-title":"Sci Rep"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2025.1557064","article-title":"A recent evaluation on the performance of LLMs on radiation oncology physics using questions of randomly shuffled options","volume":"15","author":"Wang","year":"2025","journal-title":"Front Oncol"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.2196\/84918","article-title":"Factors influencing adoption of large language models in health care: Multicenter cross-sectional mixed methods observational study","volume":"27","author":"Yang","year":"2025","journal-title":"J Med Internet Res"},{"key":"B12","article-title":"Gpt-4o system card","author":"Hurst","year":"2024","journal-title":"arXiv preprint arXiv:241021276"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11432-023-3740-x","article-title":"Feedback is all you need: From ChatGPT to autonomous driving","volume":"66","author":"Chen","year":"2023","journal-title":"Sci China Inf Sci"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.1265024","article-title":"Benchmarking ChatGPT-4 on a radiation oncology in-training exam and Red Journal Gray Zone cases: Potentials and challenges for ai-assisted medical education and decision making in radiation oncology","volume":"13","author":"Huang","year":"2023","journal-title":"Front Oncol"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2024.07.1362","article-title":"Benchmarking OpenAI ChatGPT and Google Bard on Radiation Physics Study Exams (RAPHEX) and implication for their use as a learning tool","volume":"120","author":"Dvorak","year":"2024","journal-title":"Int J Radiat Oncol Biol Phys"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.2196\/69485","article-title":"Performance of large language models in the non-english context: Qualitative study of models trained on different languages in chinese medical examinations","volume":"13","author":"Yao","year":"2025","journal-title":"JMIR Med Inf"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.2196\/48002","article-title":"Performance of GPT-3.5 and GPT-4 on the Japanese medical licensing examination: Comparison study","volume":"9","author":"Takagi","year":"2023","journal-title":"JMIR Med Educ"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.4174\/astr.2023.104.5.269","article-title":"ChatGPT goes to the operating room: Evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models","volume":"104","author":"Oh","year":"2023","journal-title":"Ann Surg Treat Res"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.2196\/63731","article-title":"Qwen-2.5 outperforms other large language models in the Chinese National Nursing Licensing Examination: Retrospective cross-sectional comparative study","volume":"13","author":"Zhu","year":"2025","journal-title":"JMIR Med Inform"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-025-09422-z","article-title":"DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning","volume":"645","author":"Guo","year":"2025","journal-title":"Nature"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.2196\/73469","article-title":"Evaluating the performance of DeepSeek-R1 and DeepSeek-V3 versus OpenAI models in the Chinese national medical licensing examination: Cross-sectional comparative study","volume":"11","author":"Wang","year":"2025","journal-title":"JMIR Med Educ"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1097\/jcma.0000000000000946","article-title":"ChatGPT failed Taiwan\u2019s family medicine board exam","volume":"86","author":"Weng","year":"2023","journal-title":"J Chin Med Assoc"},{"key":"B23","article-title":"Deepseek-v4: Towards highly efficient million-token context intelligence","author":"Xu","year":"2026","journal-title":"arXiv preprint arXiv:260619348"},{"key":"B24","volume-title":"National Health Professional and Technical Qualification Examination Guide. Oncology and Radiation Oncology Technology: Synchronized Exercise Collection (2025 Edition)","author":"Yuan","year":"2024"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0346518","article-title":"Performance benchmarking of LLMs on Chinese national medical licensing education: Cross-lingual and question-type effects","volume":"21","author":"Tang","year":"2026","journal-title":"PloS One"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1080\/0142159x.2024.2314198","article-title":"A scoping review of artificial intelligence in medical education: BEME Guide No. 84","volume":"46","author":"Gordon","year":"2024","journal-title":"Med Teacher"}],"container-title":["Frontiers in Oncology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2026.1946351\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T05:49:11Z","timestamp":1790142551000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2026.1946351\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":26,"alternative-id":["10.3389\/fonc.2026.1946351"],"URL":"https:\/\/doi.org\/10.3389\/fonc.2026.1946351","relation":{},"ISSN":["2234-943X"],"issn-type":[{"value":"2234-943X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]},"article-number":"1946351"}}