{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T05:48:58Z","timestamp":1790660938190,"version":"4.1.0"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec>\n                    <jats:title>Rationale and objectives<\/jats:title>\n                    <jats:p>Large language model (LLM) are increasingly explored for oncology decision support, yet their alignment with real-world clinical practice across varying disease complexities remains insufficiently characterized. This study aimed to evaluate and compare the accuracy, stability, and concordance of two advanced LLMs\u2014DeepSeek V3.1 and ChatGPT-5\u2014against experienced oncologists in generating breast cancer treatment plans within a specific clinical setting.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>This retrospective study compared the performance of DeepSeek V3.1 and ChatGPT-5 with senior oncologists using de-identified records from 213 breast cancer patients (Stages I\u2013IV). To assess effectiveness, we implemented a multidimensional evaluation framework: accuracy was measured using a 5-point Likert scale by three independent, blinded expert reviewers; internal consistency was quantified via variance and coefficient of variation; and clinical concordance was evaluated using a structured five-level scoring system. Statistical analyses, including ANOVA and ordinal regression, were used to examine the impact of disease stage on AI-human agreement.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Under standardized retrospective review conditions, LLM-generated recommendations demonstrated higher expert-rated guideline concordance and lower variability than historical real-world oncologist plans. Specifically, DeepSeek V3.1 achieved the highest expert-rated accuracy scores with minimal internal variance (4.91\u2009\u00b1\u20090.36), outperforming both ChatGPT-5 (4.65\u2009\u00b1\u20090.62) and clinicians (3.82\u2009\u00b1\u20090.63,\n                      <jats:italic>P<\/jats:italic>\n                      \u2009&amp;lt;\u20090.001). While AI outputs exhibited high mutual consistency (74.2%), expert evaluations revealed a significant decline in AI-clinician agreement as disease stage advanced (\n                      <jats:italic>P<\/jats:italic>\n                      \u2009&amp;lt;\u20090.001), particularly in Stage IV cases where clinicians prioritized real-world constraints such as financial toxicity.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Advanced LLMs, particularly DeepSeek V3.1, demonstrated strong performance in generating standardized, guideline-concordant breast cancer treatment plans, showing superior consistency over human specialists in protocol-driven scenarios. However, the widening gap in complex late-stage cases highlights limitations in accounting for clinical context and socioeconomic factors. These findings support the role of LLMs as robust clinician-supervised decision-support tools while emphasizing the necessity of human judgment for individualized care.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdgth.2026.1862730","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T11:28:20Z","timestamp":1782818900000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Large language models for breast cancer treatment planning: a blinded real-world evaluation of DeepSeek, ChatGPT, and oncologist recommendations"],"prefix":"10.3389","volume":"8","author":[{"given":"Ming","family":"Li","sequence":"first","affiliation":[{"name":"Department of Health Policy Management, Bloomberg School of Public Health, Johns Hopkins University","place":["Baltimore, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiran","family":"Yu","sequence":"additional","affiliation":[{"name":"Graduate School of Inner Mongolia Medical University","place":["Hohhot, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gang","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Health Policy Management, Bloomberg School of Public Health, Johns Hopkins University","place":["Baltimore, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoli","family":"Zhang","sequence":"additional","affiliation":[{"name":"Oncology Department of Inner Mongolia People's Hospital","place":["Hohhot, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuting","family":"Shi","sequence":"additional","affiliation":[{"name":"Oncology Department of Inner Mongolia People's Hospital","place":["Hohhot, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rila","family":"Su","sequence":"additional","affiliation":[{"name":"Oncology Department of Inner Mongolia People's Hospital","place":["Hohhot, China"]},{"name":"Clinical Medical Research Center of Inner Mongolia People's Hospital","place":["Hohhot, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1154","DOI":"10.1038\/s41591-025-03502-3","article-title":"Global patterns and trends in breast cancer incidence and mortality across 185 countries","volume":"31","author":"Kim","year":"2025","journal-title":"Nat Med"},{"key":"B2","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.jncc.2024.01.006","article-title":"Cancer incidence and mortality in China, 2022","volume":"4","author":"Han","year":"2024","journal-title":"J. Natl. Cancer Cent."},{"key":"B3","doi-asserted-by":"publisher","first-page":"462","DOI":"10.3390\/fi16120462","article-title":"Generative AI in medicine and healthcare: moving beyond the \u201cpeak of inflated expectations","volume":"16","author":"Zhang","year":"2024","journal-title":"Future Internet"},{"key":"B4","author":"Lamb","year":"2024"},{"key":"B5","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1007\/s00404-024-07565-4","article-title":"Evolution of publicly available large language models for complex decision-making in breast cancer care","volume":"310","author":"Griewing","year":"2024","journal-title":"Arch Gynecol Obstet"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1831","DOI":"10.1007\/s00404-023-07130-5","article-title":"Evaluating ChatGPT as an adjunct for the multidisciplinary tumor board decision-making in primary breast cancer cases","volume":"308","author":"Lukac","year":"2023","journal-title":"Arch Gynecol Obstet"},{"key":"B7","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41523-023-00557-8","article-title":"Large language model (ChatGPT) as a support tool for breast tumor board","volume":"9","author":"Sorin","year":"2023","journal-title":"Npj Breast Cancer"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","year":"2023","journal-title":"N Engl J Med"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"3233","DOI":"10.1038\/s41591-025-03836-y","article-title":"Rapid deployment of large language model DeepSeek in Chinese hospitals demands a regulatory response","volume":"31","author":"Shen","year":"2025","journal-title":"Nat Med"},{"key":"B11","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1038\/s41698-021-00165-4","article-title":"Characterizing advanced breast cancer heterogeneity and treatment resistance through serial biopsies and comprehensive analytics","volume":"5","author":"Li","year":"2021","journal-title":"Npj Precis. Oncol."},{"key":"B12","doi-asserted-by":"publisher","author":"Singhal","year":"2022","DOI":"10.48550\/arXiv.2212.13138"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1186\/s12885-017-3783-6","article-title":"Treatment recommendations within the leeway of clinical guidelines: a qualitative interview study on oncologists' Clinical deliberation","volume":"17","author":"Otte","year":"2017","journal-title":"BMC Cancer"},{"key":"B15","doi-asserted-by":"publisher","first-page":"31450","DOI":"10.1038\/s41598-025-16999-y","article-title":"Exploring the use of large language models for classification, clinical interpretation, and treatment recommendation in breast tumor patient records","volume":"15","author":"Miao","year":"2025","journal-title":"Sci Rep"},{"key":"B16","doi-asserted-by":"publisher","first-page":"e73469","DOI":"10.2196\/73469","article-title":"Evaluating the performance of DeepSeek-R1 and DeepSeek-V3 versus OpenAI models in the Chinese national medical licensing examination: cross-sectional comparative study","volume":"11","author":"Wang","year":"2025","journal-title":"JMIR Med. Educ."},{"key":"B17","doi-asserted-by":"publisher","first-page":"e66429","DOI":"10.2196\/66429","article-title":"Large language models as a consulting hotline for patients with breast cancer and specialists in China: cross-sectional questionnaire study","volume":"13","author":"Liu","year":"2025","journal-title":"JMIR Med Inform"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1459","DOI":"10.1001\/jamaoncol.2023.2954","article-title":"Use of artificial intelligence chatbots for cancer treatment information","volume":"9","author":"Chen","year":"2023","journal-title":"JAMA Oncol"},{"key":"B19","author":"Liu","year":"2024"},{"key":"B20","doi-asserted-by":"publisher","first-page":"e2457879","DOI":"10.1001\/jamanetworkopen.2024.57879","article-title":"Large language models for chatbot health advice studies: a systematic review","volume":"8","author":"Huo","year":"2025","journal-title":"JAMA Netw. Open"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1631\/jzus.B2400285","article-title":"Real-world data and evidence: pioneering frontiers in precision oncology","author":"Jiang","year":"2025","journal-title":"J. Zhejiang Univ.-Sci. B"},{"key":"B22","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1038\/nrclinonc.2017.31","article-title":"The high price of anticancer drugs: origins, implications, barriers, solutions","volume":"14","author":"Prasad","year":"2017","journal-title":"Nat Rev Clin Oncol"},{"key":"B23","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1186\/s41235-022-00421-6","article-title":"Algorithm-based advice taking and clinical judgement: impact of advice distance and algorithm information","volume":"7","author":"P\u00e1lfi","year":"2022","journal-title":"Cogn Res Princ Implic"},{"key":"B24","doi-asserted-by":"publisher","first-page":"900543","DOI":"10.3389\/fmed.2022.900543","article-title":"Rethinking clinical decision-making to improve clinical reasoning","volume":"9","author":"Corrao","year":"","journal-title":"Front Med"},{"key":"B25","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1146\/annurev-psych-120709-145346","article-title":"Heuristic decision making","volume":"62","author":"Gigerenzer","year":"2011","journal-title":"Annu Rev Psychol"},{"key":"B26","first-page":"610","article-title":"On the dangers of stochastic parrots: can language models be too big?","author":"Bender","year":"2021"},{"key":"B27","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1126\/science.abg1834","article-title":"Beware explanations from AI in health care","volume":"373","author":"Babic","year":"2021","journal-title":"Science"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1194","DOI":"10.1093\/annonc\/mdz173","article-title":"Early breast cancer: eSMO clinical practice guidelines for diagnosis, treatment and follow-up","volume":"30","author":"Cardoso","year":"2019","journal-title":"Ann Oncol"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1541","DOI":"10.1093\/annonc\/mdz235","article-title":"Estimating the benefits of therapy for early-stage breast cancer: the st. Gallen international consensus guidelines for the primary therapy of early breast cancer 2019","volume":"30","author":"Burstein","year":"2019","journal-title":"Ann Oncol"},{"key":"B30","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1097\/PAS.0000000000000780","article-title":"PD-L1 Expression and intratumoral heterogeneity across breast cancer subtypes and stages: an assessment of 245 primary and 40 metastatic tumors","volume":"41","author":"Dill","year":"2017","journal-title":"Am J Surg Pathol"},{"key":"B31","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1002\/cncr.35101","article-title":"HER2-low Heterogeneity between primary and paired recurrent\/metastatic breast cancer: implications in treatment and prognosis","volume":"130","author":"Lin","year":"2024","journal-title":"Cancer"},{"key":"B32","doi-asserted-by":"publisher","first-page":"2293","DOI":"10.1056\/NEJMsb1609216","article-title":"Real-world evidence \u2014 what is it and what can it tell US?","volume":"375","author":"Sherman","year":"2016","journal-title":"N Engl J Med"},{"key":"B33","doi-asserted-by":"publisher","first-page":"2474","DOI":"10.1158\/2159-8290.CD-20-1557","article-title":"Impact of HER2 heterogeneity on treatment response of early-stage HER2-positive breast cancer: phase II neoadjuvant clinical trial of T-DM1 combined with pertuzumab","volume":"11","author":"Filho","year":"2021","journal-title":"Cancer Discov."},{"key":"B34","doi-asserted-by":"publisher","first-page":"100269","DOI":"10.1016\/j.eclinm.2020.100269","article-title":"Financial toxicity of cancer treatment: moving the discussion from acknowledgement of the problem to identifying solutions","volume":"20","author":"Desai","year":"2020","journal-title":"eClinicalMed"},{"key":"B35","doi-asserted-by":"publisher","first-page":"e746","DOI":"10.1634\/theoncologist.2019-0647","article-title":"Real-world evidence in oncology: opportunities and limitations","volume":"25","author":"Di Maio","year":"2020","journal-title":"Oncologist"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1569554","DOI":"10.3389\/fdgth.2025.1569554","article-title":"Summarizing clinical evidence utilizing large language models for cancer treatments: a blinded comparative analysis","volume":"7","author":"Rubinstein","year":"2025","journal-title":"Front Digit Health"},{"key":"B37","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s00432-024-06049-x","article-title":"Evidence-based cancer care: assessing guideline adherence of multidisciplinary tumor board recommendations for breast and colorectal cancer in a non-academic medical center","volume":"151","author":"Leonhardt","year":"2024","journal-title":"J Cancer Res Clin Oncol"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1862730\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T11:28:21Z","timestamp":1782818901000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1862730\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":37,"alternative-id":["10.3389\/fdgth.2026.1862730"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1862730","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]},"article-number":"1862730"}}