{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,12]],"date-time":"2026-11-12T13:43:48Z","timestamp":1794491028577,"version":"4.9.1"},"reference-count":23,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T00:00:00Z","timestamp":1773100800000},"content-version":"unspecified","delay-in-days":9,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["82173156"],"award-info":[{"award-number":["82173156"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Beijing Hospitals Authority\u2019s Ascent Plan","award":["DFL20241105"],"award-info":[{"award-number":["DFL20241105"]}]},{"name":"Beijing Natural Science Foundation\u2019s Beijing-Tianjin-Hebei Basic Research Project","award":["22JCZXJC00140"],"award-info":[{"award-number":["22JCZXJC00140"]}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Health Care Inform"],"accepted":{"date-parts":[[2026,2,10]]},"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To evaluate the ability of large language models (LLMs) to simulate multidisciplinary team (MDT) decision-making in colorectal cancer, a malignancy that often requires complex treatment planning.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We retrospectively analysed 1423 colorectal cancer cases discussed at MDT meetings at Peking University Cancer Hospital between January 2023 and December 2024. Three LLMs\u2014OpenAI o3-mini-2025-01-31, DeepSeek-R1 671b and Qwen qwq-plus-2025-03-05\u2014were tested for their ability to replicate MDT recommendations using a standardised treatment categorisation framework. Each case was processed three times per model; only cases with consistent outputs across all three runs were included. Concordance between AI-generated decisions and expert MDT consensus was assessed using agreement percentages and Cohen\u2019s kappa.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>O3 demonstrated the highest intramodel stability, with an agreement rate of 81.0% (Fleiss\u2019 kappa=0.794), yielding 1153 cases with consistent outputs. Concordance with MDT consensus was comparable across the three models, ranging from 62.5% to 65.4%. Multivariable analysis of O3 outputs identified treatment-na\u00efve status, non-metastatic disease and colon tumour location as independent predictors of higher concordance with experts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>LLMs showed fair overall agreement with expert MDT decisions, with stronger performance in standardised and less complex clinical scenarios. Areas of higher concordance included treatment-na\u00efve non-metastatic colon cancer, treated non-metastatic rectal cancer and treated non-metastatic colon cancer.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs can partially replicate expert MDT recommendations in colorectal cancer. Their integration into clinical workflows should aim to complement, rather than replace, human expertise.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjhci-2025-101780","type":"journal-article","created":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T14:55:17Z","timestamp":1773154517000},"page":"e101780","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":4,"title":["Comparison of large language models and expert multidisciplinary team decisions in colorectal cancer"],"prefix":"10.1136","volume":"33","author":[{"given":"Boyang","family":"Qu","sequence":"first","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Longhao","family":"Cao","sequence":"additional","affiliation":[{"name":"Peking University College of Future Technology, Beijing, China"},{"name":"Beijing Changping Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chen","family":"Wu","sequence":"additional","affiliation":[{"name":"Peking University School of Software and Microelectronics, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongjiu","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tingting","family":"Sun","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junpeng","family":"Pei","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Huang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaotong","family":"Hou","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dawei","family":"Li","sequence":"additional","affiliation":[{"name":"Peking University College of Future Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1877-7005","authenticated-orcid":false,"given":"Aiwen","family":"Wu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Holistic Integrative Management of Gastrointestinal Cancers, Beijing Key Laboratory of Carcinogenesis and Translational Research, Unit III (Colorectal & Ostomy Service), Gastrointestinal Cancer Center, Peking University Cancer Hospital & Institute, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2026,3,10]]},"reference":[{"key":"2026031007550865000_33.1.e101780.1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsb2503956"},{"key":"2026031007550865000_33.1.e101780.2","doi-asserted-by":"crossref","DOI":"10.1038\/s41523-023-00557-8","article-title":"Large language model (ChatGPT) as a support tool for breast tumor board","volume":"9","author":"Sorin","year":"2023","journal-title":"NPJ Breast Cancer"},{"key":"2026031007550865000_33.1.e101780.3","doi-asserted-by":"crossref","unstructured":"Rydzewski NR , Dinakaran D , Zhao SG , et al . Comparative Evaluation of LLMs in Clinical Oncology. NEJM AI 2024;1. doi:10.1056\/aioa2300151","DOI":"10.1056\/AIoa2300151"},{"key":"2026031007550865000_33.1.e101780.4","doi-asserted-by":"crossref","unstructured":"Boussina A , Krishnamoorthy R , Quintero K , et al . Large Language Models for More Efficient Reporting of Hospital Quality Measures. NEJM AI 2024;1. doi:10.1056\/aics2400420","DOI":"10.1056\/AIcs2400420"},{"key":"2026031007550865000_33.1.e101780.5","doi-asserted-by":"crossref","first-page":"6509","DOI":"10.1097\/JS9.0000000000001850","article-title":"The performance of large language model-powered chatbots compared to oncology physicians on colorectal cancer queries","volume":"110","author":"Zhou","year":"2024","journal-title":"Int J Surg"},{"key":"2026031007550865000_33.1.e101780.6","doi-asserted-by":"crossref","first-page":"e35","DOI":"10.1016\/S2589-7500(24)00246-2","article-title":"The potential of Generative Pre-trained Transformer 4 (GPT-4) to analyse medical notes in three different languages: a retrospective model-evaluation study","volume":"7","author":"Menezes","year":"2025","journal-title":"Lancet Digit Health"},{"key":"2026031007550865000_33.1.e101780.7","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03139-8"},{"key":"2026031007550865000_33.1.e101780.8","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1038\/s41591-024-03456-y","article-title":"GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial","volume":"31","author":"Goh","year":"2025","journal-title":"Nat Med"},{"key":"2026031007550865000_33.1.e101780.9","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-024-52415-1","article-title":"Evaluating the use of large language models to provide clinical recommendations in the Emergency Department","volume":"15","author":"Williams","year":"2024","journal-title":"Nat Commun"},{"key":"2026031007550865000_33.1.e101780.10","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1111\/bju.15495","article-title":"The cancer multidisciplinary team meeting: in need of change? History, challenges and future perspectives","volume":"128","author":"Winters","year":"2021","journal-title":"BJU Int"},{"key":"2026031007550865000_33.1.e101780.11","doi-asserted-by":"publisher","DOI":"10.3322\/CAAC.21834"},{"key":"2026031007550865000_33.1.e101780.12","doi-asserted-by":"publisher","DOI":"10.1038\/s41575-020-0275-y"},{"key":"2026031007550865000_33.1.e101780.13","doi-asserted-by":"crossref","first-page":"1885","DOI":"10.1002\/ijc.34543","article-title":"China special issue on gastrointestinal tumors-Improved survival after multidisciplinary team decision for patients with advanced gastrointestinal cancer: A multicenter, noninterventional, controlled study","volume":"153","author":"Zhang","year":"2023","journal-title":"Int J Cancer"},{"key":"2026031007550865000_33.1.e101780.14","article-title":"Impact of Multidisciplinary Team Management on Survival and Recurrence in Stage I-III Colorectal Cancer: A Population-Based Study in Northern Italy","volume":"13","author":"Mangone","year":"2024","journal-title":"Biology (Basel)"},{"key":"2026031007550865000_33.1.e101780.15","doi-asserted-by":"crossref","first-page":"2398","DOI":"10.1016\/j.ejso.2021.05.008","article-title":"Preoperative multidisciplinary team assessment is associated with improved survival in patients with locally advanced colon cancer; a nationwide cohort study in 3157 patients","volume":"47","author":"Rosander","year":"2021","journal-title":"Eur J Surg Oncol"},{"key":"2026031007550865000_33.1.e101780.16","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1007\/s00384-015-2459-4","article-title":"Improved outcomes of colorectal cancer patients with liver metastases in the era of the multidisciplinary teams","volume":"31","author":"Lan","year":"2016","journal-title":"Int J Colorectal Dis"},{"key":"2026031007550865000_33.1.e101780.17","doi-asserted-by":"crossref","DOI":"10.3389\/fonc.2024.1353031","article-title":"Assessing the role of advanced artificial intelligence as a tool in multidisciplinary tumor board decision-making for primary head and neck cancer cases","volume":"14","author":"Schmidl","year":"2024","journal-title":"Front Oncol"},{"key":"2026031007550865000_33.1.e101780.18","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med"},{"key":"2026031007550865000_33.1.e101780.19","doi-asserted-by":"publisher","DOI":"10.1093\/annonc\/mdx224"},{"key":"2026031007550865000_33.1.e101780.20","first-page":"e361","article-title":"Comparative Oncologic Outcomes of Upper Third Rectal Cancers: A Meta-analysis","volume":"18","author":"Clancy","year":"2019","journal-title":"Clin Colorectal Cancer"},{"key":"2026031007550865000_33.1.e101780.21","doi-asserted-by":"publisher","DOI":"10.1097\/01.sla.0000141194.27992.32"},{"key":"2026031007550865000_33.1.e101780.22","doi-asserted-by":"crossref","first-page":"2537","DOI":"10.1016\/S0140-6736(18)31078-X","article-title":"Long-term outcomes of clinical complete responders after neoadjuvant treatment for rectal cancer in the International Watch & Wait Database (IWWD): an international multicentre registry study","volume":"391","author":"van der Valk","year":"2018","journal-title":"The Lancet"},{"key":"2026031007550865000_33.1.e101780.23","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1097\/SLA.0000000000005507","article-title":"Intentional Watch and Wait or Organ Preservation Surgery Following Neoadjuvant Chemoradiotherapy Plus Consolidation CAPEOX for MRI-defined Low-risk Rectal Cancer: Findings From a Prospective Phase 2 Trial (PKUCH-R01 Trial, NCT02860234)","volume":"277","author":"Wang","year":"2023","journal-title":"Ann Surg"}],"container-title":["BMJ Health &amp; Care Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjhci-2025-101780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T14:55:20Z","timestamp":1773154520000},"score":1,"resource":{"primary":{"URL":"https:\/\/informatics.bmj.com\/lookup\/doi\/10.1136\/bmjhci-2025-101780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3]]},"references-count":23,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,3,10]]},"published-print":{"date-parts":[[2026,3]]}},"alternative-id":["10.1136\/bmjhci-2025-101780"],"URL":"https:\/\/doi.org\/10.1136\/bmjhci-2025-101780","relation":{},"ISSN":["2632-1009"],"issn-type":[{"value":"2632-1009","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3]]}}}