{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T03:11:43Z","timestamp":1790392303102,"version":"4.1.0"},"reference-count":25,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"3","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>To develop a large language model (LLM) (Truveta Language Model Oncology [TLM-Oncology]) to extract real-world oncology staging data across multiple cancer types from clinical documentation with high precision.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>We selected patients from a large integrated health system with a bladder, cervical, colorectal, breast, or prostate cancer diagnosis in their structured data. We identified relevant notes using note metadata and keywords and annotated overall stage; T, N, and M; associated timeframe; and cancer diagnosis on a sample of 700 notes as ground truth. Of the 700 notes, 450 were divided equally between training, validation, and test sets for bladder, cervical, and colorectal cancers; 150 were used for targeted error-pattern training on these cancers; and the remaining 100 were split equally between breast and prostate cancer test sets. We started with a pretrained LLM and applied supervised fine-tuning to adapt the model to structured clinical information extraction. Model performance was measured using precision, recall, and F1 scores at the relation level and individual attribute level.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>We extracted over 2.5 million staging records for 217,768 patients from over two million notes. Relation-level precision across the six attributes ranged from 0.77 to 1.0 for the first three cancers and, without further training, 0.83 to 1.0 for two additional cancers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>TLM-Oncology extracted detailed cancer staging information for five cancers from a variety of clinical documentation within a single integrated health system with high precision and turned data that were previously inaccessible into a valuable resource for downstream use. We are currently evaluating TLM-Oncology on other solid tumors within three additional health systems to assess its generalizability.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00388","type":"journal-article","created":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T20:00:17Z","timestamp":1783972817000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":1,"title":["Large Language Models to Extract Cancer Staging Data From Clinical Documentation at Scale"],"prefix":"10.1200","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6475-5344","authenticated-orcid":true,"given":"Swapna","family":"Abhyankar","sequence":"first","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6854-2469","authenticated-orcid":true,"given":"Rajesh M.","family":"Rao","sequence":"additional","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mehraveh","family":"Salehi","sequence":"additional","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5197-1590","authenticated-orcid":true,"given":"Jennifer J.","family":"Liang","sequence":"additional","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jamalynne","family":"Deckard","sequence":"additional","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sujatha","family":"Sagiraju","sequence":"additional","affiliation":[{"name":"Truveta, Inc, Bellevue, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_6_2_2","volume-title":"Epic Specialties","author":"Epic Systems Corporation","year":"2025","unstructured":"Epic Systems Corporation: Epic Specialties. Verona (WI), Epic Systems Corporation, 2025"},{"key":"e_1_3_6_3_2","volume-title":"Healthcare Service Lines and Departments","author":"Oracle","year":"2025","unstructured":"Oracle: Healthcare Service Lines and Departments. Santa Clara (CA), Oracle, 2025"},{"key":"e_1_3_6_4_2","doi-asserted-by":"publisher","DOI":"10.6004\/jnccn.2020.7799"},{"key":"e_1_3_6_5_2","volume-title":"Improving Health System Cancer Staging Completion Rates in the Epic Staging Module","author":"American College of Surgeons","year":"2021","unstructured":"American College of Surgeons: Improving Health System Cancer Staging Completion Rates in the Epic Staging Module. Chicago (IL), American College of Surgeons, 2021"},{"key":"e_1_3_6_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/app13106209"},{"key":"e_1_3_6_7_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2022.40.28_suppl.309"},{"key":"e_1_3_6_8_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooae144"},{"key":"e_1_3_6_9_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-53190-9"},{"key":"e_1_3_6_10_2","unstructured":"OncLive Staff: Development of EHR-integrated cancer staging module identifies barriers increases discrete capture. OncLive 2024"},{"key":"e_1_3_6_11_2","doi-asserted-by":"publisher","unstructured":"Zahnd WE Ruiz-Alvarez V Rilling M et al: Enhancing the National Cancer Database content using natural language processing of unstructured documents. JCO Clin Cancer Inform 10.1016\/j.soi.2024.100058","DOI":"10.1016\/j.soi.2024.100058"},{"key":"e_1_3_6_12_2","volume-title":"American Joint Committee on Cancer (AJCC)","author":"National Cancer Institute SEER Training Modules","year":"2025","unstructured":"National Cancer Institute SEER Training Modules: American Joint Committee on Cancer (AJCC). Bethesda (MD), National Cancer Institute, 2025. https:\/\/www.training.seer.cancer.gov\/staging\/systems\/ajcc.html"},{"key":"e_1_3_6_13_2","doi-asserted-by":"publisher","DOI":"10.2196\/67056"},{"key":"e_1_3_6_14_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-10709-4"},{"key":"e_1_3_6_15_2","doi-asserted-by":"publisher","unstructured":"Chang CH Lucas MM Lu-Yao G et al: Classifying cancer stage with open-source clinical large language models. ArXiv. 10.48550\/arXiv:2404.01589","DOI":"10.48550\/arXiv:2404.01589"},{"key":"e_1_3_6_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-66538-7_23"},{"key":"e_1_3_6_17_2","doi-asserted-by":"publisher","DOI":"10.1158\/1557-3265.AIMACHINE-B006"},{"key":"e_1_3_6_18_2","article-title":"CORAL: Expert-curated oncology reports to advance language model inference (v1.0)","author":"Radhakrishnan L","year":"2024","unstructured":"Radhakrishnan L, Karthik N, Muenzen K, et al: CORAL: Expert-curated oncology reports to advance language model inference (v1.0). PhysioNet, 2024","journal-title":"PhysioNet"},{"key":"e_1_3_6_19_2","doi-asserted-by":"publisher","unstructured":"Li M Huang J Yeung J et al: CancerLLM: A large language model in the cancer domain. Arxiv 10.48550\/arXiv:2406.10459","DOI":"10.48550\/arXiv:2406.10459"},{"key":"e_1_3_6_20_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2025.43.5_suppl.136"},{"key":"e_1_3_6_21_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2022.793316"},{"key":"e_1_3_6_22_2","volume-title":"Poster presented at: OHDSI Oncology CDM Working Group","author":"Kim S","year":"2025","unstructured":"Kim S, Choi JE, Ko CJ, et al: Leveraging LLM for populating OMOP Oncology CDM from the EHR: Feasibility study (colorectal cancer staging). Poster presented at: OHDSI Oncology CDM Working Group, 2025"},{"key":"e_1_3_6_23_2","unstructured":"Truveta Inc: Truveta Data: Powering real-time Insights. Bellevue WA Truveta 2025. https:\/\/www.truveta.com\/truveta-data"},{"key":"e_1_3_6_24_2","unstructured":"Truveta Inc: Truveta Language Model [white paper on the Internet]. Bellevue WA Truveta. Inc 2023. https:\/\/www.truveta.com\/resources\/whitepaper\/truveta-language-model\/"},{"key":"e_1_3_6_25_2","unstructured":"Mistral AI: Mixtral-8x7B [large language model on the Internet] 2023. https:\/\/huggingface.co\/mistralai\/Mixtral-8x7B-v0.1"},{"key":"e_1_3_6_26_2","volume-title":"US Department of Health and Human Services, Centers for Disease Control and Prevention","author":"Centers for Disease Control and Prevention","year":"2025","unstructured":"Centers for Disease Control and Prevention: U.S. Cancer Statistics Female Breast Cancer Stat Bite. Atlanta (GA), US Department of Health and Human Services, Centers for Disease Control and Prevention, 2025. https:\/\/www.cdc.gov\/united-states-cancer-statistics\/publications\/breast-cancer-stat-bite.html"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00388","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T20:00:19Z","timestamp":1783972819000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00388"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,7]]}},"alternative-id":["10.1200\/CCI-25-00388"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00388","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7]]},"assertion":[{"value":"2026-01-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-28","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500388"}}