{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T03:51:16Z","timestamp":1781927476931,"version":"3.54.5"},"posted":{"date-parts":[[2026,6,17]]},"group-title":"Radiology and Imaging","reference-count":19,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institutes of Health \/ National Cancer Institute","award":["P01CA285249"],"award-info":[{"award-number":["P01CA285249"]}]},{"name":"National Institutes of Health \/ National Cancer Institute","award":["P30CA016672"],"award-info":[{"award-number":["P30CA016672"]}]},{"name":"National Institutes of Health \/ National Institute of Dental and Craniofacial Research","award":["R01DE034406"],"award-info":[{"award-number":["R01DE034406"]}]},{"DOI":"10.13039\/100004917","name":"Cancer Prevention and Research Institute of Texas","doi-asserted-by":"crossref","award":["RP210042"],"award-info":[{"award-number":["RP210042"]}],"id":[{"id":"10.13039\/100004917","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Charles and Daneen Stiefel Oropharyngeal Cancer Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,17]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:sec>\n                  <jats:title>Purpose<\/jats:title>\n                  <jats:p>Radiologic surveillance is essential for oropharyngeal cancer (OPC) survivors, guiding recurrence detection and follow-up strategies. The Neck Imaging Reporting and Data System provides a standardized framework for post-treatment risk reporting at both the primary tumor site (pNI-RADs) and cervical lymph nodes (nNI-RADS). Comprehensive surveillance additionally requires assessment of disease status, including the primary tumor, nodal involvement, and distant metastases. These clinical results are often embedded as unstructured data within free-text radiology reports. We hypothesized that a large language model (LLM) can reliably extract NI-RADS score criteria and summarize key imaging features from unstructured radiology text, achieving high concordance with expert review.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Previously untreated OPC patients who received definitive cancer therapy were identified. Eligible imaging reports included post-treatment head and neck CT, MRI, or FDG PET\/CT scans containing narrative and impression text. Examinations lacking narrative or impression text, containing pre-existing NI-RADS annotations, or involving non-surveillance imaging modalities were excluded. A total of 200 reports were randomly selected from 7,076 eligible examinations for manual abstraction using a three-reviewer consensus framework to establish a reference dataset. Using the Palantir Foundry Pipeline Builder, a GPT-5-based LLM was deployed to extract pNI-RADS and nNI-RADS scores, and key imaging features of disease status from these reports. Performance was evaluated using exact agreement and F1-based metrics.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Agreement for no evidence of disease (score of 1) was 93.3% (126\/135; F1 = 0.94) and 90.3% (130\/144; F1 = 0.93) for pNI-RADS and nNI-RADS, respectively. For NI-RADS \u22652, exact category agreement was 73.1% (38\/52; macro-F1 = 0.75) for pNI-RADS and 64.3% (27\/42; macro-F1 = 0.56) for nNI-RADS. Quadratic weighted \u03ba was 0.81 and 0.59, respectively. For post-treatment disease surveillance variables, agreement was 94.9% (149\/157; F1 = 0.87) for primary tumor presence, 89.1% (164\/184; F1 = 0.87) for nodal disease presence, and 94.7% (126\/133; F1 = 0.70) for distant metastasis detection. Specificity was high across disease-status variables (0.95-0.99), with negative predictive values of 0.95 for primary tumor, 0.87 for nodal disease, and 0.99 for distant metastasis.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Our LLM-based information retrieval and classification approach for radiographic treatment response from unstructured, multidimensional imaging reports achieved high performance for disease exclusion and moderate performance for detecting suspected residual and\/or new disease. This pipeline supports scalable and standardized surveillance data capture for longitudinal monitoring, clinical analytics, and survivorship research in head and neck oncology.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.06.11.26355483","type":"posted-content","created":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T01:00:24Z","timestamp":1781744424000},"source":"Crossref","is-referenced-by-count":0,"title":["LLM-Driven Extraction of NI-RADS and Imaging Tumor Characteristics to Enhance Oropharyngeal Cancer Survivorship Surveillance"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9630-3417","authenticated-orcid":false,"given":"Wenye","family":"Song","sequence":"first","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lina","family":"Shbita","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Isabelle Jia Hui","family":"Jang","sequence":"additional","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olga","family":"Starostina","sequence":"additional","affiliation":[{"name":"Department of Enterprise Data Engineering & Analytics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan","family":"Lewis","sequence":"additional","affiliation":[{"name":"Department of Enterprise Data Engineering & Analytics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ariana","family":"Sahli","sequence":"additional","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Warren","family":"Floyd","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Md","family":"Mahin","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Waree","family":"Rinsurongkawong","sequence":"additional","affiliation":[{"name":"Quantitative Research Computing, Division of Quantitative Sciences, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carly","family":"Barbon","sequence":"additional","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen","family":"Lai","sequence":"additional","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J. Jack","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Komal","family":"Shah","sequence":"additional","affiliation":[{"name":"Neuroradiology Department, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melissa","family":"Chen","sequence":"additional","affiliation":[{"name":"Cancer Network, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3710-5706","authenticated-orcid":false,"given":"Katherine A.","family":"Hutcheson","sequence":"additional","affiliation":[{"name":"Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"},{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-3994","authenticated-orcid":false,"given":"Clifton David","family":"Fuller","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6762-6807","authenticated-orcid":false,"given":"Amy","family":"Moreno","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2026061920050945000_2026.06.11.26355483v1.1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa0912217"},{"key":"2026061920050945000_2026.06.11.26355483v1.2","doi-asserted-by":"publisher","DOI":"10.1038\/s41571-022-00603-7"},{"key":"2026061920050945000_2026.06.11.26355483v1.3","doi-asserted-by":"publisher","DOI":"10.1186\/s13244-023-01578-4"},{"key":"2026061920050945000_2026.06.11.26355483v1.4","doi-asserted-by":"publisher","DOI":"10.1200\/EDBK_389718"},{"key":"2026061920050945000_2026.06.11.26355483v1.5","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacr.2018.05.006"},{"key":"2026061920050945000_2026.06.11.26355483v1.6","doi-asserted-by":"publisher","DOI":"10.3174\/ajnr.A5157"},{"key":"2026061920050945000_2026.06.11.26355483v1.7","doi-asserted-by":"publisher","DOI":"10.1177\/0194599819865235"},{"key":"2026061920050945000_2026.06.11.26355483v1.8","doi-asserted-by":"publisher","DOI":"10.3174\/ajnr.A7992"},{"key":"2026061920050945000_2026.06.11.26355483v1.9","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2025-106845"},{"key":"2026061920050945000_2026.06.11.26355483v1.10","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.16142770"},{"key":"2026061920050945000_2026.06.11.26355483v1.11","doi-asserted-by":"publisher","DOI":"10.2214\/AJR.24.32341"},{"key":"2026061920050945000_2026.06.11.26355483v1.12","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics15070930"},{"key":"2026061920050945000_2026.06.11.26355483v1.13","doi-asserted-by":"publisher","DOI":"10.1200\/CCI.23.00219"},{"key":"2026061920050945000_2026.06.11.26355483v1.14","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0214775"},{"key":"2026061920050945000_2026.06.11.26355483v1.15","doi-asserted-by":"publisher","DOI":"10.1200\/CCI.23.00219"},{"key":"2026061920050945000_2026.06.11.26355483v1.16","doi-asserted-by":"publisher","DOI":"10.3174\/AJNR.A6529"},{"key":"2026061920050945000_2026.06.11.26355483v1.17","doi-asserted-by":"publisher","DOI":"10.1016\/j.oraloncology.2018.07.006"},{"key":"2026061920050945000_2026.06.11.26355483v1.18","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoto.2018.1791"},{"key":"2026061920050945000_2026.06.11.26355483v1.19","doi-asserted-by":"publisher","DOI":"10.1002\/hed.26991"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.11.26355483","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T03:05:23Z","timestamp":1781924723000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.06.11.26355483"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":19,"URL":"https:\/\/doi.org\/10.64898\/2026.06.11.26355483","relation":{},"subject":[],"published":{"date-parts":[[2026,6,17]]},"subtype":"preprint"}}