{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T02:44:01Z","timestamp":1792550641165,"version":"4.5.0"},"reference-count":17,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"2","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>Large language models (LLMs) may improve extraction of prognostic variables in prostate cancer from unstructured clinical text compared with traditional, rule-based natural language processing.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>We used iterative prompt engineering with few-shot examples to develop LLM prompts for 30 phenotypes from prostate biopsy, radical prostatectomy (RP), and transurethral resection of the prostate (TURP) pathology reports, as well as magnetic resonance imaging (MRI) pelvis, computed tomography [CT] abdomen\/pelvis, Tc-99m bone scan, and prostate-specific membrane antigen [PSMA] PET\/CT reports. Data were drawn from &gt;130 Veterans Affairs facilities (1999-2025). Inference was performed with Llama 3.3 70B or GPT-4o depending on the task. Performance was evaluated on independent test sets with metrics including overall accuracy, sensitivity, positive predictive value (PPV), negative predictive value (NPV), and macro-F1.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>Pathology extraction tasks achieved near-perfect accuracy. For prostate biopsy reports, exact extraction of total cores and involved cores was highly accurate (total cores: accuracy 98.0% [95% CI, 93.0 to 99.5]; involved cores: accuracy 95.0% [95% CI, 88.8 to 98.5]). Performance was similarly strong for RP and TURP reports. On MRI pelvis, extraction of PIRADS scores (accuracy 98.0% [95% CI, 93.0 to 99.5]), lesion locations (accuracy 100% [95% CI, 96.3 to 100]), and lesion dimensions (accuracy 100% [95% CI, 96.3 to 100]) was excellent. For PSMA PET\/CT, PPVs were 100% (95% CI, 93.5 to 100) for nodal metastases and 97.9% (95% CI, 89.9 to 99.6) for bone metastases; Tc-99m bone scan performance was comparable. Lower PPVs were observed for nodal and bone metastases on pelvic MRI (84.2%-86.4%) and CT (88.0%-90.3%), largely due to ambiguous language in radiology report texts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>LLMs can reliably extract key prostate cancer phenotypes from unstructured text across multiple pathology and radiology report types. Ambiguous or indeterminate language remains the principal challenge for optimal performance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00314","type":"journal-article","created":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T20:00:15Z","timestamp":1781294415000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":1,"title":["Phenotyping Prostate Cancer in a National Health System Using Large Language Models"],"prefix":"10.1200","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0834-7669","authenticated-orcid":true,"given":"Michael P.","family":"Dykstra","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Department of Radiation Oncology, University of Michigan, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8694-3186","authenticated-orcid":true,"given":"Phoebe A.","family":"Tsao","sequence":"additional","affiliation":[{"name":"Division of Medical Oncology, Department of Internal Medicine, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Division of Hematology\/Oncology, Department of Internal Medicine, University of Michigan, Ann Arbor, MI"},{"name":"Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1597-5428","authenticated-orcid":true,"given":"Megan E.V.","family":"Caram","sequence":"additional","affiliation":[{"name":"Division of Medical Oncology, Department of Internal Medicine, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Division of Hematology\/Oncology, Department of Internal Medicine, University of Michigan, Ann Arbor, MI"},{"name":"Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-3930-0708","authenticated-orcid":true,"given":"Josiah","family":"Nieto","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, University of Michigan, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Schipper","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, University of Michigan, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4917-2984","authenticated-orcid":true,"given":"Kristian D.","family":"Stensland","sequence":"additional","affiliation":[{"name":"Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI"},{"name":"Section of Urology, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Department of Urology, University of Michigan, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Elliott","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Department of Radiation Oncology, University of Michigan, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brent S.","family":"Rose","sequence":"additional","affiliation":[{"name":"Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0194-8381","authenticated-orcid":true,"given":"Alex K.","family":"Bryant","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI"},{"name":"Department of Radiation Oncology, University of Michigan, Ann Arbor, MI"},{"name":"Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_9_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103850"},{"key":"e_1_3_9_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-024-10812-6"},{"key":"e_1_3_9_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.acra.2022.06.024"},{"key":"e_1_3_9_5_2","article-title":"Identification of patients with metastatic prostate cancer with natural language processing and machine learning","author":"Yang R","unstructured":"Yang R, Zhu D, Howard LE, et al: Identification of patients with metastatic prostate cancer with natural language processing and machine learning. JCO Clin Cancer Inform 10.1200\/CCI.21.00071","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_9_6_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01697-2"},{"key":"e_1_3_9_7_2","article-title":"Ascertainment of veterans with metastatic prostate cancer in electronic health records: Demonstrating the case for natural Language processing","author":"Alba PR","unstructured":"Alba PR, Gao A, Lee KM, et al: Ascertainment of veterans with metastatic prostate cancer in electronic health records: Demonstrating the case for natural Language processing. JCO Clin Cancer Inform 10.1200\/CCI.21.00030","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_9_8_2","doi-asserted-by":"publisher","DOI":"10.2196\/27970"},{"key":"e_1_3_9_9_2","article-title":"Automating the capture of structured pathology data for prostate cancer clinical care and research","author":"Odisho AY","unstructured":"Odisho AY, Bridge M, Webb M, et al: Automating the capture of structured pathology data for prostate cancer clinical care and research. JCO Clin Cancer Inform 10.1200\/CCI.18.00084","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_9_10_2","article-title":"Automating the determination of prostate cancer risk strata from electronic medical records","author":"Gregg JR","unstructured":"Gregg JR, Lang M, Wang LL, et al: Automating the determination of prostate cancer risk strata from electronic medical records. JCO Clin Cancer Inform 10.1200\/CCI.16.00045","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_9_11_2","doi-asserted-by":"publisher","DOI":"10.1089\/end.2014.0221"},{"key":"e_1_3_9_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00345-013-1040-4"},{"key":"e_1_3_9_13_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2022.4319"},{"key":"e_1_3_9_14_2","doi-asserted-by":"publisher","DOI":"10.2967\/jnumed.124.268555"},{"key":"e_1_3_9_15_2","unstructured":"Reference deleted"},{"key":"e_1_3_9_16_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjhci-2024-101139"},{"key":"e_1_3_9_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-28767-z"},{"key":"e_1_3_9_18_2","article-title":"Instruction-following evaluation for large language models","author":"Zhou J","unstructured":"Zhou J, Lu T, Mishra S, et al: Instruction-following evaluation for large language models. arXiv [csCL]. 10.48550\/arXiv.2311.07911","journal-title":"arXiv [csCL]"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00314","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T20:00:18Z","timestamp":1781294418000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00314"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4]]},"references-count":17,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["10.1200\/CCI-25-00314"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00314","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4]]},"assertion":[{"value":"2025-10-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-16","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500314"}}