{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T22:00:36Z","timestamp":1775167236665,"version":"3.50.1"},"reference-count":32,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"1","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>Eastern Cooperative Oncology Group (ECOG) performance status is critical for cancer patient management, yet it is often documented only in unstructured clinical notes. This study compares several approaches to extract ECOG status from oncology notes, focusing on advanced prompting techniques for large language models (LLMs).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>\n                      We evaluated four ECOG extraction approaches on unstructured clinical notes from patients with non\u2013small cell lung cancer, multiple myeloma, or ovarian cancer (2017-2021). The approaches were a rule-based natural language processing algorithm, simple LLM prompting, and two advanced prompts (chain-of-thought and Double Filtering) using a domain-tuned LLM (LLAMAv3.2). Performance was measured on a binary outcome (any ECOG documented\n                      <jats:italic toggle=\"yes\">v<\/jats:italic>\n                      none) and a three-class outcome (ECOG 0-1\n                      <jats:italic toggle=\"yes\">v<\/jats:italic>\n                      \u22652\n                      <jats:italic toggle=\"yes\">v<\/jats:italic>\n                      none) and via an adapted QUEST questionnaire for human evaluation.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>\n                      Both CoT and double filtering technique (DFT) achieved 94% accuracy, outperforming the rule-based method (91%) and simple prompting (86%). DFT had the highest specificity (0.91) and positive predictive value (PPV; 0.93), whereas CoT attained the highest sensitivity (0.98). In the QUEST evaluation, DFT and CoT scored higher on output quality, reasoning, bias reduction, and user satisfaction than the simple prompt. DFT received the top satisfaction rating. In the three-class analysis, DFT and CoT again performed best (accuracy 0.91\n                      <jats:italic toggle=\"yes\">v<\/jats:italic>\n                      0.87) and DFT was most sensitive for ECOG \u22652 cases. Estimates for ECOG \u22652 remained imprecise because of the small sample (n = 20). All methods sometimes hallucinated ECOG status.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>Advanced LLM prompting improved ECOG extraction over basic methods. DFT and CoT each showed specific strengths (DFT had higher PPV and user satisfaction; CoT achieved higher sensitivity). These approaches appear to be generalizable across cancer types. Key implementation considerations include computational cost and human oversight. Overall, advanced prompting can standardize ECOG documentation, accelerate patient cohort identification, and inform personalized treatment planning.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00226","type":"journal-article","created":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T21:00:11Z","timestamp":1770930011000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":0,"title":["Prompt Engineering for Eastern Cooperative Oncology Group Status Extraction: Comparing Large Language Model Techniques"],"prefix":"10.1200","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3162-3225","authenticated-orcid":true,"given":"Meenakshi","family":"Dubey","sequence":"first","affiliation":[{"name":"Saw Swee Hock School of Public Health, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8593-7780","authenticated-orcid":true,"given":"Kok Joon","family":"Chong","sequence":"additional","affiliation":[{"name":"Saw Swee Hock School of Public Health, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2942-5953","authenticated-orcid":true,"given":"Yuba Raj","family":"Pun","sequence":"additional","affiliation":[{"name":"Saw Swee Hock School of Public Health, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lee Yi","family":"Foo","sequence":"additional","affiliation":[{"name":"Saw Swee Hock School of Public Health, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7392-0655","authenticated-orcid":true,"given":"Melissa","family":"Ooi","sequence":"additional","affiliation":[{"name":"National University Cancer Institute, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4951-0354","authenticated-orcid":true,"given":"Iain Bee Huat","family":"Tan","sequence":"additional","affiliation":[{"name":"National Cancer Centre Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9087-5262","authenticated-orcid":true,"given":"David Shao Peng","family":"Tan","sequence":"additional","affiliation":[{"name":"Yong Loo Lin School of Medicine, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5676-2520","authenticated-orcid":true,"given":"Kee Yuan","family":"Ngiam","sequence":"additional","affiliation":[{"name":"Department of Surgery, National University Hospital, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7150-1801","authenticated-orcid":true,"given":"Hwee Lin","family":"Wee","sequence":"additional","affiliation":[{"name":"Saw Swee Hock School of Public Health, National University of Singapore, Singapore City, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"233","reference":[{"key":"e_1_3_6_2_2","doi-asserted-by":"publisher","DOI":"10.1097\/00000421-198212000-00014"},{"key":"e_1_3_6_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-43742-2_24"},{"key":"e_1_3_6_4_2","doi-asserted-by":"publisher","DOI":"10.2165\/00019053-200826090-00009"},{"key":"e_1_3_6_5_2","doi-asserted-by":"publisher","DOI":"10.1097\/JOM.0000000000000130"},{"key":"e_1_3_6_6_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.20.00721"},{"key":"e_1_3_6_7_2","first-page":"e237111","article-title":"Revisiting the association of ECOG performance status with clinical outcomes in diverse patients with cancer","volume":"22","author":"Kumar D","year":"2024","unstructured":"Kumar D, Neeman E, Zhu S, et al: Revisiting the association of ECOG performance status with clinical outcomes in diverse patients with cancer. J Natl Compr Cancer Netw 22:e237111, 2024","journal-title":"J Natl Compr Cancer Netw"},{"key":"e_1_3_6_8_2","first-page":"6209","article-title":"A natural language processing algorithm to improve completeness of ECOG performance status in real-world data","volume":"13","author":"Cohen AB","year":"2023","unstructured":"Cohen AB, Rosic A, Harrison K, et al: A natural language processing algorithm to improve completeness of ECOG performance status in real-world data. Appl Sci (Basel, Switzerland) 13:6209, 2023","journal-title":"Appl Sci (Basel, Switzerland)"},{"key":"e_1_3_6_9_2","article-title":"Raising the bar for real-world data in oncology: Approaches to quality across multiple dimensions","author":"Castellanos EH","unstructured":"Castellanos EH, Wittmershaus BK, Chandwani S: Raising the bar for real-world data in oncology: Approaches to quality across multiple dimensions. JCO Clin Cancer Inform 10.1200\/CCI.23.00046","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_6_10_2","first-page":"107327482412795","article-title":"A multi-institutional natural language processing pipeline to extract performance status from electronic health records","volume":"31","author":"Maghsoudi A","year":"2024","unstructured":"Maghsoudi A, Sada YH, Nowakowski S, et al: A multi-institutional natural language processing pipeline to extract performance status from electronic health records. Cancer Control J Moffitt Cancer Cent 31:10732748241279518, 2024","journal-title":"Cancer Control J Moffitt Cancer Cent"},{"key":"e_1_3_6_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaccao.2022.07.008"},{"key":"e_1_3_6_12_2","first-page":"100254","article-title":"Obesity-specific association of statin use and reduced risk of recurrence of early stage NSCLC","volume":"2","author":"Patnaik SK","year":"2021","unstructured":"Patnaik SK, Petrucci C, Barbi J, et al: Obesity-specific association of statin use and reduced risk of recurrence of early stage NSCLC. JTO Clin Res Rep 2:100254, 2021","journal-title":"JTO Clin Res Rep"},{"key":"e_1_3_6_13_2","doi-asserted-by":"publisher","DOI":"10.3332\/ecancer.2019.913"},{"key":"e_1_3_6_14_2","article-title":"Assessment of electronic health record for cancer research and patient care through a scoping review of cancer natural language processing","author":"Wang L","unstructured":"Wang L, Fu S, Wen A, et al: Assessment of electronic health record for cancer research and patient care through a scoping review of cancer natural language processing. JCO Clin Cancer Inform 10.1200\/CCI.22.00006","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_6_15_2","article-title":"Efficient transformers: A survey","author":"Tay Y","year":"2020","unstructured":"Tay Y, Dehghani M, Bahri D, et al: Efficient transformers: A survey. ArXiv 2020. http:\/\/arxiv.org\/abs\/2009.06732","journal-title":"ArXiv"},{"key":"e_1_3_6_16_2","article-title":"Extraction and imputation of Eastern Cooperative Oncology Group performance status from unstructured oncology notes using language models","author":"Xu W","unstructured":"Xu W, Gu B, Lotter WE, et al: Extraction and imputation of Eastern Cooperative Oncology Group performance status from unstructured oncology notes using language models. JCO Clin Cancer Inform 10.1200\/CCI.23.00269","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_6_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0316-z"},{"key":"e_1_3_6_18_2","first-page":"6000","article-title":"Attention is all you need","volume":"30","author":"Vaswani A","unstructured":"Vaswani A. Attention is all you need. Adv Neural Inf Process Syst 30:6000-6010, 2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"e_1_3_6_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00742-2"},{"key":"e_1_3_6_20_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2"},{"key":"e_1_3_6_21_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-05881-4"},{"key":"e_1_3_6_22_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.891"},{"key":"e_1_3_6_23_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000800"},{"key":"e_1_3_6_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.findings-acl.740"},{"key":"e_1_3_6_25_2","doi-asserted-by":"publisher","DOI":"10.47102\/annals-acadmedsg.202414"},{"key":"e_1_3_6_26_2","first-page":"2024","article-title":"Dohopolski: Large language model augmented clinical trial screening","author":"Beattie J","year":"2024","unstructured":"Beattie J, Owens D, Navar AM, et al: Dohopolski: Large language model augmented clinical trial screening. medRxiv 2024-2028, 2024","journal-title":"medRxiv"},{"key":"e_1_3_6_27_2","article-title":"Universal abstraction: Harnessing frontier models to structure real-world data at scale","author":"Wong C","year":"2025","unstructured":"Wong C, Preston S, Liu Q, et al: Universal abstraction: Harnessing frontier models to structure real-world data at scale. Arxiv 2025. http:\/\/arxiv.org\/abs\/2502.00943","journal-title":"Arxiv"},{"key":"e_1_3_6_28_2","volume-title":"AI Transforms Frontline Operations","author":"National University of Singapore","year":"2018","unstructured":"National University of Singapore: AI Transforms Frontline Operations; National University of Singapore, 2018. https:\/\/news.nus.edu.sg\/ai-transforms-frontline-operations\/"},{"key":"e_1_3_6_29_2","volume-title":"Cancer Statistics","author":"NCCS","unstructured":"NCCS: Cancer Statistics. n.d. https:\/\/www.nccs.com.sg\/patient-care\/cancer-types\/cancer-statistics"},{"key":"e_1_3_6_30_2","volume-title":"AI Healthcare in NUHS Receives Boost from Supercomputer","author":"NUHS","unstructured":"NUHS: AI Healthcare in NUHS Receives Boost from Supercomputer. Default. n.d. https:\/\/nuhsplus.edu.sg\/article\/ai-healthcare-in-nuhs-receives-boost-from-supercomputer"},{"key":"e_1_3_6_31_2","article-title":"Chain-of-thought prompting elicits reasoning in large language models","author":"Wei J","year":"2022","unstructured":"Wei J, Wang X, Schuurmans D, et al: Chain-of-thought prompting elicits reasoning in large language models. arXiv 2022. http:\/\/arxiv.org\/abs\/2201.11903","journal-title":"arXiv"},{"key":"e_1_3_6_32_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7"},{"key":"e_1_3_6_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-46000-9"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00226","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T21:31:54Z","timestamp":1775165514000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00226"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["10.1200\/CCI-25-00226"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00226","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1]]},"assertion":[{"value":"2025-07-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-30","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500226"}}