{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T22:33:54Z","timestamp":1792794834609,"version":"4.5.0"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T00:00:00Z","timestamp":1758153600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Clinical Research Training Scholarship from the American Academy of Neurology","id":[{"id":"https:\/\/ror.org\/00mv9dj85","id-type":"ROR","asserted-by":"crossref"}]},{"name":"The Gray Matters Brain Cancer Foundation, Pediatric Brain Tumor Foundation","id":[{"id":"https:\/\/ror.org\/052ckfz09","id-type":"ROR","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000054","name":"NCI","doi-asserted-by":"publisher","award":["R01s CA188228"],"award-info":[{"award-number":["R01s CA188228"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"NCI","doi-asserted-by":"publisher","award":["CA262462"],"award-info":[{"award-number":["CA262462"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Advances in large language models (LLMs) provide a means for scalable tracking of patient symptoms in clinical trials and post-marking surveillance using the electronic health record (EHR). Therefore, we sought to validate symptoms extracted from the EHR using a LLM to scale symptom extraction from the EHR.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Across a dataset of 499 randomly chosen clinical notes from patients seen in a neuro-oncology clinic, GPT-4o annotated symptoms (headache, fatigue, nausea, anxiety, difficulties sleeping, numbness and tingling, rash, constipation, and diarrhea) with an average sensitivity and specificity of 0.97 relative to expert manual review. We then applied the LLM to an external dataset of 51,541 notes representing 1,642 patients to obtain real-world symptom prevalence for temozolomide, bevacizumab, lomustine, immune checkpoint inhibitors (ICI), and methotrexate.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In the external dataset, the average number of symptoms per note was 3.92, and the most common symptom was fatigue (83% of patients). Surprisingly, patients receiving ICIs suffered from the most symptoms (mean\u2009=\u20094.68) and those receiving methotrexate had the least (mean\u2009=\u20092.92). We also found that the prevalence of reported symptoms in this real-world cohort was often much greater than the prevalence of reported symptoms in clinical trials of similar treatment regimens.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Large language models offer the ability to scale symptom extraction from health records, which is crucial to understand symptom burden and power symptom-related interventions and studies in real-world patient cohorts.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/neuonc\/noaf223","type":"journal-article","created":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T12:14:22Z","timestamp":1758111262000},"page":"206-217","source":"Crossref","is-referenced-by-count":5,"title":["Scalable tracking of symptoms in the electronic health record using large language models in patients with central nervous system cancers undergoing therapy"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2081-5667","authenticated-orcid":false,"given":"John Y","family":"Rhee","sequence":"first","affiliation":[{"name":"Center for Neuro-Oncology, Department of Medical Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., R.B.)"]},{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]},{"name":"Broad Institute of Harvard and MIT , Cambridge, Massachusetts,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6181-1833","authenticated-orcid":false,"given":"Zachary","family":"Tentor","sequence":"additional","affiliation":[{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]},{"name":"Broad Institute of Harvard and MIT , Cambridge, Massachusetts,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Sounack","sequence":"additional","affiliation":[{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6036-1420","authenticated-orcid":false,"given":"Brigitte","family":"Durieux","sequence":"additional","affiliation":[{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8572-7183","authenticated-orcid":false,"given":"Paul J","family":"Miller","sequence":"additional","affiliation":[{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rameen","family":"Beroukhim","sequence":"additional","affiliation":[{"name":"Center for Neuro-Oncology, Department of Medical Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., R.B.)"]},{"name":"Broad Institute of Harvard and MIT , Cambridge, Massachusetts,","place":["United States"]},{"name":"Department of Medicine, Harvard Medical School , Boston, Massachusetts,","place":["United States"]},{"name":"Department of Cancer Biology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charlotta","family":"Lindvall","sequence":"additional","affiliation":[{"name":"Division of Adult Palliative Care, Department of Supportive Oncology, Dana-Farber Cancer Institute, Harvard Medical School , Boston, Massachusetts,","place":["United States (J.Y.R., Z.T., T.S., B.D., P.J.M., C.L.)"]},{"name":"Department of Medicine, Harvard Medical School , Boston, Massachusetts,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,9,18]]},"reference":[{"key":"2026030516315302400_noaf223-B1","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1007\/s11060-018-03015-9","article-title":"Prevalence of symptoms in glioma patients throughout the disease trajectory: a systematic review","volume":"140","author":"IJzerman-Korevaar","year":"2018","journal-title":"J Neurooncol"},{"key":"2026030516315302400_noaf223-B2","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1007\/s11910-023-01301-2","article-title":"Palliative care in neuro-oncology: an update","volume":"23","author":"Rhee","year":"2023","journal-title":"Curr Neurol Neurosci Rep"},{"key":"2026030516315302400_noaf223-B3","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/j.jpain.2021.07.001","article-title":"Worst pain severity profiles of oncology patients are associated with significant stress and multiple co-occurring symptoms","volume":"23","author":"Shin","year":"2022","journal-title":"J Pain"},{"key":"2026030516315302400_noaf223-B4","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.1007\/s00520-012-1688-0","article-title":"A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment","volume":"21","author":"Reilly","year":"2013","journal-title":"Support Care Cancer"},{"key":"2026030516315302400_noaf223-B5","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1200\/JCO.19.00470","article-title":"Palliative care and the management of common distressing symptoms in advanced cancer: pain, breathlessness, nausea and vomiting, and fatigue","volume":"38","author":"Henson","year":"2020","journal-title":"J Clin Oncol"},{"key":"2026030516315302400_noaf223-B6"},{"key":"2026030516315302400_noaf223-B7","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1186\/1477-7525-8-104","article-title":"Health care providers underestimate symptom intensities of cancer patients: a multicenter European study","volume":"8","author":"Laugsand","year":"2010","journal-title":"Health Qual Life Outcomes"},{"key":"2026030516315302400_noaf223-B8","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1016\/S1470-2045(06)70910-X","article-title":"Patient versus clinician symptom reporting using the National Cancer Institute common terminology criteria for adverse events: results of a questionnaire-based study","volume":"7","author":"Basch","year":"2006","journal-title":"Lancet Oncol"},{"key":"2026030516315302400_noaf223-B9","doi-asserted-by":"publisher","first-page":"2413","DOI":"10.1001\/jama.2022.9265","article-title":"Effect of electronic symptom monitoring on patient-reported outcomes among patients with metastatic cancer: a randomized clinical trial","volume":"327","author":"Basch","year":"2022","journal-title":"JAMA"},{"key":"2026030516315302400_noaf223-B10","doi-asserted-by":"publisher","first-page":"e2433153","DOI":"10.1001\/jamanetworkopen.2024.33153","article-title":"Essential components of an electronic patient-reported symptom monitoring and management system: a randomized clinical trial","volume":"7","author":"Mooney","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2026030516315302400_noaf223-B11","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1097\/CCO.0000000000001078","article-title":"Patient-reported outcomes in neuro-oncology","volume":"36","author":"Scheepens","year":"2024","journal-title":"Curr Opin Oncol"},{"key":"2026030516315302400_noaf223-B12","doi-asserted-by":"publisher","first-page":"1120","DOI":"10.1038\/s41591-022-01781-8","article-title":"Patient reported outcome assessment must be inclusive and equitable","volume":"28","author":"Calvert","year":"2022","journal-title":"Nat Med"},{"key":"2026030516315302400_noaf223-B13","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1186\/s12874-021-01419-2","article-title":"Response rate of patient reported outcomes: the delivery method matters","volume":"21","author":"Neve","year":"2021","journal-title":"BMC Med Res Methodol"},{"key":"2026030516315302400_noaf223-B14","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1007\/s00520-024-08547-7","article-title":"Longitudinal assessment of real-world patient adherence: a 12-month electronic patient-reported outcomes follow-up of women with early breast cancer undergoing treatment","volume":"32","author":"Gebert","year":"2024","journal-title":"Support Care Cancer"},{"key":"2026030516315302400_noaf223-B15","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1186\/s12955-023-02155-5","article-title":"Response rates in clinical quality registries and databases that collect patient reported outcome measures: a scoping review","volume":"21","author":"Ruseckaite","year":"2023","journal-title":"Health Qual Life Outcomes"},{"key":"2026030516315302400_noaf223-B16","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1038\/s41746-021-00464-x","article-title":"Considering the possibilities and pitfalls of generative pre-trained transformer 3 (GPT-3) in healthcare delivery","volume":"4","author":"Korngiebel","year":"2021","journal-title":"NPJ Digit Med"},{"key":"2026030516315302400_noaf223-B17","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","year":"2023","journal-title":"Commun Med (Lond)"},{"key":"2026030516315302400_noaf223-B18","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","year":"2024","journal-title":"Nat Med"},{"key":"2026030516315302400_noaf223-B19","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1038\/s41746-024-01180-y","article-title":"The long but necessary road to responsible use of large language models in healthcare research","volume":"7","author":"Kwong","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2026030516315302400_noaf223-B20"},{"key":"2026030516315302400_noaf223-B21","author":"Institute","year":"2023"},{"key":"2026030516315302400_noaf223-B22"},{"key":"2026030516315302400_noaf223-B23","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1093\/neuonc\/noac216","article-title":"Palliative care and end-of-life care in adults with malignant brain tumors","volume":"25","author":"Koekkoek","year":"2023","journal-title":"Neuro Oncol"},{"key":"2026030516315302400_noaf223-B24","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1056\/NEJMoa043330","article-title":"Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma","volume":"352","author":"Stupp","year":"2005","journal-title":"N Engl J Med"},{"key":"2026030516315302400_noaf223-B25","doi-asserted-by":"publisher","author":"Davis","DOI":"10.48550\/arXiv.2501.14105"},{"key":"2026030516315302400_noaf223-B26"},{"key":"2026030516315302400_noaf223-B27","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1093\/neuonc\/noab106","article-title":"The 2021 WHO classification of tumors of the central nervous system: a summary","volume":"23","author":"Louis","year":"2021","journal-title":"Neuro Oncol"},{"key":"2026030516315302400_noaf223-B28","doi-asserted-by":"publisher","first-page":"678","DOI":"10.1016\/S0140-6736(18)31791-4","article-title":"Lomustine-temozolomide combination therapy versus standard temozolomide therapy in patients with newly diagnosed glioblastoma with methylated MGMT promoter (CeTeG\/NOA-09): a randomised, open-label, phase 3 trial","volume":"393","author":"Herrlinger","year":"2019","journal-title":"Lancet"},{"key":"2026030516315302400_noaf223-B29","doi-asserted-by":"publisher","first-page":"1003","DOI":"10.1001\/jamaoncol.2020.1024","article-title":"Effect of nivolumab vs bevacizumab in patients with recurrent glioblastoma: the CheckMate 143 phase 3 randomized clinical trial","volume":"6","author":"Reardon","year":"2020","journal-title":"JAMA Oncol"},{"key":"2026030516315302400_noaf223-B30","doi-asserted-by":"publisher","first-page":"e1911390","DOI":"10.1001\/jamanetworkopen.2019.11390","article-title":"Concordance between electronic clinical documentation and physicians\u2019 observed behavior","volume":"2","author":"Berdahl","year":"2019","journal-title":"JAMA Netw Open"},{"key":"2026030516315302400_noaf223-B31","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1093\/jamia\/ocaa027","article-title":"How accurate is the medical record? A comparison of the physician\u2019s note with a concealed audio recording in unannounced standardized patient encounters","volume":"27","author":"Weiner","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2026030516315302400_noaf223-B32","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1111\/acem.14883","article-title":"Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models","volume":"31","author":"Iscoe","year":"2024","journal-title":"Acad Emerg Med"},{"key":"2026030516315302400_noaf223-B33","doi-asserted-by":"publisher","first-page":"1737","DOI":"10.1109\/JBHI.2021.3123192","article-title":"A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media","volume":"26","author":"Luo","year":"2022","journal-title":"IEEE J Biomed Health Inform"},{"key":"2026030516315302400_noaf223-B34","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1192\/bjp.2024.134","article-title":"Detection of suicidality from medical text using privacy-preserving large language models","volume":"225","author":"Wiest","year":"2024","journal-title":"Br J Psychiatry"},{"key":"2026030516315302400_noaf223-B35","year":"2024"},{"key":"2026030516315302400_noaf223-B36","doi-asserted-by":"publisher","first-page":"998","DOI":"10.1111\/jgs.15864","article-title":"Association between symptom burden and physical function in older patients with cancer","volume":"67","author":"Pandya","year":"2019","journal-title":"J Am Geriatr Soc"},{"key":"2026030516315302400_noaf223-B37","doi-asserted-by":"publisher","first-page":"950","DOI":"10.1186\/s12885-022-10026-3","article-title":"Tolerance to oral anticancer agent treatment in older adults with cancer: a secondary analysis of data from electronic health records and a pilot study of patient-reported outcomes","volume":"22","author":"Jiang","year":"2022","journal-title":"BMC Cancer"},{"key":"2026030516315302400_noaf223-B38","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1186\/1471-2407-13-6","article-title":"Differences in the symptom experience of older versus younger oncology outpatients: a cross-sectional study","volume":"13","author":"Cataldo","year":"2013","journal-title":"BMC Cancer"},{"key":"2026030516315302400_noaf223-B39","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1186\/s12885-022-09480-w","article-title":"Age as a potential predictor of acute side effects during chemoradiotherapy in primary cervical cancer patients","volume":"22","author":"Holmqvist","year":"2022","journal-title":"BMC Cancer"},{"key":"2026030516315302400_noaf223-B40","doi-asserted-by":"publisher","first-page":"2587","DOI":"10.1200\/JCO.2013.55.0418","article-title":"Designing therapeutic clinical trials for older and frail adults with cancer: U13 conference recommendations","volume":"32","author":"Hurria","year":"2014","journal-title":"J Clin Oncol"},{"key":"2026030516315302400_noaf223-B41","doi-asserted-by":"publisher","first-page":"7656","DOI":"10.1038\/s41598-024-56324-7","article-title":"Information heterogeneity between progress notes by physicians and nurses for inpatients with digestive system diseases","volume":"14","author":"Mashima","year":"2024","journal-title":"Sci Rep"},{"key":"2026030516315302400_noaf223-B42","doi-asserted-by":"publisher","first-page":"e6450","DOI":"10.1097\/GOX.0000000000006450","article-title":"Artificial intelligence scribe and large language model technology in healthcare documentation: advantages, limitations, and recommendations","volume":"13","author":"Mess","year":"2025","journal-title":"Plast Reconstr Surg Glob Open"},{"key":"2026030516315302400_noaf223-B43","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.3390\/healthcare13121447","article-title":"The impact of AI scribes on streamlining clinical documentation: a systematic review","volume":"13","author":"Sasseville","year":"2025","journal-title":"Healthcare (Basel)"}],"container-title":["Neuro-Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/advance-article-pdf\/doi\/10.1093\/neuonc\/noaf223\/64314052\/noaf223.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/28\/1\/206\/64314052\/noaf223.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/28\/1\/206\/64314052\/noaf223.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T21:31:57Z","timestamp":1772746317000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article\/28\/1\/206\/8257591"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,18]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,9,18]]},"published-print":{"date-parts":[[2026,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/neuonc\/noaf223","relation":{},"ISSN":["1522-8517","1523-5866"],"issn-type":[{"value":"1522-8517","type":"print"},{"value":"1523-5866","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,1]]},"published":{"date-parts":[[2025,9,18]]}}}