{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T02:39:36Z","timestamp":1792031976902,"version":"4.4.0"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,11,24]],"date-time":"2020-11-24T00:00:00Z","timestamp":1606176000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"UCSF Open Access Publishing Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,2,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Expert abstraction of acute toxicities is critical in oncology research but is labor-intensive and variable. We assessed the accuracy of a natural language processing (NLP) pipeline to extract symptoms from clinical notes compared to physicians.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Two independent reviewers identified present and negated National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE) v5.0 symptoms from 100 randomly selected notes for on-treatment visits during radiation therapy with adjudication by a third reviewer. A NLP pipeline based on Apache clinical Text Analysis Knowledge Extraction System was developed and used to extract CTCAE terms. Accuracy was assessed by precision, recall, and F1.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The NLP pipeline demonstrated high accuracy for common physician-abstracted symptoms, such as radiation dermatitis (F1 0.88), fatigue (0.85), and nausea (0.88). NLP had poor sensitivity for negated symptoms.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>NLP accurately detects a subset of documented present CTCAE symptoms, though is limited for negated symptoms. It may facilitate strategies to more consistently identify toxicities during cancer therapy.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooaa064","type":"journal-article","created":{"date-parts":[[2020,11,24]],"date-time":"2020-11-24T05:46:55Z","timestamp":1606196815000},"page":"513-517","source":"Crossref","is-referenced-by-count":53,"title":["Natural language processing for abstraction of cancer treatment toxicities: accuracy versus human experts"],"prefix":"10.1093","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5172-6889","authenticated-orcid":false,"given":"Julian C","family":"Hong","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, California, USA"},{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA"},{"name":"Department of Radiation Oncology, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew T","family":"Fairchild","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jarred P","family":"Tanksley","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manisha","family":"Palta","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica D","family":"Tenenbaum","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,12,5]]},"reference":[{"issue":"13","key":"2021031806101972800_ooaa064-B1","doi-asserted-by":"crossref","first-page":"1537","DOI":"10.1200\/JCO.2015.65.5860","article-title":"Accuracy of adverse event ascertainment in clinical trials for pediatric acute myeloid leukemia","volume":"34","author":"Miller","year":"2016","journal-title":"J Clin Oncol"},{"issue":"5","key":"2021031806101972800_ooaa064-B2","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1016\/j.ijrobp.2020.04.040","article-title":"Inter-rater reliability in toxicity identification: limitations of current standards","author":"Fairchild","year":"2020","journal-title":"Int J Radiat Oncol Biol Phys"},{"issue":"7","key":"2021031806101972800_ooaa064-B3","doi-asserted-by":"crossref","first-page":"e27747","DOI":"10.1002\/pbc.27747","article-title":"Unintended consequences of evolution of the common terminology criteria for adverse events","volume":"66","author":"Miller","year":"2019","journal-title":"Pediatr Blood Cancer"},{"issue":"2","key":"2021031806101972800_ooaa064-B4","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1136\/jamia.2010.007237","article-title":"Data from clinical notes: a perspective on the tension between structure and flexible documentation","volume":"18","author":"Rosenbloom","year":"2011","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2021031806101972800_ooaa064-B5","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2021031806101972800_ooaa064-B6","first-page":"17","article-title":"Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program","author":"Aronson","year":"2001","journal-title":"Proc AMIA Symp"},{"issue":"3","key":"2021031806101972800_ooaa064-B7","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1093\/jamia\/ocx132","article-title":"CLAMP - a toolkit for efficiently building customized clinical natural language processing pipelines","volume":"25","author":"Soysal","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"2021031806101972800_ooaa064-B8","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1001\/jamaoncol.2019.1800","article-title":"Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports","volume":"5","author":"Kehl","year":"2019","journal-title":"JAMA Oncol"},{"issue":"1","key":"2021031806101972800_ooaa064-B9","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1148\/radiol.2241011118","article-title":"Use of natural language processing to translate clinical information from a database of 889,921 chest radiographic reports","volume":"224","author":"Hripcsak","year":"2002","journal-title":"Radiology"},{"issue":"Pt 1","key":"2021031806101972800_ooaa064-B10","first-page":"565","article-title":"Facilitating cancer research using natural language processing of pathology reports","volume":"107","author":"Xu","year":"2004","journal-title":"Stud Health Technol Inform"},{"issue":"2","key":"2021031806101972800_ooaa064-B11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1200\/CCI.18.00037","article-title":"Predicting emergency visits and hospital admissions during radiation and chemoradiation: an internally validated pretreatment machine learning algorithm","volume":"2","author":"Hong","year":"2018","journal-title":"JCO Clin Cancer Inform"},{"issue":"31","key":"2021031806101972800_ooaa064-B12","doi-asserted-by":"crossref","first-page":"3652","DOI":"10.1200\/JCO.20.01688","article-title":"System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning\u2013Directed Clinical Evaluations During Radiation and Chemoradiation","volume":"38","author":"Hong","year":"2020","journal-title":"JCO"},{"key":"2021031806101972800_ooaa064-B13","year":"2019"},{"key":"2021031806101972800_ooaa064-B14","author":"Hong","year":"2020"},{"key":"2021031806101972800_ooaa064-B15","year":"2020"},{"issue":"3","key":"2021031806101972800_ooaa064-B16","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1197\/jamia.M1733","article-title":"Agreement, the F-measure, and reliability in information retrieval","volume":"12","author":"Hripcsak","year":"2005","journal-title":"J Am Med Inform Assoc"},{"key":"2021031806101972800_ooaa064-B17","author":"Revelle","year":"2020"},{"key":"2021031806101972800_ooaa064-B18","author":"Gamer","year":"2019"},{"issue":"2","key":"2021031806101972800_ooaa064-B19","doi-asserted-by":"crossref","first-page":"e0192360","DOI":"10.1371\/journal.pone.0192360","article-title":"Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives","volume":"13","author":"Gehrmann","year":"2018","journal-title":"PLoS One"},{"issue":"21","key":"2021031806101972800_ooaa064-B20","doi-asserted-by":"crossref","first-page":"e115\u20138","DOI":"10.1158\/0008-5472.CAN-17-0615","article-title":"DeepPhe: a natural language processing system for extracting cancer phenotypes from clinical records","volume":"77","author":"Savova","year":"2017","journal-title":"Cancer Res"},{"key":"2021031806101972800_ooaa064-B21","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1093\/jnci\/djy178","article-title":"Automated survival prediction in metastatic cancer patients using high-dimensional electronic medical record data","author":"Gensheimer","year":"2019","journal-title":"J Natl Cancer Inst"},{"issue":"11","key":"2021031806101972800_ooaa064-B22","doi-asserted-by":"crossref","first-page":"1444","DOI":"10.1093\/jamia\/ocy118","article-title":"The therapy is making me sick: how online portal communications between breast cancer patients and physicians indicate medication discontinuation","volume":"25","author":"Yin","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2021031806101972800_ooaa064-B23","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1093\/jamia\/ocy169","article-title":"Patient free text reporting of symptomatic adverse events in cancer clinical research using the National Cancer Institute\u2019s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)","volume":"26","author":"Chung","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2021031806101972800_ooaa064-B24","author":"Ogren","year":"2020"},{"key":"2021031806101972800_ooaa064-B25","first-page":"22","author":"Miller","year":"2019"},{"issue":"5","key":"2021031806101972800_ooaa064-B26","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1016\/j.jbi.2009.08.007","article-title":"What can natural language processing do for clinical decision support?","volume":"42","author":"Demner-Fushman","year":"2009","journal-title":"J Biomed Inform"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamiaopen\/article-pdf\/3\/4\/513\/36625805\/ooaa064.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamiaopen\/article-pdf\/3\/4\/513\/36625805\/ooaa064.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T06:10:58Z","timestamp":1616047858000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/3\/4\/513\/5999181"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,1]]},"references-count":26,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,12,5]]},"published-print":{"date-parts":[[2021,2,15]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooaa064","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,12,1]]},"published":{"date-parts":[[2020,12,1]]}}}