{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:59:24Z","timestamp":1781369964601,"version":"3.54.1"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"e1","funder":[{"DOI":"10.13039\/100007217","name":"VA HSR&D","doi-asserted-by":"crossref","award":["IBE 09-069, HIR 08-374"],"award-info":[{"award-number":["IBE 09-069, HIR 08-374"]}],"id":[{"id":"10.13039\/100007217","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Consortium for Healthcare Informatics Research","award":["HIR 09-007"],"award-info":[{"award-number":["HIR 09-007"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,4,1]]},"abstract":"<jats:p>Objective: This paper describes a new congestive heart failure (CHF) treatment performance measure information extraction system \u2013 CHIEF \u2013 developed as part of the Automated Data Acquisition for Heart Failure project, a Veterans Health Administration project aiming at improving the detection of patients not receiving recommended care for CHF.<\/jats:p>\n               <jats:p>Design: CHIEF is based on the Apache Unstructured Information Management Architecture framework, and uses a combination of rules, dictionaries, and machine learning methods to extract left ventricular function mentions and values, CHF medications, and documented reasons for a patient not receiving these medications.<\/jats:p>\n               <jats:p>Measurements: The training and evaluation of CHIEF were based on subsets of a reference standard of various clinical notes from 1083 Veterans Health Administration patients. Domain experts manually annotated these notes to create our reference standard. Metrics used included recall, precision, and the F1-measure.<\/jats:p>\n               <jats:p>Results: In general, CHIEF extracted CHF medications with high recall (&amp;gt;0.990) and good precision (0.960\u20130.978). Mentions of Left Ventricular Ejection Fraction were also extracted with high recall (0.978\u20130.986) and precision (0.986\u20130.994), and quantitative values of Left Ventricular Ejection Fraction were found with 0.910\u20130.945 recall and with high precision (0.939\u20130.976). Reasons for not prescribing CHF medications were more difficult to extract, only reaching fair accuracy with about 0.310\u20130.400 recall and 0.250\u20130.320 precision.<\/jats:p>\n               <jats:p>Conclusion: This study demonstrated that applying natural language processing to unlock the rich and detailed clinical information found in clinical narrative text notes makes fast and scalable quality improvement approaches possible, eventually improving management and outpatient treatment of patients suffering from CHF.<\/jats:p>","DOI":"10.1093\/jamia\/ocw097","type":"journal-article","created":{"date-parts":[[2016,7,14]],"date-time":"2016-07-14T04:37:07Z","timestamp":1468471027000},"page":"e40-e46","source":"Crossref","is-referenced-by-count":34,"title":["Congestive heart failure information extraction framework for automated treatment performance measures assessment"],"prefix":"10.1093","volume":"24","author":[{"given":"St\u00e9phane M","family":"Meystre","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA"},{"name":"Salt Lake City VA Healthcare System, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youngjun","family":"Kim","sequence":"additional","affiliation":[{"name":"Salt Lake City VA Healthcare System, Salt Lake City, UT, USA"},{"name":"School of Computing, University of Utah, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Glenn T","family":"Gobbel","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael E","family":"Matheny","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Redd","sequence":"additional","affiliation":[{"name":"Salt Lake City VA Healthcare System, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bruce E","family":"Bray","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer H","family":"Garvin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA"},{"name":"Salt Lake City VA Healthcare System, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2016,7,12]]},"reference":[{"key":"2020110612433276800_ocw097-B1","volume-title":"Most Frequent Conditions in U.S. Hospitals","author":"Pfuntner","year":"2010"},{"key":"2020110612433276800_ocw097-B2","author":"Health Services Research and Development Service Quality Enhancement Research Initiative (QUERI) Chronic Heart Failure","year":"2011"},{"issue":"8","key":"2020110612433276800_ocw097-B3","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1161\/CIR.0b013e31820a55f5","article-title":"Forecasting the future of cardiovascular disease in the United States: a policy statement from the American Heart Association","volume":"123","author":"Heidenreich","year":"2011","journal-title":"Circulation"},{"issue":"7","key":"2020110612433276800_ocw097-B4","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1016\/S0735-1097(03)00947-1","article-title":"Noncardiac comorbidity increases preventable hospitalizations and mortality among Medicare beneficiaries with chronic heart failure","volume":"42","author":"Braunstein","year":"2003","journal-title":"J Am Coll Cardiol"},{"issue":"16","key":"2020110612433276800_ocw097-B5","first-page":"e240","volume":"128","author":"Yancy","year":"2013"},{"key":"2020110612433276800_ocw097-B6","author":"American College of Cardiology Foundation, American Heart Association, Physician Consortium for Performance Improvement","year":"2012","journal-title":"Heart Failure"},{"key":"2020110612433276800_ocw097-B7","first-page":"128","article-title":"Extracting information from textual documents in the electronic health record: a review of recent research","author":"Meystre","year":"2008","journal-title":"Yearb Med Inform"},{"key":"2020110612433276800_ocw097-B8","first-page":"269","article-title":"A natural language processing system to extract and code concepts relating to congestive heart failure from chest radiology reports","author":"Friedlin","year":"2006","journal-title":"AMIA Annu Symp Proc"},{"issue":"6 Part 1","key":"2020110612433276800_ocw097-B9","first-page":"281","article-title":"Electronic medical records for clinical research: application to the identification of heart failure","volume":"13","author":"Pakhomov","year":"2007","journal-title":"Am J Manag Care"},{"issue":"12","key":"2020110612433276800_ocw097-B10","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1016\/j.ijmedinf.2012.12.005","article-title":"Automatic identification of heart failure diagnostic criteria, using text analysis of clinical notes from electronic health records","volume":"83","author":"Byrd","year":"2014","journal-title":"Int J Med Inform"},{"issue":"7","key":"2020110612433276800_ocw097-B11","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.cardfail.2014.03.008","article-title":"Prevalence of heart failure signs and symptoms in a large primary care population identified through the use of text and data mining of the electronic health record","volume":"20","author":"Vijayakrishnan","year":"2014","journal-title":"J Cardiac Failure"},{"issue":"5","key":"2020110612433276800_ocw097-B12","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1136\/jamia.2010.003947","article-title":"Extracting medication information from clinical text","volume":"17","author":"Uzuner","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110612433276800_ocw097-B13","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1136\/jamia.2010.004028","article-title":"Textractor: a hybrid system for medications and reason for their prescription extraction from clinical text documents","volume":"17","author":"Meystre","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110612433276800_ocw097-B14","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1136\/jamia.2010.003939","article-title":"High accuracy information extraction of medication information from clinical notes: 2009 i2b2 medication extraction challenge","volume":"17","author":"Patrick","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612433276800_ocw097-B15","first-page":"138","article-title":"Comparing methods for left ventricular ejection fraction clinical information extraction","author":"Meystre","year":"2012","journal-title":"AMIA Clinical Research Informatics Summit"},{"key":"2020110612433276800_ocw097-B16","first-page":"123","article-title":"Relatedness analysis of LVEF qualitative assessments and quantitative values","author":"Kim","year":"2013","journal-title":"AMIA Clinical Research Informatics Summit"},{"key":"2020110612433276800_ocw097-B17","first-page":"185","article-title":"Improving heart failure information extraction by domain adaptation","volume":"192","author":"Kim","year":"2013","journal-title":"Stud Health Technol Inform"},{"key":"2020110612433276800_ocw097-B18"},{"key":"2020110612433276800_ocw097-B19"},{"key":"2020110612433276800_ocw097-B20","author":"Ogren","year":""},{"issue":"5","key":"2020110612433276800_ocw097-B21","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1136\/amiajnl-2011-000535","article-title":"Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure","volume":"19","author":"Garvin","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612433276800_ocw097-B22","volume-title":"eHOST","author":"South"},{"key":"2020110612433276800_ocw097-B23","volume-title":"Information Retrieval","author":"van Rijsbergen","year":"1979"},{"key":"2020110612433276800_ocw097-B24"},{"key":"2020110612433276800_ocw097-B25","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.jbi.2013.11.008","article-title":"Development and evaluation of RapTAT: a machine learning system for concept mapping of phrases from medical narratives","volume":"48","author":"Gobbel","year":"2014","journal-title":"J Biomed Inform"},{"key":"2020110612433276800_ocw097-B26"},{"issue":"5","key":"2020110612433276800_ocw097-B27","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110612433276800_ocw097-B28","first-page":"39","article-title":"A maximum entropy approach to natural language processing","volume":"22","author":"Berger","year":"1996","journal-title":"Computational Linguistics."},{"issue":"2014","key":"2020110612433276800_ocw097-B29","first-page":"182","article-title":"Automatic clinical note type classification for heart failure patients","author":"Kim","journal-title":"AMIA Clinical Research Informatics Summit"},{"key":"2020110612433276800_ocw097-B30"},{"key":"2020110612433276800_ocw097-B31","article-title":"Ultraconservative online algorithms for multiclass problems","volume":"3","author":"Crammer","year":"2003","journal-title":"J Machine Learning Res"},{"key":"2020110612433276800_ocw097-B32","first-page":"715","article-title":"Using UMLS lexical resources to disambiguate abbreviations in clinical text","volume":"2011","author":"Kim","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"issue":"5","key":"2020110612433276800_ocw097-B33","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1136\/amiajnl-2011-000302","article-title":"A knowledge discovery and reuse pipeline for information extraction in clinical notes","volume":"18","author":"Patrick","year":"2011","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612433276800_ocw097-B34","volume-title":"U.S. National Library of Medicine","author":"RxNorm"},{"key":"2020110612433276800_ocw097-B35","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions and reversals","volume":"10","author":"Levenshtein","year":"1966","journal-title":"Soviet Physics Doklady"},{"key":"2020110612433276800_ocw097-B36","volume-title":"LIBSVM: a Library for Support Vector Machines","author":"Chang","year":"2001"},{"key":"2020110612433276800_ocw097-B37","first-page":"609","article-title":"Heart failure medications detection and prescription status classification in clinical narrative documents","volume":"216","author":"Meystre","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"2020110612433276800_ocw097-B38","first-page":"311","author":"Kim","year":"2011"},{"issue":"2","key":"2020110612433276800_ocw097-B39","first-page":"184","article-title":"UMLS knowledge for biomedical language processing","volume":"81","author":"McCray","year":"1993","journal-title":"Bull Med Libr Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/e1\/e40\/34149615\/ocw097.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/e1\/e40\/34149615\/ocw097.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:12:48Z","timestamp":1604686368000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/e1\/e40\/2631486"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,7,12]]},"references-count":39,"journal-issue":{"issue":"e1","published-online":{"date-parts":[[2016,7,12]]},"published-print":{"date-parts":[[2017,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw097","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,4]]},"published":{"date-parts":[[2016,7,12]]}}}