{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T13:00:21Z","timestamp":1777640421617,"version":"3.51.4"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"5","funder":[{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R01LM011366"],"award-info":[{"award-number":["R01LM011366"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Objective: Widespread application of clinical natural language processing (NLP) systems requires taking existing NLP systems and adapting them to diverse and heterogeneous settings. We describe the challenges faced and lessons learned in adapting an existing NLP system for measuring colonoscopy quality.<\/jats:p><jats:p>Materials and Methods: Colonoscopy and pathology reports from 4 settings during 2013\u20132015, varying by geographic location, practice type, compensation structure, and electronic health record.<\/jats:p><jats:p>Results: Though successful, adaptation required considerably more time and effort than anticipated. Typical NLP challenges in assembling corpora, diverse report structures, and idiosyncratic linguistic content were greatly magnified.<\/jats:p><jats:p>Discussion: Strategies for addressing adaptation challenges include assessing site-specific diversity, setting realistic timelines, leveraging local electronic health record expertise, and undertaking extensive iterative development. More research is needed on how to make it easier to adapt NLP systems to new clinical settings.<\/jats:p><jats:p>Conclusions: A key challenge in widespread application of NLP is adapting existing systems to new clinical settings.<\/jats:p>","DOI":"10.1093\/jamia\/ocx039","type":"journal-article","created":{"date-parts":[[2017,3,31]],"date-time":"2017-03-31T07:45:04Z","timestamp":1490946304000},"page":"986-991","source":"Crossref","is-referenced-by-count":116,"title":["Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings"],"prefix":"10.1093","volume":"24","author":[{"given":"David S","family":"Carrell","sequence":"first","affiliation":[{"name":"Kaiser Permanente of Washington Health Research Institute (formerly Group Health Research Institute), Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert E","family":"Schoen","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, Hepatology, and Nutrition, Department of Medicine and Epidemiology, University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel A","family":"Leffler","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, Beth Israel Deaconess Medical Center, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michele","family":"Morris","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sherri","family":"Rose","sequence":"additional","affiliation":[{"name":"Department of Health Care Policy, Harvard Medical School, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew","family":"Baer","sequence":"additional","affiliation":[{"name":"Kaiser Permanente of Washington Health Research Institute (formerly Group Health Research Institute), Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seth D","family":"Crockett","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology and Hepatology, University of North Carolina School of Medicine, Chapel Hill, NC, USA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rebecca A","family":"Gourevitch","sequence":"additional","affiliation":[{"name":"Department of Health Care Policy, Harvard Medical School, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katie M","family":"Dean","sequence":"additional","affiliation":[{"name":"Department of Health Care Policy, Harvard Medical School, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ateev","family":"Mehrotra","sequence":"additional","affiliation":[{"name":"Department of Health Care Policy, Harvard Medical School, Boston, MA, USA"},{"name":"Division of General Internal Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2017,4,17]]},"reference":[{"key":"2020110612445829200_ocx039-B1","doi-asserted-by":"crossref","first-page":"880","DOI":"10.1001\/jama.2011.1219","article-title":"The promise of electronic records: around the corner or down the road?","volume":"306","author":"Jha","year":"2011","journal-title":"JAMA."},{"issue":"2","key":"2020110612445829200_ocx039-B2","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1093\/jamia\/ocw082","article-title":"Use of electronic healthcare records to identify complex patients with atrial fibrillation for targeted intervention","volume":"24","author":"Wang","year":"2017","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B3","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1136\/amiajnl-2014-002768","article-title":"A novel method of adverse event detection can accurately identify venous thromboembolisms (VTEs) from narrative electronic health record data","volume":"22","author":"Rochefort","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B4","doi-asserted-by":"crossref","first-page":"e151","DOI":"10.1136\/amiajnl-2014-002642","article-title":"Automatic identification of methotrexate-induced liver toxicity in patients with rheumatoid arthritis from the electronic medical record","volume":"22","author":"Lin","year":"2015","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612445829200_ocx039-B5","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1093\/jamia\/ocw071","article-title":"Evaluating electronic health record data sources and algorithmic approaches to identify hypertensive individuals","volume":"24","author":"Teixeira","year":"2017","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B6","doi-asserted-by":"crossref","first-page":"e239","DOI":"10.1136\/amiajnl-2013-001889","article-title":"Deriving comorbidities from medical records using natural language processing","volume":"20","author":"Salmasian","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B7","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1136\/amiajnl-2013-001914","article-title":"Phenotyping for patient safety: algorithm development for electronic health record based automated adverse event and medical error detection in neonatal intensive care","volume":"21","author":"Li","year":"2014","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B8","doi-asserted-by":"crossref","first-page":"e306","DOI":"10.1136\/amiajnl-2013-001924","article-title":"Use of computerized algorithm to identify individuals in need of testing for celiac disease","volume":"20","author":"Ludvigsson","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B9","doi-asserted-by":"crossref","first-page":"e93","DOI":"10.1136\/amiajnl-2014-002702","article-title":"Quantifying care coordination using natural language processing and domain-specific ontology","volume":"22","author":"Popejoy","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B10","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1055\/s-0038-1638827","article-title":"State of the art in clinical informatics: evidence and examples","volume":"8","author":"McCoy","year":"2013","journal-title":"Yearb Med Inform."},{"issue":"Suppl 1","key":"2020110612445829200_ocx039-B11","doi-asserted-by":"crossref","first-page":"i150","DOI":"10.1136\/amiajnl-2011-000431","article-title":"Developing a natural language processing application for measuring the quality of colonoscopy procedures","volume":"18","author":"Harkema","year":"2011","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B12","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1016\/j.gie.2012.01.045","article-title":"Applying a natural language processing tool to electronic health records to assess performance on colonoscopy quality measures","volume":"75","author":"Mehrotra","year":"2012","journal-title":"Gastrointest Endosc."},{"key":"2020110612445829200_ocx039-B13","doi-asserted-by":"crossref","first-page":"1670","DOI":"10.1053\/j.gastro.2004.09.051","article-title":"How many endoscopies are performed for colorectal cancer screening? Results from CDC\u2019s survey of endoscopic capacity","volume":"127","author":"Seeff","year":"2004","journal-title":"Gastroenterology."},{"key":"2020110612445829200_ocx039-B14","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1056\/NEJMoa1309086","article-title":"Adenoma detection rate and risk of colorectal cancer and death","volume":"370","author":"Corley","year":"2014","journal-title":"N Engl J Med."},{"key":"2020110612445829200_ocx039-B15","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1136\/gutjnl-2011-300167","article-title":"Factors determining the quality of screening colonoscopy: a prospective study on adenoma detection rates, from 12,134 examinations (Berlin colonoscopy project 3, BECOP-3)","volume":"62","author":"Adler","year":"2013","journal-title":"Gut."},{"key":"2020110612445829200_ocx039-B16","doi-asserted-by":"crossref","first-page":"2533","DOI":"10.1056\/NEJMoa055498","article-title":"Colonoscopic withdrawal times and adenoma detection during screening colonoscopy","volume":"355","author":"Barclay","year":"2006","journal-title":"N Engl J Med."},{"key":"2020110612445829200_ocx039-B17","doi-asserted-by":"crossref","first-page":"1795","DOI":"10.1056\/NEJMoa0907667","article-title":"Quality indicators for colonoscopy and the risk of interval cancer","volume":"362","author":"Kaminski","year":"2010","journal-title":"N Engl J Med."},{"key":"2020110612445829200_ocx039-B18","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.1056\/NEJMoa1309086","article-title":"Adenoma detection rate and risk of colorectal cancer and death","volume":"370","author":"Corley","year":"2014","journal-title":"N Engl J Med."},{"key":"2020110612445829200_ocx039-B19","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1111\/j.1572-0241.2006.00673.x","article-title":"Taskforce on quality in endoscopy. Quality indicators for colonoscopy","volume":"101","author":"Rex","year":"2006","journal-title":"Am J Gastroenterol."},{"key":"2020110612445829200_ocx039-B20","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1038\/ajg.2015.51","article-title":"Multi-center colonoscopy quality measurement utilizing natural language processing","volume":"110","author":"Imler","year":"2015","journal-title":"Am J Gastroenterol."},{"key":"2020110612445829200_ocx039-B21","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/j.gie.2015.01.049","article-title":"Natural language processing as an alternative to manual reporting of colonoscopy quality metrics","volume":"82","author":"Raju","year":"2015","journal-title":"Gastrointest Endosc."},{"key":"2020110612445829200_ocx039-B22","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1016\/j.cgh.2014.05.013","article-title":"Current and future applications of natural language processing in the field of digestive diseases","volume":"12","author":"Hou","year":"2014","journal-title":"Clin Gastroenterol Hepatol."},{"key":"2020110612445829200_ocx039-B23","doi-asserted-by":"crossref","first-page":"1844","DOI":"10.1038\/ajg.2014.147","article-title":"Anatomic and advanced adenoma detection rates as quality metrics determined via natural language processing","volume":"109","author":"Gawron","year":"2014","journal-title":"Am J Gastroenterol."},{"key":"2020110612445829200_ocx039-B24","doi-asserted-by":"crossref","first-page":"1240","DOI":"10.1016\/j.gie.2012.02.031","article-title":"Colonoscopy quality, quality measures, and a natural language processing tool for electronic health records","volume":"75","author":"Deutsch","year":"2012","journal-title":"Gastrointest Endosc."},{"key":"2020110612445829200_ocx039-B25","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/j.ijmedinf.2010.09.007","article-title":"The MITRE Identification Scrubber Toolkit: design, training, and assessment","volume":"79","author":"Aberdeen","year":"2010","journal-title":"Int J Med Inform."},{"key":"2020110612445829200_ocx039-B26","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1309\/E6K33GBPE5C27FYU","article-title":"Evaluation of a deidentification (De-Id) software engine to share pathology reports and clinical documents for research","volume":"121","author":"Gupta","year":"2004","journal-title":"Am J Clin Pathol."},{"key":"2020110612445829200_ocx039-B27","first-page":"168","article-title":"GATE: an Architecture for Development of Robust HLT Applications","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics","author":"Cunningham","year":"2002"},{"key":"2020110612445829200_ocx039-B28","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1136\/jamia.2009.002733","article-title":"An overview of MetaMap: historical perspective and recent advances","volume":"17","author":"Aronson","year":"2010","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B29","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1055\/s-0038-1634945","article-title":"The Unified Medical Language System","volume":"32","author":"Lindberg","year":"1993","journal-title":"Methods Inf Med."},{"key":"2020110612445829200_ocx039-B30","volume-title":"Unified Medical Language System (UMLS)","author":"US National Library of Medicine","year":"2016"},{"key":"2020110612445829200_ocx039-B31","first-page":"17","article-title":"Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program","author":"Aronson","year":"2001","journal-title":"Proc AMIA Annual Fall Symp."},{"key":"2020110612445829200_ocx039-B32","volume-title":"MetaMap Transfer (MMTx)","author":"U.S. National Library of Medicine","year":"2011"},{"key":"2020110612445829200_ocx039-B33","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1016\/j.jbi.2009.05.002","article-title":"ConText: An algorithm for determining negation, experiencer, and temporal status from clinical reports","volume":"42","author":"Harkema","year":"2009","journal-title":"J Biomed Inform."},{"key":"2020110612445829200_ocx039-B34","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/978-94-007-5149-1_9","article-title":"Natural Language Processing \u2013 The Basics","volume-title":"Pediatric Biomedical Informatics: Computer Applications in Pediatric Research","author":"Pestian","year":"2012"},{"key":"2020110612445829200_ocx039-B35","doi-asserted-by":"crossref","first-page":"544","DOI":"10.1136\/amiajnl-2011-000464","article-title":"Natural language processing: an introduction","volume":"18","author":"Nadkarni","year":"2011","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612445829200_ocx039-B36","volume-title":"Speech and Language Processing: an Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition","author":"Jurafsky","year":"2009","edition":"2nd ed"},{"key":"2020110612445829200_ocx039-B37","doi-asserted-by":"crossref","first-page":"873012","DOI":"10.1155\/2015\/873012","article-title":"Recognition and evaluation of clinical section headings in clinical documents using token-based formulation with conditional random fields","volume":"2015","author":"Dai","year":"2015","journal-title":"Biomed Res Int."},{"key":"2020110612445829200_ocx039-B38","first-page":"156","article-title":"Development and evaluation of a clinical note section header terminology","author":"Denny","year":"2008","journal-title":"Proc AMIA Annual Fall Symp."},{"key":"2020110612445829200_ocx039-B39","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1045\/november14-klampfl","article-title":"A comparison of two unsupervised table recognition methods from digital scientific articles","volume":"20","author":"Klampfl","year":"2014","journal-title":"D-Lib Magazine."},{"key":"2020110612445829200_ocx039-B40","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1093\/aje\/kwt443","article-title":"Invited commentary: observational research in the age of the electronic health record","volume":"179","author":"Chute","year":"2014","journal-title":"Am J Epidemiol."},{"key":"2020110612445829200_ocx039-B41","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1136\/amiajnl-2011-000648","article-title":"Active learning for clinical text classification: is it better than random sampling?","volume":"19","author":"Figueroa","year":"2012","journal-title":"J Am Med Inform Assoc."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/5\/986\/34149380\/ocx039.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/5\/986\/34149380\/ocx039.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T02:31:21Z","timestamp":1658889081000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/5\/986\/3737804"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,4,17]]},"references-count":41,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2017,4,17]]},"published-print":{"date-parts":[[2017,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocx039","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,9]]},"published":{"date-parts":[[2017,4,17]]}}}