{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T03:26:50Z","timestamp":1788838010729,"version":"build-2803163510"},"reference-count":34,"publisher":"Wiley","issue":"8","license":[{"start":{"date-parts":[[2019,6,19]],"date-time":"2019-06-19T00:00:00Z","timestamp":1560902400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"Opioid Postmarketing Consortium"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Pharmacoepidemiology and Drug"],"published-print":{"date-parts":[[2019,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Purpose<\/jats:title><jats:p>To enhance automated methods for accurately identifying opioid\u2010related overdoses and classifying types of overdose using electronic health record (EHR) databases.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We developed a natural language processing (NLP) software application to code clinical text documentation of overdose, including identification of intention for self\u2010harm, substances involved, substance abuse, and error in medication usage. Using datasets balanced with cases of suspected overdose and records of individuals at elevated risk for overdose, we developed and validated the application using Kaiser Permanente Northwest data, then tested portability of the application using Kaiser Permanente Washington data. Datasets were chart\u2010reviewed to provide a gold standard for comparison and evaluation of the automated method.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The method performed well in identifying overdose (sensitivity\u00a0=\u00a00.80, specificity\u00a0=\u00a00.93), intentional overdose (sensitivity\u00a0=\u00a00.81, specificity\u00a0=\u00a00.98), and involvement of opioids (excluding heroin, sensitivity\u00a0=\u00a00.72, specificity\u00a0=\u00a00.96) and heroin (sensitivity\u00a0=\u00a00.84, specificity\u00a0=\u00a01.0). The method performed poorly at identifying adverse drug reactions and overdose due to patient error and fairly at identifying substance abuse in opioid\u2010related unintentional overdose (sensitivity\u00a0=\u00a00.67, specificity\u00a0=\u00a00.96). Evaluation using validation datasets yielded significant reductions, in specificity and negative predictive values only, for many classifications mentioned above. However, these measures remained above 0.80, thus, performance observed during development was largely maintained during validation. Similar results were obtained when evaluating portability, although there was a significant reduction in sensitivity for unintentional overdose that was attributed to missing text clinical notes in the database.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>Methods that process text clinical notes show promise for improving accuracy and fidelity at identifying and classifying overdoses according to type using EHR data.<\/jats:p><\/jats:sec>","DOI":"10.1002\/pds.4810","type":"journal-article","created":{"date-parts":[[2019,6,20]],"date-time":"2019-06-20T04:13:32Z","timestamp":1561004012000},"page":"1143-1151","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Using natural language processing of clinical text to enhance identification of opioid\u2010related overdoses in electronic health records data"],"prefix":"10.1002","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9365-3964","authenticated-orcid":false,"given":"Brian","family":"Hazlehurst","sequence":"first","affiliation":[{"name":"Center for Health Research Kaiser Permanente Northwest  Portland OR"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0000-4381","authenticated-orcid":false,"given":"Carla A.","family":"Green","sequence":"additional","affiliation":[{"name":"Center for Health Research Kaiser Permanente Northwest  Portland OR"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2025-6543","authenticated-orcid":false,"given":"Nancy A.","family":"Perrin","sequence":"additional","affiliation":[{"name":"Center for Health Research Kaiser Permanente Northwest  Portland OR"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Brandes","sequence":"additional","affiliation":[{"name":"Center for Health Research Kaiser Permanente Northwest  Portland OR"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8471-0928","authenticated-orcid":false,"given":"David S.","family":"Carrell","sequence":"additional","affiliation":[{"name":"Health Research Institute Kaiser Permanente Washington  Seattle WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Baer","sequence":"additional","affiliation":[{"name":"Group Health Research Institute, Group Health Cooperative  Seattle WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angela","family":"DeVeaugh\u2010Geiss","sequence":"additional","affiliation":[{"name":"Epidemiology, Medical Affairs, Purdue Pharma, LP  Stamford CT"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul M.","family":"Coplan","sequence":"additional","affiliation":[{"name":"Epidemiology, Medical Affairs, Purdue Pharma, LP  Stamford CT"},{"name":"Adjunct, Perelman School of Medicine University of Pennsylvania  Philadelphia PA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2019,6,19]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMp1011512"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-012-2225-z"},{"issue":"42","key":"e_1_2_9_4_1","first-page":"1171","article-title":"Overdose deaths involving prescription opioids among Medicaid enrollees\u2014Washington, 2004\u20102007","volume":"58","author":"Centers for Disease Control and Prevention","year":"2009","journal-title":"MMWR"},{"issue":"43","key":"e_1_2_9_5_1","first-page":"1487","article-title":"Vital signs: overdoses of prescription opioid pain relievers\u2014United States, 199\u20132008","volume":"60","author":"Centers for Disease Control and Prevention","year":"2011","journal-title":"MMWR"},{"issue":"81","key":"e_1_2_9_6_1","first-page":"1","article-title":"Drug poisoning deaths in the United States, 1980\u20102008","author":"Warner M","year":"2011","journal-title":"NCHS Data Brief"},{"key":"e_1_2_9_7_1","first-page":"1","article-title":"Increase in fatal poisonings involving opioid analgesics in the United States, 1999\u20102006","volume":"22","author":"Warner M","year":"2009","journal-title":"NCHS Data Brief"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0054496"},{"issue":"39","key":"e_1_2_9_9_1","first-page":"849","article-title":"Increases in heroin overdose deaths\u201328 states, 2010 to 2012","volume":"63","author":"Rudd RA","year":"2014","journal-title":"MMWR"},{"key":"e_1_2_9_10_1","first-page":"1","article-title":"Increases in drug and opioid overdose deaths\u2014United States, 2000\u20132014","volume":"64","author":"Rudd RA","year":"2016","journal-title":"Morb Mortal Wkly Rep"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.5811\/westjem.2012.7.12936"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsat.2014.07.012"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpain.2008.10.008"},{"key":"e_1_2_9_14_1","unstructured":"Washington State Agency Medical Directors' Group (AMDG).Interagency Guideline on Prescribing Opioids for Pain 06\/2015; 3rd. Available athttp:\/\/www.agencymeddirectors.wa.gov\/Files\/2015AMDGOpioidGuideline.pdf. Accessed 11\/17\/2016."},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.15585\/mmwr.rr6501e1"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2012.8165"},{"key":"e_1_2_9_17_1","unstructured":"U. S. Department of Health & Human Services.Abuse\u2010Deterrent Opioids \u2010 Evaluation and Labeling: Guidance for Industry. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER);2015 Washington DC. Available Online at:https:\/\/www.fda.gov\/regulatory\u2010information\/search\u2010fdaguidance\u2010documents\/abuse\u2010deterrent\u2010opioids\u2010evaluation\u2010and\u2010labeling"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/ajim.21998"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.3109\/00952990.2014.988339"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.2165\/11635860-000000000-00000"},{"issue":"7","key":"e_1_2_9_21_1","first-page":"412","article-title":"A review of abuse\u2010deterrent opioids for chronic nonmalignant pain","volume":"37","author":"Moorman\u2010Li R","year":"2012","journal-title":"Pharm Ther"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/pds.4157"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-152-2-201001190-00006"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/acem.13121"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.1002\/pds.4772"},{"key":"e_1_2_9_26_1","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1771"},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2005.08.007"},{"key":"e_1_2_9_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.vaccine.2009.01.105"},{"issue":"6","key":"e_1_2_9_29_1","first-page":"313","article-title":"Automating care quality measurement with health information technology","volume":"18","author":"Hazelhurst B","year":"2012","journal-title":"Am J Manag Care"},{"key":"e_1_2_9_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2014.01.001"},{"key":"e_1_2_9_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2015.06.002"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-001935"},{"key":"e_1_2_9_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2015.09.002"},{"key":"e_1_2_9_34_1","doi-asserted-by":"publisher","DOI":"10.1097\/j.pain.0000000000000145"},{"key":"e_1_2_9_35_1","doi-asserted-by":"publisher","DOI":"10.1093\/pm\/pnw283"}],"container-title":["Pharmacoepidemiology and Drug Safety"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpds.4810","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pds.4810","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/pds.4810","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pds.4810","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T17:39:53Z","timestamp":1694281193000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.4810"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,19]]},"references-count":34,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2019,8]]}},"alternative-id":["10.1002\/pds.4810"],"URL":"https:\/\/doi.org\/10.1002\/pds.4810","archive":["Portico"],"relation":{},"ISSN":["1053-8569","1099-1557"],"issn-type":[{"value":"1053-8569","type":"print"},{"value":"1099-1557","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,19]]},"assertion":[{"value":"2018-09-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-05-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-06-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}