{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T21:56:52Z","timestamp":1776376612257,"version":"3.51.2"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T00:00:00Z","timestamp":1775174400000},"content-version":"vor","delay-in-days":28,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["K23DA055061"],"award-info":[{"award-number":["K23DA055061"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA051464"],"award-info":[{"award-number":["R01DA051464"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R61DA057629"],"award-info":[{"award-number":["R61DA057629"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"crossref","award":["UL1TR002003"],"award-info":[{"award-number":["UL1TR002003"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,3,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>The accurate identification of Emergency Department (ED) encounters involving opioid misuse is critical for health services, research, and surveillance. We sought to develop natural language processing (NLP)-based models for the detection of ED encounters involving opioid misuse.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A sample of ED encounters enriched for opioid misuse was manually annotated and clinical notes extracted. We evaluated classic machine learning (ML) methods, fine-tuning of publicly available pretrained language models, and a previously developed convolutional neural network opioid classifier for use on hospitalized patients (SMART-AI). Performance was benchmarked to opioid-related ICD-10-CM codes. Both raw text and text transformed to the Unified Medical Language System were evaluated. Face validity was evaluated by term feature importance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>There were 1123 encounters used for training, validation, and testing. Of the classic ML methods, XGBoost had the highest AU_PRC 0.9358 (95% CI 0.8945, 0.9681), accuracy 0.8874 (0.8402, 0.9349), and F1 score 0.8624 (0.7969, 0.9197) which performed comparably to ICD-10-CM codes [accuracy 0.8687 (0.8155, 0.9167); F1 0.8296 (0.7544, 0.8939)]. Excluding XGBoost, fine-tuned pre-trained language models generally outperformed classic ML methods. The best performing model by point estimate was the fine-tuned SMART-AI based model with domain adaptation [AU_PRC 0.9474 (0.9113, 0.9749); accuracy 0.8816 (0.8284, 0.9290); F1 0.8499 (0.7805, 0.9103)] but confidence intervals overlapped with other models. Explainability analyses showed the most predictive terms were \u201cheroin,\u201d \u201copioids,\u201d \u201calcoholic intoxication, chronic,\u201d \u201ccocaine,\u201d \u201copiates,\u201d and \u201csuboxone.\u201d<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>NLP-based models perform comparably to entry of ICD-10-CM diagnosis codes for the detection of ED encounters with opioid misuse. Fine tuning with domain adaptation for pre-trained language models resulted in improved performance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooag042","type":"journal-article","created":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T12:41:49Z","timestamp":1773751309000},"source":"Crossref","is-referenced-by-count":0,"title":["Development and evaluation of machine learning models for the detection of emergency department patients with opioid misuse from clinical notes"],"prefix":"10.1093","volume":"9","author":[{"given":"Usman","family":"Shahid","sequence":"first","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Natural Language Processing Laboratory, Department of Computer Science, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Natalie","family":"Parde","sequence":"additional","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Natural Language Processing Laboratory, Department of Computer Science, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dale L","family":"Smith","sequence":"additional","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Department of Psychiatry, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Grayson","family":"Dickinson","sequence":"additional","affiliation":[{"name":"College of Osteopathic Medicine, William Carey University , Hattiesburg, MS,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Bianco","sequence":"additional","affiliation":[{"name":"University of Illinois College of Medicine , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dillon","family":"Thorpe","sequence":"additional","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Department of Psychiatry, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Madhav","family":"Hota","sequence":"additional","affiliation":[{"name":"University of Missouri-Kansas City , Kansas City, MO,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Medicine, School of Medicine and Public Health, University of Wisconsin Madison , Madison, WI,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Niranjan S","family":"Karnik","sequence":"additional","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Department of Psychiatry, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Institute for Research on Addictions, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0945-1233","authenticated-orcid":false,"given":"Neeraj","family":"Chhabra","sequence":"additional","affiliation":[{"name":"AI.Health4All Center for Health Equity using Machine Learning and Artificial Intelligence, College of Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Institute for Research on Addictions, University of Illinois Chicago , Chicago, IL,","place":["United States"]},{"name":"Department of Emergency Medicine, University of Illinois Chicago , Chicago, IL,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2026,4,3]]},"reference":[{"key":"2026041617102104300_ooag042-B1","author":"Abuse NI on D. Drug Overdose Death Rates | National Institute on Drug Abuse (NIDA) [Internet]","year":"2024"},{"key":"2026041617102104300_ooag042-B2","doi-asserted-by":"crossref","first-page":"2487","DOI":"10.1056\/NEJMp1811988","article-title":"Emergency Departments - A 24\/7\/365 option for combating the opioid crisis","volume":"379","author":"D\u2019Onofrio","year":"2018","journal-title":"N Engl J Med"},{"key":"2026041617102104300_ooag042-B3","doi-asserted-by":"crossref","first-page":"567","DOI":"10.15585\/mmwr.mm7325a1","article-title":"Treatment for opioid use disorder: Population estimates\u2013United States, 2022","volume":"73","author":"Dowell","year":"2024","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"2026041617102104300_ooag042-B4","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1007\/s11606-019-05249-3","article-title":"US ED Opioid-Related visits increase, while use of medication for opioid use disorder undetectable, 2011-2016","volume":"35","author":"Shastry","year":"2020","journal-title":"J Gen Intern Med"},{"key":"2026041617102104300_ooag042-B5","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1016\/j.jemermed.2019.03.018","article-title":"Screening tools validated in the outpatient pain management setting poorly predict opioid misuse in the emergency department: a pilot study","volume":"56","author":"Chalmers","year":"2019","journal-title":"J Emerg Med"},{"key":"2026041617102104300_ooag042-B6","author":"Abuse NI on D. Summary of Misuse of Prescription Drugs [Internet]","year":"2023"},{"key":"2026041617102104300_ooag042-B7","author":"Commonly Used Terms | Opioids | CDC [Internet]","year":"2023"},{"key":"2026041617102104300_ooag042-B8","author":"2015 National Survey on Drug Use and Health: Methodological Summary and Definitions | CBHSQ Data [Internet]","year":"2023"},{"key":"2026041617102104300_ooag042-B9","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1016\/j.annemergmed.2023.04.030","article-title":"Death by a thousand screens: a practical role for machine learning in emergency medicine","volume":"82","author":"Chhabra","year":"2023","journal-title":"Ann Emerg Med"},{"key":"2026041617102104300_ooag042-B10","first-page":"766","article-title":"Performance of international classification of disease-10 codes in detecting emergency department patients with opioid misuse","volume-title":"Addiction.","author":"Chhabra","year":"2023"},{"key":"2026041617102104300_ooag042-B11","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1186\/s12911-020-1099-y","article-title":"Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients","volume":"20","author":"Sharma","year":"2020","journal-title":"BMC Med Inform Decis Mak"},{"key":"2026041617102104300_ooag042-B12","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s13722-021-00229-7","article-title":"External validation of an opioid misuse machine learning classifier in hospitalized adult patients","volume":"16","author":"Afshar","year":"2021","journal-title":"Addict Sci Clin Pract"},{"key":"2026041617102104300_ooag042-B13","first-page":"e426\u201335","article-title":"Development and multimodal validation of a substance misuse algorithm for referral to treatment using artificial intelligence (SMART-AI): a retrospective deep learning study","volume":"4","author":"Afshar","year":"2022","journal-title":"Lancet Digit Health"},{"key":"2026041617102104300_ooag042-B14","author":"Anthony","year":"2021"},{"key":"2026041617102104300_ooag042-B15","author":"\u2022. As Opioid Overdose Deaths Hit New Record, Pressure Grows for Safe Places to Inject Drugs in Chicago [Internet]","year":"2024"},{"key":"2026041617102104300_ooag042-B16","doi-asserted-by":"crossref","first-page":"e0219717","DOI":"10.1371\/journal.pone.0219717","article-title":"Subtypes in patients with opioid misuse: a prognostic enrichment strategy using electronic health record data in hospitalized patients","volume":"14","author":"Afshar","year":"2019","journal-title":"PLoS One"},{"key":"2026041617102104300_ooag042-B17","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"Harris","year":"2009","journal-title":"J Biomed Inform"},{"key":"2026041617102104300_ooag042-B18","doi-asserted-by":"crossref","first-page":"103208","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: Building an international community of software platform partners","volume":"95","author":"Harris","year":"2019","journal-title":"J Biomed Inform"},{"key":"2026041617102104300_ooag042-B19","doi-asserted-by":"crossref","first-page":"103626","DOI":"10.1016\/j.jbi.2020.103626","article-title":"Pre-training phenotyping classifiers","volume":"113","author":"Dligach","year":"2021","journal-title":"J Biomed Inform"},{"key":"2026041617102104300_ooag042-B20","author":"Devlin","year":"2019"},{"key":"2026041617102104300_ooag042-B21","author":"Beltagy","year":"2020"},{"key":"2026041617102104300_ooag042-B22","author":"Lee","year":"2019"},{"key":"2026041617102104300_ooag042-B23","author":"Li","year":"2022"},{"key":"2026041617102104300_ooag042-B24","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2026041617102104300_ooag042-B25","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1186\/s12911-018-0654-2","article-title":"Comparison of MetaMap and cTAKES for entity extraction in clinical notes","volume":"18","author":"Re\u00e1tegui","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"key":"2026041617102104300_ooag042-B26","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1177\/0033354920904087","article-title":"ICD-10-CM-Based definitions for emergency department opioid poisoning surveillance: Electronic health record case confirmation study","volume":"135","author":"Slavova","year":"2020","journal-title":"Public Health Rep"},{"key":"2026041617102104300_ooag042-B27","author":"Weiss","year":"2019"},{"key":"2026041617102104300_ooag042-B28","author":"Ribeiro","year":"2016"},{"key":"2026041617102104300_ooag042-B29","author":"Paszke","year":"2019"},{"key":"2026041617102104300_ooag042-B30","doi-asserted-by":"crossref","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"Bmj"},{"key":"2026041617102104300_ooag042-B31","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1002\/pds.5591","article-title":"Development and validation of an electronic health records-based opioid use disorder algorithm by expert clinical adjudication among patients with prescribed opioids","volume":"32","author":"Ranapurwala","year":"2023","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"2026041617102104300_ooag042-B32","doi-asserted-by":"crossref","first-page":"e2435603","DOI":"10.1001\/jamanetworkopen.2024.35603","article-title":"Trends and disparities in initiation of buprenorphine in US emergency departments, 2013-2022","volume":"7","author":"Chhabra","year":"2024","journal-title":"JAMA Netw Open"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag042\/8586317","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag042\/8586317","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T21:10:28Z","timestamp":1776373828000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag042\/8586317"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,6]]},"references-count":32,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,3,6]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooag042","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,4]]},"published":{"date-parts":[[2026,3,6]]},"article-number":"ooag042"}}