{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T14:29:53Z","timestamp":1754144993136,"version":"3.41.2"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T00:00:00Z","timestamp":1750636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SGATs"},{"name":"Tufts CTSI","award":["UM1TR004398"],"award-info":[{"award-number":["UM1TR004398"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>People who use drugs (PWUD) often lack access to optimal harm reduction and substance use disorder treatment tools. Tracking the epidemiology of acute care utilization by PWUD is crucial to improving systems of care. Chart reviews and International Classification of Diseases (ICD) codes are the most common systems of identifying hospitalizations of PWUD but are limited by high labor costs and inaccuracy. This study evaluates whether natural language processing (NLP) enhances the sensitivity and specificity of ICD-10 codes in identifying hospitalizations of PWUD.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We analyzed admissions at Tufts Medical Center between 2018 and 2023. Two NLP tools (Regular Expression and Open Health NLP Toolkit) were developed to identify PWUD and were compared with ICD-10 algorithms. The NLP and ICD-10 algorithms were applied to all admissions, and demographic and hospitalization-related data were extracted. The research team manually reviewed notes written during 790 hospitalizations of PWUD as the gold standard. We calculated sensitivity, specificity, and net reclassification indices.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>ICD-10 codes alone demonstrated low sensitivity (43%) but high specificity (99%). Adding NLP systems improved sensitivity up to 94%, though specificity decreased to 46%. Threshold adjustments (eg, notes flagged \u226550%) revealed a trade-off between sensitivity (47%) and specificity (96%). The most practical model\u2014Regular Expression or ICD-10 codes\u2014resulted in a sensitivity of 74% and specificity of 87%.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>NLP is an innovative tool that can create functional, cost-effective, and accurate systems of identifying hospitalized PWUD. These findings support further development of NLP technologies to improve health care equity for PWUD.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/ofid\/ofaf370","type":"journal-article","created":{"date-parts":[[2025,6,28]],"date-time":"2025-06-28T22:16:06Z","timestamp":1751148966000},"source":"Crossref","is-referenced-by-count":0,"title":["Sensitivity and Specificity of Natural Language Processing Systems for Identification of Hospitalized People Who Use Drugs"],"prefix":"10.1093","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4179-1502","authenticated-orcid":false,"given":"Leah","family":"Benrubi","sequence":"first","affiliation":[{"name":"School of Medicine, Tufts University, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6830-2812","authenticated-orcid":false,"given":"Taisuke","family":"Sato","sequence":"additional","affiliation":[{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]},{"name":"College of Osteopathic Medicine, Kansas City University, Kansas City, MO ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leo K","family":"Westgard","sequence":"additional","affiliation":[{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1615-590X","authenticated-orcid":false,"given":"Kyle","family":"Zollo-Venecek","sequence":"additional","affiliation":[{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brindet","family":"Socrates","sequence":"additional","affiliation":[{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]},{"name":"Department of Medicine, Boston Medical Center, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"Sweigart","sequence":"additional","affiliation":[{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica P","family":"Ridgway","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Chicago, Chicago, IL ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5659-6147","authenticated-orcid":false,"given":"Joji","family":"Suzuki","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoelkys","family":"Morales","sequence":"additional","affiliation":[{"name":"School of Medicine, Tufts University, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9382-3564","authenticated-orcid":false,"given":"David","family":"Goodman-Meza","sequence":"additional","affiliation":[{"name":"Kirby Institute, University of New South Wales , Sydney ,","place":["Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8255-8387","authenticated-orcid":false,"given":"Alysse G","family":"Wurcel","sequence":"additional","affiliation":[{"name":"School of Medicine, Tufts University, Boston, MA ,","place":["USA"]},{"name":"Tufts Medicine, Boston, MA ,","place":["USA"]},{"name":"Department of Medicine, Boston Medical Center, Boston, MA ,","place":["USA"]},{"name":"School of Medicine, Boston University, Boston, MA ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,6,23]]},"reference":[{"key":"2025071604212378900_ofaf370-B1","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1136\/postgradmedj-2015-133720","article-title":"Optimising health and safety of people who inject drugs during transition from acute to outpatient care: narrative review with clinical checklist","volume":"92","author":"Thakarar","year":"2016","journal-title":"Postgrad Med J"},{"key":"2025071604212378900_ofaf370-B2","doi-asserted-by":"crossref","first-page":"e12","DOI":"10.1093\/cid\/ciad664","article-title":"Frame shift: focusing on harm reduction and shared decision making for people who use drugs hospitalized with infections","volume":"78","author":"Thakarar","year":"2024","journal-title":"Clin Infect Dis"},{"key":"2025071604212378900_ofaf370-B3","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1377\/hlthaff.2015.1424","article-title":"Hospitalizations related to opioid abuse\/dependence and associated serious infections increased sharply, 2002\u201312","volume":"35","author":"Ronan","year":"2016","journal-title":"Health Aff (Millwood)"},{"key":"2025071604212378900_ofaf370-B4","doi-asserted-by":"crossref","first-page":"ofw157","DOI":"10.1093\/ofid\/ofw157","article-title":"Increasing infectious endocarditis admissions among young people who inject drugs","volume":"3","author":"Wurcel","year":"2016","journal-title":"Open Forum Infect Dis"},{"key":"2025071604212378900_ofaf370-B5","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.drugalcdep.2016.11.029","article-title":"A systematic review of injecting-related injury and disease among people who inject drugs","volume":"171","author":"Larney","year":"2017","journal-title":"Drug Alcohol Depend"},{"author":"Brothers","key":"2025071604212378900_ofaf370-B6","doi-asserted-by":"publisher","DOI":"10.1101\/2024.09.20.24313898"},{"key":"2025071604212378900_ofaf370-B7","doi-asserted-by":"crossref","first-page":"e2016228","DOI":"10.1001\/jamanetworkopen.2020.16228","article-title":"Association of treatment with medications for opioid use disorder with mortality after hospitalization for injection drug use\u2013associated infective endocarditis","volume":"3","author":"Kimmel","year":"2020","journal-title":"JAMA Netw Open"},{"key":"2025071604212378900_ofaf370-B8","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1093\/cid\/ciaa062","article-title":"Outcomes associated with medications for opioid use disorder among persons hospitalized for infective endocarditis","volume":"72","author":"Barocas","year":"2021","journal-title":"Clin Infect Dis"},{"key":"2025071604212378900_ofaf370-B9","doi-asserted-by":"crossref","first-page":"e53510","DOI":"10.2196\/53510","article-title":"Occurrence of stigmatizing documentation among hospital medicine encounters with opioid-related diagnosis codes: cohort study","volume":"8","author":"Bradford","year":"2024","journal-title":"JMIR Form Res"},{"key":"2025071604212378900_ofaf370-B10","doi-asserted-by":"crossref","first-page":"209392","DOI":"10.1016\/j.josat.2024.209392","article-title":"Perspectives on benefits and risks of creation of an \u201cinjection drug use\u201d billing code","volume":"164","author":"Sundaram","year":"2024","journal-title":"J Subst Use Addict Treat"},{"key":"2025071604212378900_ofaf370-B11","doi-asserted-by":"crossref","first-page":"ofae030","DOI":"10.1093\/ofid\/ofae030","article-title":"Impact of potential case misclassification by administrative diagnostic codes on outcome assessment of observational study for people who inject drugs","volume":"11","author":"Goodman-Meza","year":"2024","journal-title":"Open Forum Infect Dis"},{"key":"2025071604212378900_ofaf370-B12","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1097\/ADM.0000000000000814","article-title":"International Classification of Diseases (ICD) codes fail to accurately identify injection drug use associated endocarditis cases","volume":"16","author":"Barnes","year":"2022","journal-title":"J Addict Med"},{"key":"2025071604212378900_ofaf370-B13","doi-asserted-by":"crossref","first-page":"107906","DOI":"10.1016\/j.drugalcdep.2020.107906","article-title":"Validity of ICD-based algorithms to estimate the prevalence of injection drug use among infective endocarditis hospitalizations in the absence of a reference standard","volume":"209","author":"McGrew","year":"2020","journal-title":"Drug Alcohol Depend"},{"key":"2025071604212378900_ofaf370-B14","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1080\/00952990.2022.2047713","article-title":"Sensitivity, specificity and predictive values of ICD-10 substance use codes in a cohort of substance use\u2013related endocarditis patients","volume":"48","author":"Campanile","year":"2022","journal-title":"Am J Drug Alcohol Abuse"},{"key":"2025071604212378900_ofaf370-B15","doi-asserted-by":"crossref","first-page":"ofaa414","DOI":"10.1093\/ofid\/ofaa414","article-title":"Use of ICD-10 codes for identification of injection drug use\u2013associated infective endocarditis is nonspecific and obscures critical findings on impact of medications for opioid use disorder","volume":"7","author":"Marks","year":"2020","journal-title":"Open Forum Infect Dis"},{"key":"2025071604212378900_ofaf370-B16","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.drugpo.2018.02.001","article-title":"Identifying injection drug use and estimating population size of people who inject drugs using healthcare administrative datasets","volume":"55","author":"Janjua","year":"2018","journal-title":"Int J Drug Policy"},{"key":"2025071604212378900_ofaf370-B17","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1093\/cid\/ciy632","article-title":"Many opportunities to record, diagnose, or treat injection drug\u2013related infections are missed: a population-based cohort study of inpatient and emergency department settings","volume":"68","author":"Miller","year":"2019","journal-title":"Clin Infect Dis"},{"key":"2025071604212378900_ofaf370-B18","doi-asserted-by":"crossref","first-page":"ofac471","DOI":"10.1093\/ofid\/ofac471","article-title":"Natural language processing and machine learning to identify people who inject drugs in electronic health records","volume":"9","author":"Goodman-Meza","year":"2022","journal-title":"Open Forum Infect Dis"},{"key":"2025071604212378900_ofaf370-B19","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1002\/pds.4810","article-title":"Using natural language processing of clinical text to enhance identification of opioid-related overdoses in electronic health records data","volume":"28","author":"Hazlehurst","year":"2019","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"2025071604212378900_ofaf370-B20","first-page":"233","article-title":"Using natural language processing and machine learning to identify hospitalized patients with opioid use disorder","volume":"2020","author":"Blackley","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"2025071604212378900_ofaf370-B21","doi-asserted-by":"crossref","first-page":"850619","DOI":"10.3389\/fpubh.2022.850619","article-title":"Classifying characteristics of opioid use disorder from hospital discharge summaries using natural language processing","volume":"10","author":"Poulsen","year":"2022","journal-title":"Front Public Health"},{"author":"Tufts Medical Center. Hospital profile","key":"2025071604212378900_ofaf370-B22"},{"key":"2025071604212378900_ofaf370-B23","doi-asserted-by":"crossref","first-page":"104963","DOI":"10.1016\/j.ijmedinf.2022.104963","article-title":"Using natural language processing to identify opioid use disorder in electronic health record data","volume":"170","author":"Singleton","year":"2023","journal-title":"Int J Med Inf"},{"key":"2025071604212378900_ofaf370-B24","doi-asserted-by":"crossref","first-page":"836733","DOI":"10.3389\/fdgth.2022.836733","article-title":"A method to advance adolescent sexual health research: automated algorithm finds sexual history documentation","volume":"4","author":"Robertson","year":"2022","journal-title":"Front Digit Health"},{"author":"Qqndrew","key":"2025071604212378900_ofaf370-B25"},{"key":"2025071604212378900_ofaf370-B26","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2025071604212378900_ofaf370-B27","doi-asserted-by":"crossref","first-page":"2546","DOI":"10.1093\/jamia\/ocab153","article-title":"Corrigendum to: the 2019 National Natural Language Processing (NLP) Clinical Challenges (n2c2)\/Open Health NLP (OHNLP) shared task on clinical concept normalization for clinical records","volume":"28","author":"Henry","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2025071604212378900_ofaf370-B28","doi-asserted-by":"crossref","first-page":"e49997","DOI":"10.2196\/49997","article-title":"A case demonstration of the open health natural language processing toolkit from the national COVID-19 cohort collaborative and the researching COVID to enhance recovery programs for a natural language processing system for COVID-19 or postacute sequelae of SARS CoV-2 infection: algorithm development and validation","volume":"12","author":"Wen","year":"2024","journal-title":"JMIR Med Inform"},{"key":"2025071604212378900_ofaf370-B29","doi-asserted-by":"crossref","first-page":"105144","DOI":"10.1016\/j.ijmedinf.2023.105144","article-title":"ReDWINE: a clinical datamart with text analytical capabilities to facilitate rehabilitation research","volume":"177","author":"Oniani","year":"2023","journal-title":"Int J Med Inf"},{"key":"2025071604212378900_ofaf370-B30","doi-asserted-by":"crossref","first-page":"e23099","DOI":"10.2196\/23099","article-title":"Predicting semantic similarity between clinical sentence pairs using transformer models: evaluation and representational analysis","volume":"9","author":"Ormerod","year":"2021","journal-title":"JMIR Med Inform"},{"key":"2025071604212378900_ofaf370-B31","doi-asserted-by":"crossref","first-page":"ofad600","DOI":"10.1093\/ofid\/ofad600","article-title":"Effect of antibiotic administration before joint aspiration on synovial fluid white blood cell count in native joint septic arthritis","volume":"11","author":"Puzzitiello","year":"2024","journal-title":"Open Forum Infect Dis"},{"key":"2025071604212378900_ofaf370-B32","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1089\/apc.2022.0165","article-title":"Racial and ethnic disparities in HIV testing in people who use drugs admitted to a tertiary care hospital","volume":"36","author":"Hamdan","year":"2022","journal-title":"AIDS Patient Care STDS"},{"key":"2025071604212378900_ofaf370-B33","doi-asserted-by":"crossref","first-page":"107825","DOI":"10.1016\/j.drugalcdep.2019.107825","article-title":"Validity of International Classification of Diseases codes in identifying illicit drug use target conditions using medical record data as a reference standard: a systematic review","volume":"208","author":"McGrew","year":"2020","journal-title":"Drug Alcohol Depend"},{"key":"2025071604212378900_ofaf370-B34","doi-asserted-by":"crossref","first-page":"2420","DOI":"10.1007\/s11606-021-07069-w","article-title":"National prevalence of alcohol and other substance use disorders among emergency department visits and hospitalizations: NHAMCS 2014\u20132018","volume":"37","author":"Suen","year":"2022","journal-title":"J Gen Intern Med"},{"key":"2025071604212378900_ofaf370-B35","doi-asserted-by":"crossref","first-page":"744","DOI":"10.1002\/jhm.12893","article-title":"Management of opioid use disorder and associated conditions among hospitalized adults: a consensus statement from the Society of Hospital Medicine","volume":"17","author":"Calcaterra","year":"2022","journal-title":"J Hosp Med"},{"key":"2025071604212378900_ofaf370-B36","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1002\/pds.2313","article-title":"A systematic review of validated methods for identifying heart failure using administrative data","volume":"21(suppl 1)","author":"Saczynski","year":"2012","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"2025071604212378900_ofaf370-B37","doi-asserted-by":"crossref","first-page":"108266","DOI":"10.1016\/j.jsat.2020.108266","article-title":"Increased HIV testing in people who use drugs hospitalized in the first wave of the COVID-19 pandemic","volume":"124","author":"Zubiago","year":"2021","journal-title":"J Subst Abuse Treat"},{"key":"2025071604212378900_ofaf370-B38","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.giec.2024.09.005","article-title":"Natural language processing in gastroenterology: current applications and future directions","volume":"35","author":"Parasa","year":"2025","journal-title":"Gastrointest Endosc Clin N Am"},{"key":"2025071604212378900_ofaf370-B39","doi-asserted-by":"crossref","first-page":"5456","DOI":"10.1038\/s41598-024-83540-y","article-title":"Ontology-guided machine learning outperforms zero-shot foundation models for cardiac ultrasound text reports","volume":"15","author":"Subramaniam","year":"2025","journal-title":"Sci Rep"},{"key":"2025071604212378900_ofaf370-B40","doi-asserted-by":"crossref","first-page":"e0297519","DOI":"10.1371\/journal.pone.0297519","article-title":"Development and evaluation of a computerized algorithm for the interpretation of pulmonary function tests","volume":"19","author":"Huang","year":"2024","journal-title":"PLoS One"},{"key":"2025071604212378900_ofaf370-B41","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s12954-024-00971-6","article-title":"Structural stigma within inpatient care for people who inject drugs: implications for harm reduction","volume":"21","author":"Rehman","year":"2024","journal-title":"Harm Reduct J"},{"key":"2025071604212378900_ofaf370-B42","first-page":"847","article-title":"Between public and private: electronic health record-sharing, health privacy principles, and hepatitis C","volume":"30","author":"Mulcahy","year":"2023","journal-title":"J Law Med"},{"key":"2025071604212378900_ofaf370-B43","doi-asserted-by":"crossref","first-page":"109713","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: a scoping review","volume":"27","author":"Meng","year":"2024","journal-title":"iScience"},{"key":"2025071604212378900_ofaf370-B44","doi-asserted-by":"crossref","first-page":"bbac409","DOI":"10.1093\/bib\/bbac409","article-title":"BioGPT: generative pre-trained transformer for biomedical text generation and mining","volume":"23","author":"Luo","year":"2022","journal-title":"Brief Bioinform"},{"author":"Weidinger","key":"2025071604212378900_ofaf370-B45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2112.04359"}],"container-title":["Open Forum Infectious Diseases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ofid\/advance-article-pdf\/doi\/10.1093\/ofid\/ofaf370\/63561428\/ofaf370.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ofid\/article-pdf\/12\/7\/ofaf370\/63561428\/ofaf370.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ofid\/article-pdf\/12\/7\/ofaf370\/63561428\/ofaf370.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T08:21:34Z","timestamp":1752654094000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ofid\/article\/doi\/10.1093\/ofid\/ofaf370\/8172079"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,23]]},"references-count":45,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,6,30]]}},"URL":"https:\/\/doi.org\/10.1093\/ofid\/ofaf370","relation":{},"ISSN":["2328-8957"],"issn-type":[{"type":"electronic","value":"2328-8957"}],"subject":[],"published-other":{"date-parts":[[2025,7]]},"published":{"date-parts":[[2025,6,23]]},"article-number":"ofaf370"}}