{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T16:30:36Z","timestamp":1773851436659,"version":"3.50.1"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2021,4,30]],"date-time":"2021-04-30T00:00:00Z","timestamp":1619740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Stony Brook University OVPR Seed Grant"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The United States is experiencing an opioid epidemic. In recent years, there were more than 10 million opioid misusers aged 12 years or older annually. Identifying patients at high risk of opioid use disorder (OUD) can help to make early clinical interventions to reduce the risk of OUD. Our goal is to develop and evaluate models to predict OUD for patients on opioid medications using electronic health records and deep learning methods. The resulting models help us to better understand OUD, providing new insights on the opioid epidemic. Further, these models provide a foundation for clinical tools to predict OUD before it occurs, permitting early interventions.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Electronic health records of patients who have been prescribed with medications containing active opioid ingredients were extracted from Cerner\u2019s Health Facts database for encounters between January 1, 2008, and December 31, 2017. Long short-term memory models were applied to predict OUD risk based on five recent prior encounters before the target encounter and compared with logistic regression, random forest, decision tree, and dense neural network. Prediction performance was assessed using F1 score, precision, recall, and area under the receiver-operating characteristic curve.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The long short-term memory (LSTM) model provided promising prediction results which outperformed other methods, with an F1 score of 0.8023 (about 0.016 higher than dense neural network (DNN)) and an area under the receiver-operating characteristic curve (AUROC) of 0.9369 (about 0.145 higher than DNN).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>LSTM\u2013based sequential deep learning models can accurately predict OUD using a patient\u2019s history of electronic health records, with minimal prior domain knowledge. This tool has the potential to improve clinical decision support for early intervention and prevention to combat the opioid epidemic.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocab043","type":"journal-article","created":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T20:14:45Z","timestamp":1614629685000},"page":"1683-1693","source":"Crossref","is-referenced-by-count":37,"title":["Identifying risk of opioid use disorder for patients taking opioid medications with deep learning"],"prefix":"10.1093","volume":"28","author":[{"given":"Xinyu","family":"Dong","sequence":"first","affiliation":[{"name":"Department of Computer Science, Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0647-1287","authenticated-orcid":false,"given":"Jianyuan","family":"Deng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1210-2939","authenticated-orcid":false,"given":"Sina","family":"Rashidian","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kayley","family":"Abell-Hart","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Hou","sequence":"additional","affiliation":[{"name":"Department of Family, Population and Preventive Medicine, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richard N","family":"Rosenthal","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mary","family":"Saltz","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joel H","family":"Saltz","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fusheng","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stony Brook University, Stony Brook, New York, USA"},{"name":"Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook University, Stony Brook, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,4,30]]},"reference":[{"key":"2021073020281679000_ocab043-B1","year":"2020"},{"key":"2021073020281679000_ocab043-B2","year":"2018"},{"key":"2021073020281679000_ocab043-B3","first-page":"29262202","volume-title":": StatPearls","author":"Schiller","year":"2020"},{"issue":"2","key":"2021073020281679000_ocab043-B4","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.jpainsymman.2004.01.003","article-title":"A reassessment of trends in the medical use and abuse of opioid analgesics and implications for diversion control: 1997\u20132002","volume":"28","author":"Gilson","year":"2004","journal-title":"J Pain Symptom Manage"},{"issue":"5","key":"2021073020281679000_ocab043-B5","doi-asserted-by":"crossref","first-page":"293","DOI":"10.7326\/M17-0865","article-title":"Prescription opioid use, misuse, and use disorders in U.S. adults: 2015 National Survey on Drug Use and Health","volume":"167","author":"Han","year":"2017","journal-title":"Ann Intern Med"},{"issue":"2","key":"2021073020281679000_ocab043-B6","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.drugalcdep.2005.05.009","article-title":"Major increases in opioid analgesic abuse in the United States: concerns and strategies","volume":"81","author":"Compton","year":"2006","journal-title":"Drug Alcohol Depend"},{"issue":"15","key":"2021073020281679000_ocab043-B7","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1001\/jama.2016.1464","article-title":"CDC guideline for prescribing opioids for chronic pain\u2014United States, 2016","volume":"315","author":"Dowell","year":"2016","journal-title":"JAMA"},{"key":"2021073020281679000_ocab043-B8","author":"Henry","year":"2016","journal-title":"Adoption of Electronic Health Record Systems among US Non-Federal Acute Care Hospitals: 2008-2015. ONC Data Brief 35"},{"issue":"1","key":"2021073020281679000_ocab043-B9","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1186\/s12913-015-1025-7","article-title":"A comparison of a multistate inpatient EHR database to the HCUP Nationwide Inpatient Sample","volume":"15","author":"DeShazo","year":"2015","journal-title":"BMC Health Serv Res"},{"issue":"9","key":"2021073020281679000_ocab043-B10","doi-asserted-by":"crossref","first-page":"E247","DOI":"10.1503\/cmaj.170958","article-title":"Management of opioid use disorders: a national clinical practice guideline","volume":"190","author":"Bruneau","year":"2018","journal-title":"CMAJ"},{"key":"2021073020281679000_ocab043-B11","first-page":"432","author":"Cheng","year":"2016"},{"issue":"6","key":"2021073020281679000_ocab043-B12","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1093\/bib\/bbx044","article-title":"Deep learning for healthcare: review, opportunities and challenges","volume":"19","author":"Miotto","year":"2018","journal-title":"Brief Bioinform"},{"issue":"5","key":"2021073020281679000_ocab043-B13","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","article-title":"Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis","volume":"22","author":"Shickel","year":"2017","journal-title":"IEEE J Biomed Health Inform"},{"issue":"3","key":"2021073020281679000_ocab043-B14","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1001\/jamainternmed.2018.7117","article-title":"Deep learning in medicine\u2014promise, progress, and challenges","volume":"179","author":"Wang","year":"2018","journal-title":"JAMA Intern Med"},{"issue":"10","key":"2021073020281679000_ocab043-B15","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1007\/s11606-017-4288-3","article-title":"Prediction model for two-year risk of opioid overdose among patients prescribed chronic opioid therapy","volume":"33","author":"Glanz","year":"2018","journal-title":"J Gen Intern Med"},{"issue":"1","key":"2021073020281679000_ocab043-B16","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s13040-019-0193-0","article-title":"Predicting opioid dependence from electronic health records with machine learning","volume":"12","author":"Ellis","year":"2019","journal-title":"BioData Min"},{"key":"2021073020281679000_ocab043-B17","doi-asserted-by":"crossref","first-page":"107839","DOI":"10.1016\/j.drugalcdep.2020.107839","article-title":"Understanding opioid use disorder (OUD) using tree-based classifiers","volume":"208","author":"Wadekar","year":"2020","journal-title":"Drug Alcohol Depend"},{"issue":"7","key":"2021073020281679000_ocab043-B18","doi-asserted-by":"crossref","first-page":"e0235981","DOI":"10.1371\/journal.pone.0235981","article-title":"Using machine learning to predict risk of incident opioid use disorder among fee-for-service Medicare beneficiaries: A prognostic study","volume":"15","author":"Lo-Ciganic","year":"2020","journal-title":"PLoS One"},{"issue":"3","key":"2021073020281679000_ocab043-B19","doi-asserted-by":"crossref","first-page":"e190968","DOI":"10.1001\/jamanetworkopen.2019.0968","article-title":"Evaluation of machine-learning algorithms for predicting opioid overdose risk among medicare beneficiaries with opioid prescriptions","volume":"2","author":"Lo-Ciganic","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"6","key":"2021073020281679000_ocab043-B20","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1007\/s11606-018-4335-8","article-title":"Prediction of future chronic opioid use among hospitalized patients","volume":"33","author":"Calcaterra","year":"2018","journal-title":"J Gen Intern Med"},{"issue":"1","key":"2021073020281679000_ocab043-B21","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"NPJ Digit Med"},{"issue":"S4","key":"2021073020281679000_ocab043-B22","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1186\/s12911-018-0677-8","article-title":"Improving palliative care with deep learning","volume":"18","author":"Avati","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021073020281679000_ocab043-B23","first-page":"525","article-title":"Deep learning solutions for classifying patients on opioid use","author":"Che","year":"2017","journal-title":"AMIA Annu Symp Proc"},{"key":"2021073020281679000_ocab043-B24","first-page":"440","article-title":"Deep EHR: Chronic disease prediction using medical notes","volume":"85","author":"Liu","year":"2018","journal-title":"Proc Mach Learn Res"},{"key":"2021073020281679000_ocab043-B25","author":"Rashidian","year":"2018"},{"key":"2021073020281679000_ocab043-B26","first-page":"620","article-title":"Deep learning on electronic health records to improve disease coding accuracy","volume":"2019","author":"Rashidian","year":"2019","journal-title":"AMIA Summits Transl Sci Proc"},{"key":"2021073020281679000_ocab043-B27","first-page":"389","article-title":"Machine learning based opioid overdose prediction using electronic health records","author":"Dong","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2021073020281679000_ocab043-B28","first-page":"A1","article-title":"Prescription drug use and misuse in the United States: Results from the 2015 National Survey on Drug Use and Health","author":"Hughes","year":"2016","journal-title":"NSDUH Data Review"},{"issue":"D1","key":"2021073020281679000_ocab043-B29","doi-asserted-by":"crossref","first-page":"D1074","DOI":"10.1093\/nar\/gkx1037","article-title":"DrugBank 5.0: a major update to the DrugBank database for 2018","volume":"46","author":"Wishart","year":"2018","journal-title":"Nucleic Acids Res"},{"key":"2021073020281679000_ocab043-B30","author":"Moore","year":"2017"},{"issue":"2","key":"2021073020281679000_ocab043-B31","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/0304-3959(86)90091-6","article-title":"Chronic use of opioid analgesics in non-malignant pain: report of 38 cases","volume":"25","author":"Portenoy","year":"1986","journal-title":"Pain"},{"key":"2021073020281679000_ocab043-B32","year":"2020"},{"key":"2021073020281679000_ocab043-B33","year":"2020"},{"key":"2021073020281679000_ocab043-B34"},{"issue":"2","key":"2021073020281679000_ocab043-B35","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/0020-7101(95)01135-2","article-title":"A new drug classification for computer systems: the ATC extension code","volume":"40","author":"Miller","year":"1995","journal-title":"Int J Biomed Comput"},{"key":"2021073020281679000_ocab043-B36"},{"key":"2021073020281679000_ocab043-B37"},{"issue":"1","key":"2021073020281679000_ocab043-B38","first-page":"85","article-title":"A multivariate technique for multiply imputing missing values using a sequence of regression models","volume":"27","author":"Raghunathan","year":"2001","journal-title":"Surv Methodol"},{"issue":"3","key":"2021073020281679000_ocab043-B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v045.i03","article-title":"mice: Multivariate imputation by chained equations in R","volume":"45","author":"Buuren","year":"2011","journal-title":"J Stat Soft"},{"issue":"8","key":"2021073020281679000_ocab043-B40","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"J Neural Comput"},{"key":"2021073020281679000_ocab043-B41","author":"Bahdanau","year":"2014"},{"issue":"11","key":"2021073020281679000_ocab043-B42","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans Signal Process"},{"key":"2021073020281679000_ocab043-B43","author":"Devlin","year":"2018"},{"key":"2021073020281679000_ocab043-B44","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2021073020281679000_ocab043-B45"},{"key":"2021073020281679000_ocab043-B46","first-page":"2825","volume":"12","author":"Pedregosa","year":"2011","journal-title":"Scikit-learn: machine learning in Python. J Mach Learn Res"},{"key":"2021073020281679000_ocab043-B47","first-page":"265","author":"Abadi"},{"key":"2021073020281679000_ocab043-B48","volume-title":"Deep Learning with Keras","author":"Gulli","year":"2017"},{"key":"2021073020281679000_ocab043-B49","volume-title":"SciPy and NumPy: An Overview for Developers","author":"Bressert","year":"2012"},{"issue":"9","key":"2021073020281679000_ocab043-B50","first-page":"1","article-title":"pandas: a foundational Python library for data analysis and statistics","volume":"14","author":"McKinney","year":"2011","journal-title":"Python for High Performance and Scientific Computing;"},{"issue":"3","key":"2021073020281679000_ocab043-B51","doi-asserted-by":"crossref","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"Saito","year":"2015","journal-title":"PLoS One"},{"issue":"5","key":"2021073020281679000_ocab043-B52","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1007\/s10979-005-6832-7","article-title":"Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r","volume":"29","author":"Rice","year":"2005","journal-title":"Law Hum Behav"},{"issue":"1","key":"2021073020281679000_ocab043-B53","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"2021073020281679000_ocab043-B54","author":"Fan"},{"key":"2021073020281679000_ocab043-B55","doi-asserted-by":"crossref","first-page":"106644","DOI":"10.1016\/j.addbeh.2020.106644","article-title":"Does anxiety sensitivity predict addiction severity in opioid use disorder?","volume":"112","author":"Stathopoulou","year":"2020","journal-title":"Addict Behav"},{"key":"2021073020281679000_ocab043-B56","first-page":"1903","author":"Ma","year":"2017"},{"issue":"1","key":"2021073020281679000_ocab043-B57","doi-asserted-by":"crossref","first-page":"7155","DOI":"10.1038\/s41598-020-62922-y","article-title":"BeHRt: transformer for electronic health records","volume":"10","author":"Li","year":"2020","journal-title":"Sci Rep"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1683\/39502344\/ocab043.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1683\/39502344\/ocab043.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,30]],"date-time":"2021-07-30T23:21:22Z","timestamp":1627687282000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/8\/1683\/6261158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,30]]},"references-count":57,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4,30]]},"published-print":{"date-parts":[[2021,7,30]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab043","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,8,1]]},"published":{"date-parts":[[2021,4,30]]}}}