{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T13:13:29Z","timestamp":1777036409257,"version":"3.51.4"},"reference-count":20,"publisher":"Oxford University Press (OUP)","issue":"1","funder":[{"name":"Million Veteran Program","award":["MVP011"],"award-info":[{"award-number":["MVP011"]}]},{"name":"Million Veteran Program, Office of Research and Development, Veterans Health Administration","award":["I01CX001729"],"award-info":[{"award-number":["I01CX001729"]}]},{"name":"VA Research Career Scientist","award":["lK6BX003777"],"award-info":[{"award-number":["lK6BX003777"]}]},{"name":"VA Clinical Sciences Research and Development"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,12,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To apply deep neural networks (DNNs) to longitudinal EHR data in order to predict suicide attempt risk among veterans. Local explainability techniques were used to provide explanations for each prediction with the goal of ultimately improving outreach and intervention efforts.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and methods<\/jats:title>\n                  <jats:p>The DNNs fused demographic information with diagnostic, prescription, and procedure codes. Models were trained and tested on EHR data of approximately 500\u00a0000 US veterans: all veterans with recorded suicide attempts from April 1, 2005, through January 1, 2016, each paired with 5 veterans of the same age who did not attempt suicide. Shapley Additive Explanation (SHAP) values were calculated to provide explanations of DNN predictions.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The DNNs outperformed logistic and linear regression models in predicting suicide attempts. After adjusting for the sampling technique, the convolutional neural network (CNN) model achieved a positive predictive value (PPV) of 0.54 for suicide attempts within 12 months by veterans in the top 0.1% risk tier. Explainability methods identified meaningful subgroups of high-risk veterans as well as key determinants of suicide attempt risk at both the group and individual level.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion and conclusion<\/jats:title>\n                  <jats:p>The deep learning methods employed in the present study have the potential to significantly enhance existing suicide risk models for veterans. These methods can also provide important clues to explore the relative value of long-term and short-term intervention strategies. Furthermore, the explainability methods utilized here could also be used to communicate to clinicians the key features which increase specific veterans\u2019 risk for attempting suicide.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad167","type":"journal-article","created":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T22:32:15Z","timestamp":1695940335000},"page":"220-230","source":"Crossref","is-referenced-by-count":15,"title":["Deep sequential neural network models improve stratification of suicide attempt risk among US veterans"],"prefix":"10.1093","volume":"31","author":[{"given":"Carianne","family":"Martinez","sequence":"first","affiliation":[{"name":"Sandia National Laboratories , Albuquerque, NM 87185, United States"},{"name":"Arizona State University , Tempe, AZ 85287, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6955-8315","authenticated-orcid":false,"given":"Drew","family":"Levin","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories , Albuquerque, NM 87185, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica","family":"Jones","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories , Albuquerque, NM 87185, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick D","family":"Finley","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories , Albuquerque, NM 87185, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"McMahon","sequence":"additional","affiliation":[{"name":"Los Alamos National Laboratory , Los Alamos, NM 87544, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sayera","family":"Dhaubhadel","sequence":"additional","affiliation":[{"name":"Los Alamos National Laboratory , Los Alamos, NM 87544, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Judith","family":"Cohn","sequence":"additional","affiliation":[{"name":"Los Alamos National Laboratory , Los Alamos, NM 87544, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"name":"Million Veteran Program","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"name":"MVP Suicide Exemplar Workgroup","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David W","family":"Oslin","sequence":"additional","affiliation":[{"name":"VISN 4 Mental Illness Research, Education, and Clinical Center, Center of Excellence, Corporal Michael J. Crescenz VA Medical Center , Philadelphia, PA 19104, United States"},{"name":"Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA 19104, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nathan A","family":"Kimbrel","sequence":"additional","affiliation":[{"name":"Durham Veterans Affairs (VA) Health Care System , Durham, NC 27705, United States"},{"name":"Education and Clinical Center, VA Mid-Atlantic Mental Illness Research , Durham, NC 27707, United States"},{"name":"VA Health Services Research and Development Center of Innovation to Accelerate Discovery and Practice Transformation , Durham, NC 27701, United States"},{"name":"Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine , Durham, NC 27701, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean C","family":"Beckham","sequence":"additional","affiliation":[{"name":"Durham Veterans Affairs (VA) Health Care System , Durham, NC 27705, United States"},{"name":"Education and Clinical Center, VA Mid-Atlantic Mental Illness Research , Durham, NC 27707, United States"},{"name":"Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine , Durham, NC 27701, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,9,28]]},"reference":[{"issue":"10","key":"2023122220312423900_ocad167-B1","doi-asserted-by":"crossref","first-page":"e2129900","DOI":"10.1001\/jamanetworkopen.2021.29900","article-title":"Evaluation of the recovery engagement and coordination for health-veterans enhanced treatment suicide risk modeling clinical program in the veterans health administration","volume":"4","author":"McCarthy","year":"2021","journal-title":"JAMA Netw Open"},{"issue":"1","key":"2023122220312423900_ocad167-B2","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1176\/appi.ps.201800242","article-title":"Integrating predictive modeling into mental health care: an example in suicide prevention","volume":"70","author":"Reger","year":"2019","journal-title":"Psychiatr Serv"},{"key":"2023122220312423900_ocad167-B3","doi-asserted-by":"crossref","first-page":"707916","DOI":"10.3389\/fpsyt.2021.707916","article-title":"Applying machine learning approaches to suicide prediction using healthcare data: overview and future directions","volume":"12","author":"Boudreaux","year":"2021","journal-title":"Front Psychiatry"},{"issue":"3","key":"2023122220312423900_ocad167-B4","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int J Comput Vis"},{"key":"2023122220312423900_ocad167-B5","article-title":"Google\u2019s neural machine translation system: bridging the gap between human and machine translation","author":"Wu","year":"2016","journal-title":"CoRR"},{"issue":"1","key":"2023122220312423900_ocad167-B6","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1038\/s41398-020-0684-2","article-title":"Development of an early-warning system for high-risk patients for suicide attempt using deep learning and electronic health records","volume":"10","author":"Zheng","year":"2020","journal-title":"Transl Psychiatry"},{"key":"2023122220312423900_ocad167-B7","doi-asserted-by":"crossref","first-page":"390","DOI":"10.3389\/fpsyt.2020.00390","article-title":"Using administrative data to predict suicide after psychiatric hospitalization in the veterans health administration system","volume":"11","author":"Kessler","year":"2020","journal-title":"Front Psychiatry"},{"key":"2023122220312423900_ocad167-B8","author":"Alterovitz"},{"issue":"8","key":"2023122220312423900_ocad167-B9","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2023122220312423900_ocad167-B10","first-page":"8026","article-title":"Pytorch: an imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023122220312423900_ocad167-B11","first-page":"4768","author":"Lundberg","year":"2017"},{"key":"2023122220312423900_ocad167-B12","first-page":"3145","author":"Shrikumar","year":"2017"},{"key":"2023122220312423900_ocad167-B13","first-page":"1309.0238","author":"Buitinck","year":"2013"},{"issue":"9","key":"2023122220312423900_ocad167-B14","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.2105\/AJPH.2015.302737","article-title":"Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US department of veterans affairs","volume":"105","author":"McCarthy","year":"2015","journal-title":"Am J Public Health"},{"key":"2023122220312423900_ocad167-B15","author":"Office of Mental Health and Suicide Prevention","year":"2021"},{"issue":"Suppl 1","key":"2023122220312423900_ocad167-B16","doi-asserted-by":"crossref","first-page":"S131","DOI":"10.2105\/AJPH.2011.300445","article-title":"Suicide risk and precipitating circumstances among young, middle-aged, and older male veterans","volume":"102","author":"Kaplan","year":"2012","journal-title":"Am J Public Health"},{"key":"2023122220312423900_ocad167-B17","article-title":"Suicide attempts among racial and ethnic groups in a nationally representative sample","author":"Carter","year":"2021","journal-title":"J Racial Ethn Health Disparities"},{"issue":"11","key":"2023122220312423900_ocad167-B18","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1001\/archgenpsychiatry.2010.129","article-title":"Psychiatric diagnoses and risk of suicide in veterans","volume":"67","author":"Ilgen","year":"2010","journal-title":"Arch Gen Psychiatry"},{"key":"2023122220312423900_ocad167-B19","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.jpsychires.2022.04.009","article-title":"A case for developing domain-specific vocabularies for extracting suicide factors from healthcare notes","volume":"151","author":"Morrow","year":"2022","journal-title":"J Psychiatr Res"},{"issue":"6","key":"2023122220312423900_ocad167-B20","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1177\/0033354916670133","article-title":"VA suicide prevention applications network: a national health care system-based suicide event tracking system","volume":"131","author":"Hoffmire","year":"2016","journal-title":"Public Health Rep"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/1\/220\/54762268\/ocad167.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/1\/220\/54762268\/ocad167.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T20:32:16Z","timestamp":1703277136000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/1\/220\/7285660"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":20,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,9,28]]},"published-print":{"date-parts":[[2023,12,22]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad167","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,1,1]]},"published":{"date-parts":[[2023,9,28]]}}}