{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T20:39:46Z","timestamp":1772138386588,"version":"3.50.1"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T00:00:00Z","timestamp":1635811200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["R01MH117599"],"award-info":[{"award-number":["R01MH117599"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,12,28]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Suicide is one of the leading causes of death worldwide, yet clinicians find it difficult to reliably identify individuals at high risk for suicide. Algorithmic approaches for suicide risk detection have been developed in recent years, mostly based on data from electronic health records (EHRs). Significant room for improvement remains in the way these models take advantage of temporal information to improve predictions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We propose a temporally enhanced variant of the random forest (RF) model\u2014Omni-Temporal Balanced Random Forests (OT-BRFs)\u2014that incorporates temporal information in every tree within the forest. We develop and validate this model using longitudinal EHRs and clinician notes from the Mass General Brigham Health System recorded between 1998 and 2018, and compare its performance to a baseline Naive Bayes Classifier and 2 standard versions of balanced RFs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Temporal variables were found to be associated with suicide risk: Elevated suicide risk was observed in individuals with a higher total number of visits as well as those with a low rate of visits over time, while lower suicide risk was observed in individuals with a longer period of EHR coverage. RF models were more accurate than Naive Bayesian classifiers at predicting suicide risk in advance (area under the receiver operating curve\u2009=\u20090.824 vs. 0.754, respectively). The proposed OT-BRF model performed best among all RF approaches, yielding a sensitivity of 0.339 at 95% specificity, compared to 0.290 and 0.286 for the other 2 RF models. Temporal variables were assigned high importance by the models that incorporated them.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>We demonstrate that temporal variables have an important role to play in suicide risk detection and that requiring their inclusion in all RF trees leads to increased predictive performance. Integrating temporal information into risk prediction models helps the models interpret patient data in temporal context, improving predictive performance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocab225","type":"journal-article","created":{"date-parts":[[2021,10,5]],"date-time":"2021-10-05T19:59:55Z","timestamp":1633463995000},"page":"62-71","source":"Crossref","is-referenced-by-count":26,"title":["Temporally informed random forests for suicide risk prediction"],"prefix":"10.1093","volume":"29","author":[{"given":"Ilkin","family":"Bayramli","sequence":"first","affiliation":[{"name":"Predictive Medicine Group, Computational Health Informatics Program, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Harvard University, Cambridge, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victor","family":"Castro","sequence":"additional","affiliation":[{"name":"Mass General Brigham Research Information Science and Computing, Boston, Massachusetts, USA"},{"name":"Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuval","family":"Barak-Corren","sequence":"additional","affiliation":[{"name":"Predictive Medicine Group, Computational Health Informatics Program, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emily M","family":"Madsen","sequence":"additional","affiliation":[{"name":"Psychiatric and Neurodevelopmental Genetics Unit, Center for Genomic Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Department of Psychiatry, Center for Precision Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew K","family":"Nock","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Department of Psychology, Harvard University, Cambridge, Massachusetts, USA"},{"name":"Mental Health Research Program, Franciscan Children\u2019s, Brighton, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jordan W","family":"Smoller","sequence":"additional","affiliation":[{"name":"Psychiatric and Neurodevelopmental Genetics Unit, Center for Genomic Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Department of Psychiatry, Center for Precision Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9908-5523","authenticated-orcid":false,"given":"Ben Y","family":"Reis","sequence":"additional","affiliation":[{"name":"Predictive Medicine Group, Computational Health Informatics Program, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,11,2]]},"reference":[{"issue":"3","key":"2021122823072126200_ocab225-B1","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1056\/NEJMra1902944","article-title":"Suicide","volume":"382","author":"Fazel","year":"2020","journal-title":"N Engl J Med"},{"key":"2021122823072126200_ocab225-B2","doi-asserted-by":"crossref","first-page":"l94","DOI":"10.1136\/bmj.l94","article-title":"Global, regional, and national burden of suicide mortality 1990 to 2016: systematic analysis for the Global Burden of Disease Study 2016","volume":"364","author":"Naghavi","year":"2019","journal-title":"BMJ"},{"key":"2021122823072126200_ocab225-B3","first-page":"1","article-title":"Suicide rates in the United States continue to increase","author":"Hedegaard","year":"2018","journal-title":"NCHS Data Brief"},{"issue":"4","key":"2021122823072126200_ocab225-B4","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1177\/0956797610364762","article-title":"Measuring the suicidal mind: implicit cognition predicts suicidal behavior","volume":"21","author":"Nock","year":"2010","journal-title":"Psychol Sci"},{"issue":"2","key":"2021122823072126200_ocab225-B5","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1176\/appi.ajp.2016.16010077","article-title":"Predicting suicidal behavior from longitudinal electronic health records","volume":"174","author":"Barak-Corren","year":"2017","journal-title":"Am J Psychiatry"},{"issue":"10","key":"2021122823072126200_ocab225-B6","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1176\/appi.ajp.2018.17101167","article-title":"Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records","volume":"175","author":"Simon","year":"2018","journal-title":"Am J Psychiatry"},{"issue":"12","key":"2021122823072126200_ocab225-B7","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.1111\/jcpp.12916","article-title":"Predicting suicide attempts in adolescents with longitudinal clinical data and machine learning","volume":"59","author":"Walsh","year":"2018","journal-title":"J Child Psychol Psychiatry"},{"issue":"3","key":"2021122823072126200_ocab225-B8","doi-asserted-by":"crossref","first-page":"e201262","DOI":"10.1001\/jamanetworkopen.2020.1262","article-title":"Validation of an electronic health record-based suicide risk prediction modeling approach across multiple health care systems","volume":"3","author":"Barak-Corren","year":"2020","journal-title":"JAMA Netw Open"},{"key":"2021122823072126200_ocab225-B9","first-page":"1558","article-title":"Explaining the success of adaboost and random forests as interpolating classifiers","volume":"18","author":"Wyner","year":"2017","journal-title":"J Mach Learn Res"},{"issue":"1","key":"2021122823072126200_ocab225-B10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12874-019-0863-0","article-title":"Clinical risk prediction with random forests for survival, longitudinal, and multivariate (RF-SLAM) data analysis","volume":"20","author":"Wongvibulsin","year":"2019","journal-title":"BMC Med Res Methodol"},{"key":"2021122823072126200_ocab225-B11","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1186\/1472-6947-11-51","article-title":"Predicting disease risks from highly imbalanced data using random forest","volume":"11","author":"Khalilia","year":"2011","journal-title":"BMC Med Inform Decis Mak"},{"issue":"12","key":"2021122823072126200_ocab225-B12","doi-asserted-by":"crossref","first-page":"3283","DOI":"10.3390\/en11123283","article-title":"Hybrid short-term load forecasting scheme using random forest and multilayer perceptron","volume":"11","author":"Moon","year":"2018","journal-title":"Energies"},{"key":"2021122823072126200_ocab225-B13","article-title":"Calculating the benefits of a research patient data repository","volume":"1044","author":"Nalichowski","year":"2006","journal-title":"AMIA Annu Symp Proc"},{"issue":"Database issue","key":"2021122823072126200_ocab225-B14","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2021122823072126200_ocab225-B15","author":"Ross","year":"2018"},{"issue":"12","key":"2021122823072126200_ocab225-B16","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1016\/j.biopsych.2018.01.011","article-title":"High throughput phenotyping for dimensional psychopathology in electronic health records","volume":"83","author":"McCoy","year":"2018","journal-title":"Biol Psychiatry"},{"key":"2021122823072126200_ocab225-B17","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1186\/1472-6947-6-30","article-title":"Extracting principal diagnosis, co-morbidity and smoking status for asthma research: evaluation of a natural language processing system","volume":"6","author":"Zeng","year":"2006","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021122823072126200_ocab225-B18","first-page":"81","author":"Chapman","year":"2007"},{"key":"2021122823072126200_ocab225-B19","doi-asserted-by":"crossref","first-page":"b3677","DOI":"10.1136\/bmj.b3677","article-title":"Longitudinal histories as predictors of future diagnoses of domestic abuse: modelling study","volume":"339","author":"Reis","year":"2009","journal-title":"BMJ"},{"key":"2021122823072126200_ocab225-B20","volume":"1\u201312","author":"Chao","year":"2004","journal-title":"Using Random Forest to Learn Imbalanced Data."},{"issue":"1","key":"2021122823072126200_ocab225-B21","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"2021122823072126200_ocab225-B22","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1471-2105-8-25","article-title":"Bias in random forest variable importance measures: illustrations, sources and a solution","volume":"8","author":"Strobl","year":"2007","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"2021122823072126200_ocab225-B23","doi-asserted-by":"crossref","first-page":"ooab011","DOI":"10.1093\/jamiaopen\/ooab011","article-title":"Natural language processing and machine learning of electronic health records for prediction of first-time suicide attempts","volume":"4","author":"Tsui","year":"2021","journal-title":"JAMIA Open"},{"key":"2021122823072126200_ocab225-B24","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-319-46681-1_1","volume-title":"Neural Information Processing","author":"Ooi","year":"2016"},{"key":"2021122823072126200_ocab225-B25","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.jag.2014.08.001","article-title":"Temporal optimisation of image acquisition for land cover classification with random forest and MODIS time-series","volume":"34","author":"Nitze","year":"2015","journal-title":"Int J Appl Earth Obs Geoinf"},{"issue":"9\u201310","key":"2021122823072126200_ocab225-B26","doi-asserted-by":"crossref","first-page":"1469","DOI":"10.1007\/s10994-017-5642-8","article-title":"Adaptive random forests for evolving data stream classification","volume":"106","author":"Gomes","year":"2017","journal-title":"Mach Learn"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/1\/62\/41955545\/ocab225.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/1\/62\/41955545\/ocab225.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,28]],"date-time":"2021-12-28T18:11:14Z","timestamp":1640715074000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/1\/62\/6415403"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,2]]},"references-count":26,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,11,2]]},"published-print":{"date-parts":[[2021,12,28]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab225","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.06.01.21258179","asserted-by":"object"}]},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,1,1]]},"published":{"date-parts":[[2021,11,2]]}}}