{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T15:22:45Z","timestamp":1777130565970,"version":"3.51.4"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,5,26]],"date-time":"2020-05-26T00:00:00Z","timestamp":1590451200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,5,26]],"date-time":"2020-05-26T00:00:00Z","timestamp":1590451200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning analysis of social media data represents a promising way to capture longitudinal environmental influences contributing to individual risk for suicidal thoughts and behaviors. Our objective was to generate an algorithm termed \u201cSuicide Artificial Intelligence Prediction Heuristic (SAIPH)\u201d capable of predicting future risk to suicidal thought by analyzing publicly available Twitter data. We trained a series of neural networks on Twitter data queried against suicide associated psychological constructs including burden, stress, loneliness, hopelessness, insomnia, depression, and anxiety. Using 512,526 tweets from <jats:italic>N<\/jats:italic>\u2009=\u2009283 suicidal ideation (SI) cases and 3,518,494 tweets from 2655 controls, we then trained a random forest model using neural network outputs to predict binary SI status. The model predicted <jats:italic>N<\/jats:italic>\u2009=\u2009830 SI events derived from an independent set of 277 suicidal ideators relative to <jats:italic>N<\/jats:italic>\u2009=\u20093159 control events in all non-SI individuals with an AUC of 0.88 (95% CI 0.86\u20130.90). Using an alternative approach, our model generates temporal prediction of risk such that peak occurrences above an individual specific threshold denote a ~7 fold increased risk for SI within the following 10 days (OR\u2009=\u20096.7\u2009\u00b1\u20091.1, <jats:italic>P<\/jats:italic>\u2009=\u20099\u2009\u00d7\u200910<jats:sup>\u221271<\/jats:sup>). We validated our model using regionally obtained Twitter data and observed significant associations of algorithm SI scores with county-wide suicide death rates across 16 days in August and in October, 2019, most significantly in younger individuals. Algorithmic approaches like SAIPH have the potential to identify individual future SI risk and could be easily adapted as clinical decision tools aiding suicide screening and risk monitoring using available technologies.<\/jats:p>","DOI":"10.1038\/s41746-020-0287-6","type":"journal-article","created":{"date-parts":[[2020,5,26]],"date-time":"2020-05-26T10:02:38Z","timestamp":1590487358000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":138,"title":["A machine learning approach predicts future risk to suicidal ideation from social media data"],"prefix":"10.1038","volume":"3","author":[{"given":"Arunima","family":"Roy","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2034-0518","authenticated-orcid":false,"given":"Katerina","family":"Nikolitch","sequence":"additional","affiliation":[]},{"given":"Rachel","family":"McGinn","sequence":"additional","affiliation":[]},{"given":"Safiya","family":"Jinah","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9616-8662","authenticated-orcid":false,"given":"William","family":"Klement","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7917-1301","authenticated-orcid":false,"given":"Zachary A.","family":"Kaminsky","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,5,26]]},"reference":[{"key":"287_CR1","unstructured":"Centers for Disease Control and Prevention. (2013) WISQARS Database. Retrieved from: https:\/\/www.cdc.gov\/injury\/wisqars\/index.html."},{"key":"287_CR2","unstructured":"National Institutes of Mental Health (2019), Conducting research with participants at elevated risk for suicide: considerations for researchers. Retrieved from: https:\/\/www.nimh.nih.gov\/funding\/clinical-research\/conducting-research-with-participants-at-elevated-risk-for-suicide-considerations-for-researchers.shtml."},{"key":"287_CR3","doi-asserted-by":"publisher","first-page":"258","DOI":"10.2174\/2210676611202030258","volume":"2","author":"EL Belfor","year":"2012","unstructured":"Belfor, E. L., Mezzacappa, E. & Ginnis, K. Similarities and differences among adolescents who communicate suicidality to others via electronic versus other means: a pilot study. Adolesc. Psychiatry 2, 258\u2013262 (2012).","journal-title":"Adolesc. Psychiatry"},{"key":"287_CR4","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0181722","volume":"12","author":"A Marchant","year":"2017","unstructured":"Marchant, A. et al. A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: the good, the bad and the unknown. PLoS ONE 12, e0181722 (2017).","journal-title":"PLoS ONE"},{"key":"287_CR5","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1027\/0227-5910\/a000086","volume":"32","author":"TD Ruder","year":"2011","unstructured":"Ruder, T. D., Hatch, G. M., Ampanozi, G., Thali, M. J. & Fischer, N. Suicide announcement on Facebook. Crisis 32, 280\u2013282 (2011).","journal-title":"Crisis"},{"key":"287_CR6","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jad.2014.08.047","volume":"170","author":"H Sueki","year":"2015","unstructured":"Sueki, H. The association of suicide-related Twitter use with suicidal behaviour: a cross-sectional study of young internet users in Japan. J. Affect. Disord. 170, 155\u2013160 (2015).","journal-title":"J. Affect. Disord."},{"key":"287_CR7","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1007\/s11126-017-9535-6","volume":"89","author":"C Berryman","year":"2018","unstructured":"Berryman, C., Ferguson, C. J. & Negy, C. Social media use and mental health among young adults. Psychiatr. Q. 89, 307\u2013314 (2018).","journal-title":"Psychiatr. Q."},{"key":"287_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.psychres.2019.05.024","volume":"278","author":"AL Calear","year":"2019","unstructured":"Calear, A. L. & Batterham, P. J. Suicidal ideation disclosure: Patterns, correlates and outcome. Psychiatry Res. 278, 1\u20136 (2019).","journal-title":"Psychiatry Res."},{"key":"287_CR9","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1046\/j.1525-1497.1997.00070.x","volume":"12","author":"DS Brody","year":"1997","unstructured":"Brody, D. S., Khaliq, A. A. & Thompson, T. L. 2nd Patients\u2019 perspectives on the management of emotional distress in primary care settings. J. Gen. Intern. Med. 12, 403\u2013406 (1997).","journal-title":"J. Gen. Intern. Med."},{"key":"287_CR10","first-page":"175","volume":"32","author":"KD Bertakis","year":"1991","unstructured":"Bertakis, K. D., Roter, D. & Putnam, S. M. The relationship of physician medical interview style to patient satisfaction. J. Fam. Pract. 32, 175\u2013181 (1991).","journal-title":"J. Fam. Pract."},{"key":"287_CR11","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1176\/appi.ajp.159.6.909","volume":"159","author":"JB Luoma","year":"2002","unstructured":"Luoma, J. B., Martin, C. E. & Pearson, J. L. Contact with mental health and primary care providers before suicide: a review of the evidence. Am. J. Psychiatry 159, 909\u2013916 (2002).","journal-title":"Am. J. Psychiatry"},{"key":"287_CR12","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1016\/S0277-9536(98)00439-0","volume":"48","author":"JW Robinson","year":"1999","unstructured":"Robinson, J. W. & Roter, D. L. Psychosocial problem disclosure by primary care patients. Soc. Sci. Med. 48, 1353\u20131362 (1999).","journal-title":"Soc. Sci. Med."},{"key":"287_CR13","doi-asserted-by":"publisher","first-page":"870","DOI":"10.1007\/s11606-014-2767-3","volume":"29","author":"BK Ahmedani","year":"2014","unstructured":"Ahmedani, B. K. et al. Health care contacts in the year before suicide death. J. Gen. Intern. Med. 29, 870\u2013877 (2014).","journal-title":"J. Gen. Intern. Med."},{"key":"287_CR14","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1001\/archpedi.154.2.162","volume":"154","author":"DL Frankenfield","year":"2000","unstructured":"Frankenfield, D. L. et al. Adolescent patients\u2013healthy or hurting? Missed opportunities to screen for suicide risk in the primary care setting. Arch. Pediatr. Adolesc. Med. 154, 162\u2013168 (2000).","journal-title":"Arch. Pediatr. Adolesc. Med."},{"key":"287_CR15","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/BF03405146","volume":"95","author":"L Farand","year":"2004","unstructured":"Farand, L., Renaud, J. & Chagnon, F. Adolescent suicide in Quebec and prior utilization of medical services. Can. J. Public Health 95, 357\u2013360 (2004).","journal-title":"Can. J. Public Health"},{"key":"287_CR16","unstructured":"The National Action Alliance for Suicide Prevention, R.P.T.F. (2014). Suicide Research Prioritization Plan of Action. Retrieved from: https:\/\/theactionalliance.org\/resource\/prioritized-research-agenda-suicide-prevention-action-plan-save-lives."},{"key":"287_CR17","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1027\/0227-5910\/a000443","volume":"38","author":"B O\u2019Dea","year":"2017","unstructured":"O\u2019Dea, B., Larsen, M. E., Batterham, P. J., Calear, A. L. & Christensen, H. A linguistic analysis of Suicide-related twitter posts. Crisis 38, 319\u2013329 (2017).","journal-title":"Crisis"},{"key":"287_CR18","unstructured":"Statista. (2018) Percentage of internet users accessing selected social media platforms in Canada as of May 2018, by age group. Retrieved from: https:\/\/www.statista.com\/statistics\/468476\/selected-social-media-user-age-share-canada\/#statisticContainer."},{"key":"287_CR19","unstructured":"Statistics Canada. (2019) Population estimates on July 1st, by age and sex. Retrieved from: https:\/\/www150.statcan.gc.ca\/t1\/tbl1\/en\/tv.action?pid=1710000501."},{"key":"287_CR20","unstructured":"Public Health Agency of Canada. (2019) Suicide in Canada: Key statistics. Retrieved from: https:\/\/www.canada.ca\/en\/public-health\/services\/publications\/healthy-living\/suicide-canada-key-statistics-infographic.html."},{"key":"287_CR21","doi-asserted-by":"publisher","first-page":"880","DOI":"10.1089\/tmj.2018.0203","volume":"25","author":"A Pourmand","year":"2018","unstructured":"Pourmand, A. et al. Social media and suicide: a review of technology-based epidemiology and risk assessment. Telemed. e-Health 25, 880\u2013888 (2018).","journal-title":"Telemed. e-Health"},{"key":"287_CR22","doi-asserted-by":"publisher","DOI":"10.2196\/11483","volume":"5","author":"Y Hswen","year":"2018","unstructured":"Hswen, Y., Naslund, J. A., Brownstein, J. S. & Hawkins, J. B. Monitoring online discussions about suicide among twitter users with schizophrenia: exploratory study. JMIR Ment. Health 5, e11483 (2018).","journal-title":"JMIR Ment. Health"},{"key":"287_CR23","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.puhe.2018.03.001","volume":"159","author":"SY Lee","year":"2018","unstructured":"Lee, S. Y. & Kwon, Y. Twitter as a place where people meet to make suicide pacts. Public Health 159, 21\u201326 (2018).","journal-title":"Public Health"},{"key":"287_CR24","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.9582","volume":"20","author":"MA Alvarez-Mon","year":"2018","unstructured":"Alvarez-Mon, M. A. et al. Increasing interest of mass communication media and the general public in the distribution of tweets about mental disorders: observational study. J. Med. Internet Res. 20, e205 (2018).","journal-title":"J. Med. Internet Res."},{"key":"287_CR25","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-018-0926-5","volume":"42","author":"MA Franco-Martin","year":"2018","unstructured":"Franco-Martin, M. A. et al. A systematic literature review of technologies for suicidal behavior prevention. J. Med. Syst. 42, 71 (2018).","journal-title":"J. Med. Syst."},{"key":"287_CR26","doi-asserted-by":"publisher","DOI":"10.2196\/mental.4822","volume":"3","author":"SR Braithwaite","year":"2016","unstructured":"Braithwaite, S. R., Giraud-Carrier, C., West, J., Barnes, M. D. & Hanson, C. L. Validating machine learning algorithms for twitter data against established measures of suicidality. JMIR Ment. Health 3, e21 (2016).","journal-title":"JMIR Ment. Health"},{"key":"287_CR27","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-018-0632-8","volume":"18","author":"J Du","year":"2018","unstructured":"Du, J. et al. Extracting psychiatric stressors for suicide from social media using deep learning. BMC Med. Inf. Decis. Mak. 18, 43 (2018).","journal-title":"BMC Med. Inf. Decis. Mak."},{"key":"287_CR28","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/j.osnem.2017.08.001","volume":"2","author":"P Burnap","year":"2017","unstructured":"Burnap, P., Colombo, G., Amery, R., Hodorog, A. & Scourfield, J. Multi-class machine classification of suicide-related communication on twitter. Online Soc. Netw. Media 2, 32\u201344 (2017).","journal-title":"Online Soc. Netw. Media"},{"key":"287_CR29","unstructured":"Luo, J., Du, J., Tao, C., Xu, H. & Zhang, Y. Exploring temporal suicidal behavior patterns on social media: insight from twitter analytics. Health Inform. J. 1460458219832043 (2019). https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30866708."},{"key":"287_CR30","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1037\/a0018697","volume":"117","author":"KA Van Orden","year":"2010","unstructured":"Van Orden, K. A. et al. The interpersonal theory of suicide. Psychol. Rev. 117, 575\u2013600 (2010).","journal-title":"Psychol. Rev."},{"key":"287_CR31","first-page":"345","volume":"22","author":"RT Liu","year":"2015","unstructured":"Liu, R. T., Kleiman, E. M., Nestor, B. A. & Cheek, S. M. The hopelessness theory of depression: a quarter century in review. Clin. Psychol. 22, 345\u2013365 (2015).","journal-title":"Clin. Psychol."},{"key":"287_CR32","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1016\/j.comppsych.2013.10.015","volume":"55","author":"EM Kleiman","year":"2014","unstructured":"Kleiman, E. M., Law, K. C. & Anestis, M. D. Do theories of suicide play well together? Integrating components of the hopelessness and interpersonal psychological theories of suicide. Compr. Psychiatry 55, 431\u2013438 (2014).","journal-title":"Compr. Psychiatry"},{"key":"287_CR33","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1192\/bjp.bp.107.040113","volume":"192","author":"MK Nock","year":"2008","unstructured":"Nock, M. K. et al. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br. J. Psychiatry 192, 98\u2013105 (2008).","journal-title":"Br. J. Psychiatry"},{"key":"287_CR34","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1097\/YCO.0000000000000294","volume":"30","author":"ED Klonsky","year":"2017","unstructured":"Klonsky, E. D., Qiu, T. & Saffer, B. Y. Recent advances in differentiating suicide attempters from suicide ideators. Curr. Opin. Psychiatry 30, 15\u201320 (2017).","journal-title":"Curr. Opin. Psychiatry"},{"key":"287_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/088342306000000060","volume":"21","author":"H DJ","year":"2006","unstructured":"DJ, H. Classifier technology and the illusion of progress. Stat. Sci. 21, 1\u201315 (2006).","journal-title":"Stat. Sci."},{"key":"287_CR36","doi-asserted-by":"crossref","unstructured":"Jiangn Y. et al. Expert feature-engineering vs. Deep neural networks: Which is better for sensor-free affect detection? In Ros\u00e9 C. P. et al. (eds), Artificial Intelligence in Educaltion 198\u2013211 (Springer, Cham, 2018).","DOI":"10.1007\/978-3-319-93843-1_15"},{"key":"287_CR37","doi-asserted-by":"crossref","unstructured":"Nguyen, V. Q., Anh, T. N., Yang, H. J. Real-time event detection using recurrent neural network in social sensors. Int. J. Distrib. Sens. Netw. 15, 1550147719856492 (2019).","DOI":"10.1177\/1550147719856492"},{"key":"287_CR38","doi-asserted-by":"publisher","first-page":"14","DOI":"10.4088\/JCP.v64n0105","volume":"64","author":"KA Busch","year":"2003","unstructured":"Busch, K. A., Fawcett, J. & Jacobs, D. G. Clinical correlates of inpatient suicide. J. Clin. Psychiatry 64, 14\u201319 (2003).","journal-title":"J. Clin. Psychiatry"},{"key":"287_CR39","doi-asserted-by":"publisher","first-page":"919","DOI":"10.1176\/ajp.152.6.919","volume":"152","author":"ET Isometsa","year":"1995","unstructured":"Isometsa, E. T. et al. The last appointment before suicide: is suicide intent communicated? Am. J. Psychiatry 152, 919\u2013922 (1995).","journal-title":"Am. J. Psychiatry"},{"key":"287_CR40","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0061809","volume":"8","author":"HH Won","year":"2013","unstructured":"Won, H. H. et al. Predicting national suicide numbers with social media data. PLoS ONE 8, e61809 (2013).","journal-title":"PLoS ONE"},{"key":"287_CR41","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.socscimed.2018.10.004","volume":"219","author":"RA Fahey","year":"2018","unstructured":"Fahey, R. A., Matsubayashi, T. & Ueda, M. Tracking the Werther effect on social media: emotional responses to prominent suicide deaths on twitter and subsequent increases in suicide. Soc. Sci. Med. 219, 19\u201329 (2018).","journal-title":"Soc. Sci. Med."},{"key":"287_CR42","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1027\/0227-5910\/a000234","volume":"35","author":"J Jashinsky","year":"2014","unstructured":"Jashinsky, J. et al. Tracking suicide risk factors through twitter in the US. Crisis 35, 51\u201359 (2014).","journal-title":"Crisis"},{"key":"287_CR43","doi-asserted-by":"publisher","first-page":"496","DOI":"10.1093\/jamia\/ocw133","volume":"24","author":"C McClellan","year":"2017","unstructured":"McClellan, C., Ali, M. M., Mutter, R., Kroutil, L. & Landwehr, J. Using social media to monitor mental health discussions\u2014evidence from twitter. J. Am. Med. Inf. Assoc. 24, 496\u2013502 (2017).","journal-title":"J. Am. Med. Inf. Assoc."},{"key":"287_CR44","doi-asserted-by":"crossref","unstructured":"Spates, K., Ye, X. & Johnson, A. \u201cI just might kill myself\u201d: Suicide expressions on twitter. Death Stud. 44, 189\u2013194 (2020).","DOI":"10.1080\/07481187.2018.1531085"},{"key":"287_CR45","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/j.comcom.2015.07.018","volume":"73","author":"GB Colombo","year":"2016","unstructured":"Colombo, G. B., Burnap, P., Hodorog, A. & Scourfield, J. Analysing the connectivity and communication of suicidal users on twitter. Comput. Commun. 73, 291\u2013300 (2016).","journal-title":"Comput. Commun."},{"key":"287_CR46","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1016\/j.jad.2017.04.006","volume":"217","author":"E Kohls","year":"2017","unstructured":"Kohls, E. et al. Public attitudes toward depression and help-seeking: impact of the OSPI-Europe depression awareness campaign in four European regions. J. Affect. Disord. 217, 252\u2013259 (2017).","journal-title":"J. Affect. Disord."},{"key":"287_CR47","doi-asserted-by":"crossref","unstructured":"Zijian Wang, S. A. H. et al. Demographic Inference and Representative Population Estimates from Multilingual Social Media Data, (AMC, 2019).","DOI":"10.1145\/3308558.3313684"},{"key":"287_CR48","doi-asserted-by":"publisher","first-page":"385","DOI":"10.2307\/2136404","volume":"24","author":"S Cohen","year":"1983","unstructured":"Cohen, S., Kamarck, T. & Mermelstein, R. A global measure of perceived stress. J. Health Soc. Behav. 24, 385\u2013396 (1983).","journal-title":"J. Health Soc. Behav."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0287-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0287-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0287-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:15:44Z","timestamp":1670379344000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0287-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,26]]},"references-count":48,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["287"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-0287-6","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,26]]},"assertion":[{"value":"12 December 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors have entered into a research partnership with TryCycle Data Systems Inc. on November 14, 2019 to evaluate the efficacy of the above algorithm at identifying SI risk in a clinical setting. This evaluation and the relationship are independent from the work as reported in this journal article. In addition, the authors declare that they do not have any other relationship with the company, and that no financial transactions are involved in this partnership.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"78"}}