{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:36:46Z","timestamp":1783438606111,"version":"3.54.6"},"reference-count":74,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,2]],"date-time":"2021-02-02T00:00:00Z","timestamp":1612224000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,2]],"date-time":"2021-02-02T00:00:00Z","timestamp":1612224000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012385","name":"Georgetown-Howard Universities Center for Clinical and Translational Science","doi-asserted-by":"publisher","award":["UL1-TR001409"],"award-info":[{"award-number":["UL1-TR001409"]}],"id":[{"id":"10.13039\/100012385","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000738","name":"U.S. Department of Veterans Affairs","doi-asserted-by":"publisher","award":["5 I01 CX000801 02"],"award-info":[{"award-number":["5 I01 CX000801 02"]}],"id":[{"id":"10.13039\/100000738","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BioData Mining"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Screening for suicidal ideation in high-risk groups such as U.S. veterans is crucial for early detection and suicide prevention. Currently, screening is based on clinical interviews or self-report measures. Both approaches rely on subjects to disclose their suicidal thoughts. Innovative approaches are necessary to develop objective and clinically applicable assessments. Speech has been investigated as an objective marker to understand various mental states including suicidal ideation. In this work, we developed a machine learning and natural language processing classifier based on speech markers to screen for suicidal ideation in US veterans.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methodology<\/jats:title>\n                    <jats:p>Veterans submitted 588 narrative audio recordings via a mobile app in a real-life setting. In addition, participants completed self-report psychiatric scales and questionnaires. Recordings were analyzed to extract voice characteristics including prosodic, phonation, and glottal. The audios were also transcribed to extract textual features for linguistic analysis. We evaluated the acoustic and linguistic features using both statistical significance and ensemble feature selection. We also examined the performance of different machine learning algorithms on multiple combinations of features to classify suicidal and non-suicidal audios.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      A combined set of 15 acoustic and linguistic features of speech were identified by the ensemble feature selection\n                      <jats:bold>.<\/jats:bold>\n                      Random Forest classifier, using the selected set of features, correctly identified suicidal ideation in veterans with 86% sensitivity, 70% specificity, and an area under the receiver operating characteristic curve (AUC) of 80%.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Speech analysis of audios collected from veterans in everyday life settings using smartphones offers a promising approach for suicidal ideation detection. A machine learning classifier may eventually help clinicians identify and monitor high-risk veterans.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s13040-021-00245-y","type":"journal-article","created":{"date-parts":[[2021,2,2]],"date-time":"2021-02-02T16:07:19Z","timestamp":1612282039000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Acoustic and language analysis of speech for suicidal ideation among US veterans"],"prefix":"10.1186","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2780-2500","authenticated-orcid":false,"given":"Anas","family":"Belouali","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samir","family":"Gupta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vaibhav","family":"Sourirajan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiawei","family":"Yu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathaniel","family":"Allen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adil","family":"Alaoui","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mary Ann","family":"Dutton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew J.","family":"Reinhard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,2]]},"reference":[{"key":"245_CR1","volume-title":"National veteran suicide prevention annual report","author":"Of Veterans Affairs D, Others","year":"2019","unstructured":"Of Veterans Affairs D, Others. National veteran suicide prevention annual report. Washington: Department of Veterans Affairs; 2019."},{"key":"245_CR2","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1037\/0022-006X.47.2.343","volume":"47","author":"AT Beck","year":"1979","unstructured":"Beck AT, Kovacs M, Weissman A. Assessment of suicidal intention: the Scale for Suicide Ideation. J Consult Clin Psychol. 1979;47:343\u201352.","journal-title":"J Consult Clin Psychol."},{"key":"245_CR3","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1037\/0022-006X.68.3.371","volume":"68","author":"GK Brown","year":"2000","unstructured":"Brown GK, Beck AT, Steer RA, Grisham JR. Risk factors for suicide in psychiatric outpatients: a 20-year prospective study. J Consult Clin Psychol. 2000;68:371\u20137.","journal-title":"J Consult Clin Psychol."},{"key":"245_CR4","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.psychres.2012.06.036","volume":"200","author":"PC Britton","year":"2012","unstructured":"Britton PC, Ilgen MA, Rudd MD, Conner KR. Warning signs for suicide within a week of healthcare contact in Veteran decedents. Psychiatry Res. 2012;200:395\u20139.","journal-title":"Psychiatry Res."},{"key":"245_CR5","volume-title":"Suicidal ideation questionnaire (SIQ)","author":"WM Reynolds","year":"1987","unstructured":"Reynolds WM. Suicidal ideation questionnaire (SIQ). Odessa: Psychological Assessment Resources; 1987. Available from: http:\/\/www.v-psyche.com\/doc\/Clinical%20Test\/Suicidal%20Ideation%20Questionnaire.doc"},{"key":"245_CR6","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.1001\/archpediatrics.2012.1276","volume":"166","author":"LM Horowitz","year":"2012","unstructured":"Horowitz LM, Bridge JA, Teach SJ, Ballard E, Klima J, Rosenstein DL, et al. Ask Suicide-Screening Questions (ASQ): a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med. 2012;166:1170\u20136.","journal-title":"Arch Pediatr Adolesc Med."},{"key":"245_CR7","volume-title":"Columbia-suicide severity rating scale (C-SSRS)","author":"K Posner","year":"2008","unstructured":"Posner K, Brent D, Lucas C, Gould M, Stanley B, Brown G, et al. Columbia-suicide severity rating scale (C-SSRS). New York: Columbia University Medical Center; 2008. Available from: https:\/\/depts.washington.edu\/ebpa\/sites\/default\/files\/C-SSRS-LifetimeRecent-Clinical.pdf"},{"key":"245_CR8","doi-asserted-by":"publisher","unstructured":"Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, et al. Prediction Models for Suicide Attempts and Deaths. JAMA Psychiatry. 2019:642. https:\/\/doi.org\/10.1001\/jamapsychiatry.2019.0174.","DOI":"10.1001\/jamapsychiatry.2019.0174"},{"key":"245_CR9","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1037\/bul0000084","volume":"143","author":"JC Franklin","year":"2017","unstructured":"Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol Bull. 2017;143:187\u2013232.","journal-title":"Psychol Bull."},{"key":"245_CR10","doi-asserted-by":"publisher","first-page":"1215","DOI":"10.1007\/s11606-013-2412-6","volume":"28","author":"L Ganzini","year":"2013","unstructured":"Ganzini L, Denneson LM, Press N, Bair MJ, Helmer DA, Poat J, et al. Trust is the basis for effective suicide risk screening and assessment in veterans. J Gen Intern Med. 2013;28:1215\u201321.","journal-title":"J Gen Intern Med."},{"key":"245_CR11","doi-asserted-by":"publisher","first-page":"239","DOI":"10.2105\/AJPH.93.2.239","volume":"93","author":"LR Snowden","year":"2003","unstructured":"Snowden LR. Bias in mental health assessment and intervention: theory and evidence. Am J Public Health. 2003;93:239\u201343.","journal-title":"Am J Public Health."},{"key":"245_CR12","doi-asserted-by":"publisher","unstructured":"Melia R, Francis K, Hickey E, Bogue J, Duggan J, O\u2019Sullivan M, et al. Mobile Health Technology Interventions for Suicide Prevention: Systematic Review. JMIR mHealth and uHealth. 2020:e12516. Available from:. https:\/\/doi.org\/10.2196\/12516.","DOI":"10.2196\/12516"},{"key":"245_CR13","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.specom.2015.03.004","volume":"71","author":"N Cummins","year":"2015","unstructured":"Cummins N, Scherer S, Krajewski J, Schnieder S, Epps J, Quatieri TF. A review of depression and suicide risk assessment using speech analysis. Speech Commun. 2015;71:10\u201349.","journal-title":"Speech Commun."},{"key":"245_CR14","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/978-3-319-28047-9_2","volume-title":"Emotion, Affect and Personality in Speech: The Bias of Language and Paralanguage","author":"S Johar","year":"2016","unstructured":"Johar S. Psychology of Voice. In: Johar S, editor. Emotion, Affect and Personality in Speech: The Bias of Language and Paralanguage. Cham: Springer International Publishing; 2016. p. 9\u201315."},{"key":"245_CR15","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1002\/lio2.354","volume":"5","author":"DM Low","year":"2020","unstructured":"Low DM, Bentley KH, Ghosh SS. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investig Otolaryngol. 2020;5:96\u2013116.","journal-title":"Laryngoscope Investig Otolaryngol."},{"key":"245_CR16","doi-asserted-by":"publisher","first-page":"300","DOI":"10.1186\/s12888-019-2300-7","volume":"19","author":"J Wang","year":"2019","unstructured":"Wang J, Zhang L, Liu T, Pan W, Hu B, Zhu T. Acoustic differences between healthy and depressed people: a cross-situation study. BMC Psychiatry. 2019;19:300.","journal-title":"BMC Psychiatry."},{"key":"245_CR17","doi-asserted-by":"publisher","first-page":"829","DOI":"10.1109\/10.846676","volume":"47","author":"DJ France","year":"2000","unstructured":"France DJ, Shiavi RG, Silverman S, Silverman M, Wilkes DM. Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Trans Biomed Eng. 2000;47:829\u201337.","journal-title":"IEEE Trans Biomed Eng."},{"key":"245_CR18","first-page":"1716","volume-title":"Detecting Depression with Audio\/Text Sequence Modeling of Interviews. Interspeech","author":"T Al Hanai","year":"2018","unstructured":"Al Hanai T, Ghassemi MM, Glass JR. Detecting Depression with Audio\/Text Sequence Modeling of Interviews. Interspeech; 2018. p. 1716\u201320."},{"key":"245_CR19","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1002\/da.22890","volume":"36","author":"CR Marmar","year":"2019","unstructured":"Marmar CR, Brown AD, Qian M, Laska E, Siegel C, Li M, et al. Speech-based markers for posttraumatic stress disorder in US veterans. Depress Anxiety. 2019;36:607\u201316.","journal-title":"Depress Anxiety."},{"key":"245_CR20","doi-asserted-by":"publisher","first-page":"e856","DOI":"10.1038\/tp.2016.123","volume":"6","author":"M Faurholt-Jepsen","year":"2016","unstructured":"Faurholt-Jepsen M, Busk J, Frost M, Vinberg M, Christensen EM, Winther O, et al. Voice analysis as an objective state marker in bipolar disorder. Transl Psychiatry. 2016;6:e856.","journal-title":"Transl Psychiatry."},{"key":"245_CR21","first-page":"91","volume-title":"A Computational Approach to Feature Extraction for Identification of Suicidal Ideation in Tweets. Proceedings of ACL 2018, Student Research Workshop","author":"R Sawhney","year":"2018","unstructured":"Sawhney R, Manchanda P, Singh R, Aggarwal S. A Computational Approach to Feature Extraction for Identification of Suicidal Ideation in Tweets. Proceedings of ACL 2018, Student Research Workshop. Melbourne, Australia: Association for Computational Linguistics; 2018. p. 91\u20138."},{"key":"245_CR22","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1177\/2167702617747074","volume":"6","author":"M Al-Mosaiwi","year":"2018","unstructured":"Al-Mosaiwi M, Johnstone T. In an Absolute State: Elevated Use of Absolutist Words Is a Marker Specific to Anxiety, Depression, and Suicidal Ideation. Clin Psychol Sci. 2018;6:529\u201342.","journal-title":"Clin Psychol Sci."},{"key":"245_CR23","first-page":"941","volume-title":"#suicidal - A Multipronged Approach to Identify and Explore Suicidal Ideation in Twitter. Proceedings of the 28th ACM International Conference on Information and Knowledge Management","author":"PP Sinha","year":"2019","unstructured":"Sinha PP, Mishra R, Sawhney R, Mahata D, Shah RR, Liu H. #suicidal - A Multipronged Approach to Identify and Explore Suicidal Ideation in Twitter. Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery; 2019. p. 941\u201350."},{"key":"245_CR24","first-page":"147","volume-title":"SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop","author":"R Mishra","year":"2019","unstructured":"Mishra R, Prakhar Sinha P, Sawhney R, Mahata D, Mathur P, Ratn SR. SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 147\u201356."},{"key":"245_CR25","first-page":"167","volume-title":"Exploring and Learning Suicidal Ideation Connotations on Social Media with Deep Learning. Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis","author":"R Sawhney","year":"2018","unstructured":"Sawhney R, Manchanda P, Mathur P, Shah R, Singh R. Exploring and Learning Suicidal Ideation Connotations on Social Media with Deep Learning. Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Brussels, Belgium: Association for Computational Linguistics; 2018. p. 167\u201375."},{"key":"245_CR26","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1186\/s12859-018-2197-z","volume":"19","author":"RN Grant","year":"2018","unstructured":"Grant RN, Kucher D, Le\u00f3n AM, Gemmell JF, Raicu DS, Fodeh SJ. Automatic extraction of informal topics from online suicidal ideation. BMC Bioinformatics. 2018;19:211.","journal-title":"BMC Bioinformatics."},{"key":"245_CR27","volume-title":"Method for detecting suicidal predisposition. US Patent","author":"SE Silverman","year":"1992","unstructured":"Silverman SE. Method for detecting suicidal predisposition. US Patent. 1992 [cited 2020 Mar 20]. Available from: https:\/\/patentimages.storage.googleapis.com\/08\/0e\/27\/15016f5fa2ae88\/US5148483.pdf"},{"key":"245_CR28","volume-title":"Method for analysis of vocal jitter for near-term suicidal risk assessment. US Patent","author":"SE Silverman","year":"2006","unstructured":"Silverman SE, Ozdas A, Silverman MK. Method for analysis of vocal jitter for near-term suicidal risk assessment. US Patent. 2006 [cited 2020 Mar 20]. Available from: https:\/\/patentimages.storage.googleapis.com\/e8\/0f\/25\/ef4db4ef5cc7d6\/US7139699.pdf"},{"key":"245_CR29","first-page":"709","volume-title":"Investigating the speech characteristics of suicidal adolescents. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing","author":"S Scherer","year":"2013","unstructured":"Scherer S, Pestian J, Morency L. Investigating the speech characteristics of suicidal adolescents. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing; 2013. p. 709\u201313."},{"key":"245_CR30","unstructured":"Hashim NW, Wilkes M, Salomon R, Meggs J. Analysis of timing pattern of speech as possible indicator for near-term suicidal risk and depression in male patients. International Proceedings of Computer Science and Information Technology, vol. 58: IACSIT Press; 2012. p. 6."},{"key":"245_CR31","first-page":"3282","volume-title":"Emotion Recognition from Natural Phone Conversations in Individuals with and without Recent Suicidal Ideation. Interspeech","author":"J Gideon","year":"2019","unstructured":"Gideon J, Schatten HT, MG MI, Provost EM. Emotion Recognition from Natural Phone Conversations in Individuals with and without Recent Suicidal Ideation. Interspeech; 2019. p. 3282\u20136."},{"key":"245_CR32","doi-asserted-by":"publisher","DOI":"10.1109\/icassp40776.2020.9053246","volume-title":"Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"SN Chakravarthula","year":"2020","unstructured":"Chakravarthula SN, Nasir M, Tseng S-Y, Li H, Park TJ, Baucom B, et al. Automatic Prediction of Suicidal Risk in Military Couples Using Multimodal Interaction Cues from Couples Conversations. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2020. Available from:. https:\/\/doi.org\/10.1109\/icassp40776.2020.9053246."},{"key":"245_CR33","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1111\/sltb.12312","volume":"47","author":"JP Pestian","year":"2017","unstructured":"Pestian JP, Sorter M, Connolly B, Bretonnel Cohen K, McCullumsmith C, Gee JT, et al. A Machine Learning Approach to Identifying the Thought Markers of Suicidal Subjects: A Prospective Multicenter Trial. Suicide Life Threat Behav. 2017;47:112\u201321.","journal-title":"Suicide Life Threat Behav."},{"key":"245_CR34","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1111\/sltb.12180","volume":"46","author":"JP Pestian","year":"2016","unstructured":"Pestian JP, Grupp-Phelan J, Bretonnel Cohen K, Meyers G, Richey LA, Matykiewicz P, et al. A Controlled Trial Using Natural Language Processing to Examine the Language of Suicidal Adolescents in the Emergency Department. Suicide Life Threat Behav. 2016;46:154\u20139.","journal-title":"Suicide Life Threat Behav."},{"key":"245_CR35","doi-asserted-by":"publisher","first-page":"981","DOI":"10.1001\/jama.280.11.981","volume":"280","author":"K Fukuda","year":"1998","unstructured":"Fukuda K, Nisenbaum R, Stewart G, Thompson WW, Robin L, Washko RM, et al. Chronic multisymptom illness affecting Air Force veterans of the Gulf War. JAMA. 1998;280:981\u20138.","journal-title":"JAMA."},{"key":"245_CR36","doi-asserted-by":"publisher","first-page":"509","DOI":"10.3928\/0048-5713-20020901-06","volume":"32","author":"K Kroenke","year":"2002","unstructured":"Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. Slack Incorporated. 2002;32:509\u201315.","journal-title":"Psychiatr Ann. Slack Incorporated"},{"key":"245_CR37","first-page":"517","volume":"67","author":"SA Louzon","year":"2016","unstructured":"Louzon SA, Bossarte R, McCarthy JF, Katz IR. Does Suicidal Ideation as Measured by the PHQ-9 Predict Suicide Among VA Patients? PS. American Psychiatric Publishing. 2016;67:517\u201322.","journal-title":"PS. American Psychiatric Publishing"},{"key":"245_CR38","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.jad.2017.03.037","volume":"215","author":"RC Rossom","year":"2017","unstructured":"Rossom RC, Coleman KJ, Ahmedani BK, Beck A, Johnson E, Oliver M, et al. Suicidal ideation reported on the PHQ9 and risk of suicidal behavior across age groups. J Affect Disord. 2017;215:77\u201384.","journal-title":"J Affect Disord."},{"key":"245_CR39","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1109\/TAFFC.2015.2457417","volume":"7","author":"F Eyben","year":"2016","unstructured":"Eyben F, Scherer KR, Schuller BW, Sundberg J, Andr\u00e9 E, Busso C, et al. The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing. IEEE Transactions on Affective Computing. 2016;7:190\u2013202.","journal-title":"IEEE Transactions on Affective Computing."},{"key":"245_CR40","volume-title":"Praat : doing phonetics by computer","author":"P Boersma","year":"2006","unstructured":"Boersma, P. Praat : doing phonetics by computer. 2006. http:\/\/www.praat.org\/ [cited 2020 Dec 4]; Available from: https:\/\/ci.nii.ac.jp\/naid\/10017594077\/"},{"key":"245_CR41","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1109\/taffc.2016.2518665","volume-title":"Adolescent Suicidal Risk Assessment in Clinician-Patient Interaction. IEEE Transactions on Affective Computing","author":"V Venek","year":"2017","unstructured":"Venek V, Scherer S, Morency L-P, Rizzo AS, Pestian J. Adolescent Suicidal Risk Assessment in Clinician-Patient Interaction. IEEE Transactions on Affective Computing; 2017. p. 204\u201315. Available from:. https:\/\/doi.org\/10.1109\/taffc.2016.2518665."},{"key":"245_CR42","doi-asserted-by":"publisher","first-page":"e0144610","DOI":"10.1371\/journal.pone.0144610","volume":"10","author":"T Giannakopoulos","year":"2015","unstructured":"Giannakopoulos T. pyAudioAnalysis: An Open-Source Python Library for Audio Signal Analysis. PLoS One. 2015;10:e0144610.","journal-title":"PLoS One"},{"key":"245_CR43","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1016\/j.dsp.2017.07.004","volume":"77","author":"JR Orozco-Arroyave","year":"2018","unstructured":"Orozco-Arroyave JR, V\u00e1squez-Correa JC, Vargas-Bonilla JF, Arora R, Dehak N, Nidadavolu PS, et al. NeuroSpeech: An open-source software for Parkinson\u2019s speech analysis. Digit Signal Process. 2018;77:207\u201321.","journal-title":"Digit Signal Process."},{"key":"245_CR44","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.jcomdis.2018.08.002","volume":"76","author":"JC V\u00e1squez-Correa","year":"2018","unstructured":"V\u00e1squez-Correa JC, Orozco-Arroyave JR, Bocklet T, N\u00f6th E. Towards an automatic evaluation of the dysarthria level of patients with Parkinson\u2019s disease. J Commun Disord. 2018;76:21\u201336.","journal-title":"J Commun Disord."},{"key":"245_CR45","first-page":"400","volume-title":"Glottal Flow Patterns Analyses for Parkinson\u2019s Disease Detection: Acoustic and Nonlinear Approaches. Text, Speech, and Dialogue","author":"EA Belalc\u00e1zar-Bola\u00f1os","year":"2016","unstructured":"Belalc\u00e1zar-Bola\u00f1os EA, Orozco-Arroyave JR, Vargas-Bonilla JF, Haderlein T, N\u00f6th E. Glottal Flow Patterns Analyses for Parkinson\u2019s Disease Detection: Acoustic and Nonlinear Approaches. Text, Speech, and Dialogue; 2016. p. 400\u20137. Available from: http:\/\/dx.doi.org\/10.1007\/978-3-319-45510-5_46"},{"key":"245_CR46","first-page":"688","volume":"22","author":"A V\u00e1zquez-Romero","year":"2020","unstructured":"V\u00e1zquez-Romero A, Gallardo-Antol\u00edn A. Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks. Entropy. Multidisciplinary Digital Publishing Institute. 2020;22:688.","journal-title":"Entropy. Multidisciplinary Digital Publishing Institute"},{"key":"245_CR47","unstructured":"Cloud Speech-to-Text - Speech Recognition | Google Cloud [Internet]. Google Cloud. [cited 2020 Mar 26]. Available from: https:\/\/cloud.google.com\/speech-to-text"},{"key":"245_CR48","doi-asserted-by":"publisher","DOI":"10.3115\/1118108.1118117","volume-title":"NLTK: The Natural Language Toolkit. arXiv [cs.CL]","author":"E Loper","year":"2002","unstructured":"Loper E, Bird S. NLTK: The Natural Language Toolkit. arXiv [cs.CL]. 2002. Available from: http:\/\/arxiv.org\/abs\/cs\/0205028"},{"key":"245_CR49","unstructured":"Pennebaker JW, Booth RJ, Boyd RL, Francis ME. Linguistic Inquiry and Word Count: LIWC 2015 [Computer software]: Pennebaker Conglomerates. Inc; 2015."},{"key":"245_CR50","volume-title":"Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ. arXiv [cs.CL]","author":"JS Kessler","year":"2017","unstructured":"Kessler JS. Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ. arXiv [cs.CL]. 2017. Available from: http:\/\/arxiv.org\/abs\/1703.00565"},{"key":"245_CR51","volume-title":"The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Akademia Wychowania Fizycznego w Poznaniu","author":"M Tomczak","year":"2014","unstructured":"Tomczak M, Tomczak E. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Akademia Wychowania Fizycznego w Poznaniu; 2014; Available from: https:\/\/www.wbc.poznan.pl\/dlibra\/publication\/413565\/edition\/325867?language=pl"},{"key":"245_CR52","unstructured":"Rea LM, Parker RA. Designing and Conducting Survey Research: A Comprehensive Guide: John Wiley & Sons; 2014."},{"key":"245_CR53","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1038\/s41746-019-0148-3","volume":"2","author":"P Shah","year":"2019","unstructured":"Shah P, Kendall F, Khozin S, Goosen R, Hu J, Laramie J, et al. Artificial intelligence and machine learning in clinical development: a translational perspective. NPJ Digit Med. 2019;2:69.","journal-title":"NPJ Digit Med."},{"key":"245_CR54","doi-asserted-by":"publisher","unstructured":"Pes B. Ensemble feature selection for high-dimensional data: a stability analysis across multiple domains. Neural Comput Appl. 2019; Available from. https:\/\/doi.org\/10.1007\/s00521-019-04082-3.","DOI":"10.1007\/s00521-019-04082-3"},{"key":"245_CR55","volume-title":"SMOTE: Synthetic Minority Over-sampling Technique. arXiv [cs.AI]","author":"NV Chawla","year":"2011","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: Synthetic Minority Over-sampling Technique. arXiv [cs.AI]. 2011. Available from: http:\/\/arxiv.org\/abs\/1106.1813"},{"key":"245_CR56","volume-title":"Overly Optimistic Prediction Results on Imbalanced Data: Flaws and Benefits of Applying Over-sampling. arXiv [eess.SP]","author":"G Vandewiele","year":"2020","unstructured":"Vandewiele G, Dehaene I, Kov\u00e1cs G, Sterckx L, Janssens O, Ongenae F, et al. Overly Optimistic Prediction Results on Imbalanced Data: Flaws and Benefits of Applying Over-sampling. arXiv [eess.SP]. 2020. Available from: http:\/\/arxiv.org\/abs\/2001.06296"},{"key":"245_CR57","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1007\/s10985-012-9238-0","volume":"19","author":"MJ Pencina","year":"2013","unstructured":"Pencina MJ, D\u2019Agostino RB, Massaro JM. Understanding increments in model performance metrics. Lifetime Data Anal. 2013;19:202\u201318.","journal-title":"Lifetime Data Anal."},{"key":"245_CR58","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1002\/bimj.200410135","volume":"47","author":"R Fluss","year":"2005","unstructured":"Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its associated cutoff point. Biom J. 2005;47:458\u201372.","journal-title":"Biom J."},{"key":"245_CR59","doi-asserted-by":"publisher","first-page":"e0224365","DOI":"10.1371\/journal.pone.0224365","volume":"14","author":"A Vabalas","year":"2019","unstructured":"Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLoS One. 2019;14:e0224365.","journal-title":"PLoS One."},{"key":"245_CR60","first-page":"2079","volume":"11","author":"GC Cawley","year":"2010","unstructured":"Cawley GC, Talbot NLC. On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation. J Mach Learn Res. 2010;11:2079\u2013107.","journal-title":"J Mach Learn Res."},{"key":"245_CR61","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1146\/annurev-clinpsy-032816-045037","volume":"14","author":"DB Dwyer","year":"2018","unstructured":"Dwyer DB, Falkai P, Koutsouleris N. Machine Learning Approaches for Clinical Psychology and Psychiatry. Annu Rev Clin Psychol. 2018;14:91\u2013118.","journal-title":"Annu Rev Clin Psychol."},{"key":"245_CR62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/gigascience\/gix019","volume":"6","author":"S Saeb","year":"2017","unstructured":"Saeb S, Lonini L, Jayaraman A, Mohr DC, Kording KP. The need to approximate the use-case in clinical machine learning. Gigascience. 2017;6:1\u20139.","journal-title":"Gigascience."},{"key":"245_CR63","unstructured":"Cross Validation of Cepstral Coefficients in Classifying Suicidal Speech from Depressed Speech. International Institute of Engineers (IIE) May 22-23, 2015 Dubai (UAE). International Institute of Engineers; 2015. Available from: http:\/\/iieng.org\/siteadmin\/upload\/8111E0515057.pdf"},{"key":"245_CR64","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1002\/da.23020","volume":"37","author":"L Zhang","year":"2020","unstructured":"Zhang L, Duvvuri R, Chandra KKL, Nguyen T, Ghomi RH. Automated voice biomarkers for depression symptoms using an online cross-sectional data collection initiative. Depress Anxiety. 2020;37:657\u201369.","journal-title":"Depress Anxiety."},{"key":"245_CR65","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1007\/s11920-013-0389-9","volume":"15","author":"WV McCall","year":"2013","unstructured":"McCall WV, Black CG. The link between suicide and insomnia: theoretical mechanisms. Curr Psychiatry Rep. 2013;15:389.","journal-title":"Curr Psychiatry Rep."},{"key":"245_CR66","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1007\/s11916-014-0436-1","volume":"18","author":"AL Hassett","year":"2014","unstructured":"Hassett AL, Aquino JK, Ilgen MA. The risk of suicide mortality in chronic pain patients. Curr Pain Headache Rep. 2014;18:436.","journal-title":"Curr Pain Headache Rep."},{"key":"245_CR67","first-page":"2098","volume":"2016","author":"M De Choudhury","year":"2016","unstructured":"De Choudhury M, Kiciman E, Dredze M, Coppersmith G, Kumar M. Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media. Proc SIGCHI Conf Hum Factor Comput Syst. 2016;2016:2098\u2013110.","journal-title":"Proc SIGCHI Conf Hum Factor Comput Syst."},{"key":"245_CR68","first-page":"31","volume-title":"Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies. Proceedings of the 14th ACM international conference on Multimodal interaction","author":"S D\u2019Mello","year":"2012","unstructured":"D\u2019Mello S, Kory J. Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies. Proceedings of the 14th ACM international conference on Multimodal interaction. New York: Association for Computing Machinery; 2012. p. 31\u20138."},{"key":"245_CR69","volume-title":"Depression Scale Recognition from Audio, Visual and Text Analysis. arXiv [cs.CV]","author":"S Dham","year":"2017","unstructured":"Dham S, Sharma A, Dhall A. Depression Scale Recognition from Audio, Visual and Text Analysis. arXiv [cs.CV]. 2017. Available from: http:\/\/arxiv.org\/abs\/1709.05865"},{"key":"245_CR70","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1109\/TCDS.2017.2721552","volume":"10","author":"A Jan","year":"2018","unstructured":"Jan A, Meng H, Gaus YFBA, Zhang F. Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions. IEEE Transactions on Cognitive and Developmental Systems. 2018;10:668\u201380.","journal-title":"IEEE Transactions on Cognitive and Developmental Systems."},{"key":"245_CR71","doi-asserted-by":"publisher","unstructured":"Silverman MK. Methods for evaluating near-term suicidal risk using vocal parameters. J Acoustical Soc Am. 2010:2259. Available from:. https:\/\/doi.org\/10.1121\/1.3500785.","DOI":"10.1121\/1.3500785"},{"key":"245_CR72","first-page":"2229","volume-title":"Screening for high risk suicidal states using mel-cepstral coefficients and energy in frequency bands. 2007 15th European Signal Processing Conference","author":"HK Keskinpala","year":"2007","unstructured":"Keskinpala HK, Yingthawornsuk T, Wilkes DM, Shiavi RG, Salomon RM. Screening for high risk suicidal states using mel-cepstral coefficients and energy in frequency bands. 2007 15th European Signal Processing Conference; 2007. p. 2229\u201333."},{"key":"245_CR73","doi-asserted-by":"crossref","first-page":"117822261982908","DOI":"10.1177\/1178222619829083","volume":"11","author":"K Kretzschmar","year":"2019","unstructured":"Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I, Group NYPA. Can your phone be your therapist? Young people\u2019s ethical perspectives on the use of fully automated conversational agents (chatbots) in mental health support. Biomed Inform Insights. SAGE Publications Sage UK: London, England. 2019;11:1178222619829083.","journal-title":"Biomed Inform Insights. SAGE Publications Sage UK: London, England"},{"key":"245_CR74","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.chb.2014.04.043","volume":"37","author":"GM Lucas","year":"2014","unstructured":"Lucas GM, Gratch J, King A, Morency L-P. It\u2019s only a computer: Virtual humans increase willingness to disclose. Comput Human Behav. Elsevier. 2014;37:94\u2013100.","journal-title":"Comput Human Behav. Elsevier"}],"container-title":["BioData Mining"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00245-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13040-021-00245-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00245-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,28]],"date-time":"2023-01-28T17:04:24Z","timestamp":1674925464000},"score":1,"resource":{"primary":{"URL":"https:\/\/biodatamining.biomedcentral.com\/articles\/10.1186\/s13040-021-00245-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,2]]},"references-count":74,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["245"],"URL":"https:\/\/doi.org\/10.1186\/s13040-021-00245-y","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.07.08.20147504","asserted-by":"object"}]},"ISSN":["1756-0381"],"issn-type":[{"value":"1756-0381","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,2]]},"assertion":[{"value":"18 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"DC VAMC IRB and the DC VAMC R&D Committee approved this research study.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"All the authors listed have read and approved the manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"11"}}