{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T18:03:25Z","timestamp":1764785005867,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,11,7]]},"DOI":"10.1145\/3536220.3558075","type":"proceedings-article","created":{"date-parts":[[2022,11,4]],"date-time":"2022-11-04T22:11:40Z","timestamp":1667599900000},"page":"171-176","source":"Crossref","is-referenced-by-count":9,"title":["Towards Multimodal Dialog-Based Speech & Facial Biomarkers of Schizophrenia"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1009-0655","authenticated-orcid":false,"given":"Vanessa","family":"Richter","sequence":"first","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1023-9061","authenticated-orcid":false,"given":"Michael","family":"Neumann","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4305-0334","authenticated-orcid":false,"given":"Hardik","family":"Kothare","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9358-8242","authenticated-orcid":false,"given":"Oliver","family":"Roesler","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"given":"Jackson","family":"Liscombe","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"given":"David","family":"Suendermann-Oeft","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]},{"given":"Sebastian","family":"Prokop","sequence":"additional","affiliation":[{"name":"Nathan Kline Institute for Psychiatric Research, United States"}]},{"given":"Anzalee","family":"Khan","sequence":"additional","affiliation":[{"name":"Nathan Kline Institute for Psychiatric Research, United States and Valis Bioscience, United States"}]},{"given":"Christian","family":"Yavorsky","sequence":"additional","affiliation":[{"name":"Valis Bioscience, United States"}]},{"given":"Jean-Pierre","family":"Lindenmayer","sequence":"additional","affiliation":[{"name":"Nathan Kline Institute for Psychiatric Research, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7810-2769","authenticated-orcid":false,"given":"Vikram","family":"Ramanarayanan","sequence":"additional","affiliation":[{"name":"Modality.AI, Inc., United States"}]}],"member":"320","published-online":{"date-parts":[[2022,11,7]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.2196\/26276"},{"key":"e_1_3_2_1_2_1","volume-title":"Calgary Depression Scale for Schizophrenia (CDSS)","author":"Addington Donald","year":"2000","unstructured":"Donald Addington , Jean Addington , and B Schissel . 2000. Calgary Depression Scale for Schizophrenia (CDSS) . American Psychiatric Association. Task Force for the Handbook of Psychiatric Measures. American Psychiatric Association . Washington DC ( 2000 ), 504\u2013507. Donald Addington, Jean Addington, and B Schissel. 2000. Calgary Depression Scale for Schizophrenia (CDSS). American Psychiatric Association. Task Force for the Handbook of Psychiatric Measures. American Psychiatric Association. Washington DC (2000), 504\u2013507."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1001\/archpsyc.1982.04290070025006"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1192\/bjp.154.5.672"},{"key":"e_1_3_2_1_5_1","volume-title":"Speak and unSpeak with PRAAT. Glot International 5, 9\/10","author":"Boersma Paul","year":"2001","unstructured":"Paul Boersma and Vincent Van\u00a0Heuven . 2001. Speak and unSpeak with PRAAT. Glot International 5, 9\/10 ( 2001 ), 341\u2013347. Paul Boersma and Vincent Van\u00a0Heuven. 2001. Speak and unSpeak with PRAAT. Glot International 5, 9\/10 (2001), 341\u2013347."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.00269"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2018.8462102"},{"key":"e_1_3_2_1_8_1","volume-title":"Language as a biomarker for psychosis: A natural language processing approach. Schizophrenia research 226","author":"Corcoran M","year":"2020","unstructured":"Cheryl\u00a0 M Corcoran , Vijay\u00a0 A Mittal , Carrie\u00a0 E Bearden , Raquel\u00a0 E Gur , Kasia Hitczenko , Zarina Bilgrami , Aleksandar Savic , Guillermo\u00a0 A Cecchi , and Phillip Wolff . 2020. Language as a biomarker for psychosis: A natural language processing approach. Schizophrenia research 226 ( 2020 ), 158\u2013166. Cheryl\u00a0M Corcoran, Vijay\u00a0A Mittal, Carrie\u00a0E Bearden, Raquel\u00a0E Gur, Kasia Hitczenko, Zarina Bilgrami, Aleksandar Savic, Guillermo\u00a0A Cecchi, and Phillip Wolff. 2020. Language as a biomarker for psychosis: A natural language processing approach. Schizophrenia research 226 (2020), 158\u2013166."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.schres.2012.10.005"},{"key":"e_1_3_2_1_10_1","volume-title":"Facial action coding system. Environmental Psychology & Nonverbal Behavior","author":"Ekman Paul","year":"1978","unstructured":"Paul Ekman and Wallace\u00a0 V Friesen . 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior ( 1978 ). Paul Ekman and Wallace\u00a0V Friesen. 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior (1978)."},{"key":"e_1_3_2_1_11_1","volume-title":"Facial expression in the course of schizophrenia and depression. European archives of psychiatry and clinical neuroscience 254 (11","author":"Gaebel Wolfgang","year":"2004","unstructured":"Wolfgang Gaebel and Wolfgang Woelwer . 2004. Facial expression in the course of schizophrenia and depression. European archives of psychiatry and clinical neuroscience 254 (11 2004 ), 335\u201342. https:\/\/doi.org\/10.1007\/s00406-004-0510-5 10.1007\/s00406-004-0510-5 Wolfgang Gaebel and Wolfgang Woelwer. 2004. Facial expression in the course of schizophrenia and depression. European archives of psychiatry and clinical neuroscience 254 (11 2004), 335\u201342. https:\/\/doi.org\/10.1007\/s00406-004-0510-5"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00172"},{"volume-title":"ECDEU assessment manual for psychopharmacology","author":"Guy William","key":"e_1_3_2_1_13_1","unstructured":"William Guy . 1976. ECDEU assessment manual for psychopharmacology . US Department of Health , Education , and Welfare, Public Health Service\u00a0\u2026. William Guy. 1976. ECDEU assessment manual for psychopharmacology. US Department of Health, Education, and Welfare, Public Health Service\u00a0\u2026."},{"key":"e_1_3_2_1_14_1","volume-title":"The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia bulletin 13, 2","author":"Kay R","year":"1987","unstructured":"Stanley\u00a0 R Kay , Abraham Fiszbein , and Lewis\u00a0 A Opler . 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia bulletin 13, 2 ( 1987 ), 261\u2013276. Stanley\u00a0R Kay, Abraham Fiszbein, and Lewis\u00a0A Opler. 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia bulletin 13, 2 (1987), 261\u2013276."},{"key":"e_1_3_2_1_15_1","volume-title":"The brief negative symptom scale: psychometric properties. Schizophrenia bulletin 37, 2","author":"Kirkpatrick Brian","year":"2011","unstructured":"Brian Kirkpatrick , Gregory\u00a0 P Strauss , Linh Nguyen , Bernard\u00a0 A Fischer , David\u00a0 G Daniel , Angel Cienfuegos , and Stephen\u00a0 R Marder . 2011. The brief negative symptom scale: psychometric properties. Schizophrenia bulletin 37, 2 ( 2011 ), 300\u2013305. Brian Kirkpatrick, Gregory\u00a0P Strauss, Linh Nguyen, Bernard\u00a0A Fischer, David\u00a0G Daniel, Angel Cienfuegos, and Stephen\u00a0R Marder. 2011. The brief negative symptom scale: psychometric properties. Schizophrenia bulletin 37, 2 (2011), 300\u2013305."},{"key":"e_1_3_2_1_16_1","volume-title":"International Workshop on Health Intelligence. Springer, 27\u201335","author":"Krell Rony","year":"2021","unstructured":"Rony Krell , Wenqing Tang , Katrin H\u00e4nsel , Michael Sobolev , Sunghye Cho , Sarah Berretta , and Sunny\u00a0 X Tang . 2021 . Lexical and acoustic correlates of clinical speech disturbance in schizophrenia . In International Workshop on Health Intelligence. Springer, 27\u201335 . Rony Krell, Wenqing Tang, Katrin H\u00e4nsel, Michael Sobolev, Sunghye Cho, Sarah Berretta, and Sunny\u00a0X Tang. 2021. Lexical and acoustic correlates of clinical speech disturbance in schizophrenia. In International Workshop on Health Intelligence. Springer, 27\u201335."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/lio2.354"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1093\/oxfordjournals.schbul.a033335"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Patrick\u00a0E McKight and Julius Najab. 2010. Kruskal-wallis test. The corsini encyclopedia of psychology(2010) 1\u20131.  Patrick\u00a0E McKight and Julius Najab. 2010. Kruskal-wallis test. The corsini encyclopedia of psychology(2010) 1\u20131.","DOI":"10.1002\/9780470479216.corpsy0491"},{"key":"#cr-split#-e_1_3_2_1_20_1.1","doi-asserted-by":"crossref","unstructured":"Michael Neumann Oliver Roesler Jackson Liscombe Hardik Kothare David Suendermann-Oeft David Pautler Indu Navar Aria Anvar Jochen Kumm Raquel Norel Ernest Fraenkel Alex Sherman James Berry Gary Pattee Jun Wang Jordan Green and Vikram Ramanarayanan. 2021. Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale. Brno Czech Republic 4783-4787. https:\/\/doi.org\/10.21437\/Interspeech.2021-1801 10.21437\/Interspeech.2021-1801","DOI":"10.21437\/Interspeech.2021-1801"},{"key":"#cr-split#-e_1_3_2_1_20_1.2","doi-asserted-by":"crossref","unstructured":"Michael Neumann Oliver Roesler Jackson Liscombe Hardik Kothare David Suendermann-Oeft David Pautler Indu Navar Aria Anvar Jochen Kumm Raquel Norel Ernest Fraenkel Alex Sherman James Berry Gary Pattee Jun Wang Jordan Green and Vikram Ramanarayanan. 2021. Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale. Brno Czech Republic 4783-4787. https:\/\/doi.org\/10.21437\/Interspeech.2021-1801","DOI":"10.21437\/Interspeech.2021-1801"},{"key":"e_1_3_2_1_21_1","volume-title":"More than a biomarker: could language be a biosocial marker of psychosis?npj Schizophrenia 7, 1","author":"Palaniyappan Lena","year":"2021","unstructured":"Lena Palaniyappan . 2021. More than a biomarker: could language be a biosocial marker of psychosis?npj Schizophrenia 7, 1 ( 2021 ), 1\u20135. Lena Palaniyappan. 2021. More than a biomarker: could language be a biosocial marker of psychosis?npj Schizophrenia 7, 1 (2021), 1\u20135."},{"key":"e_1_3_2_1_22_1","volume-title":"Multimodal assessment of communicative-pragmatic features in schizophrenia: a machine learning approach. NPJ schizophrenia 7, 1","author":"Parola Alberto","year":"2021","unstructured":"Alberto Parola , Ilaria Gabbatore , Laura Berardinelli , Rogerio Salvini , and Francesca\u00a0 M Bosco . 2021. Multimodal assessment of communicative-pragmatic features in schizophrenia: a machine learning approach. NPJ schizophrenia 7, 1 ( 2021 ), 1\u20139. Alberto Parola, Ilaria Gabbatore, Laura Berardinelli, Rogerio Salvini, and Francesca\u00a0M Bosco. 2021. Multimodal assessment of communicative-pragmatic features in schizophrenia: a machine learning approach. NPJ schizophrenia 7, 1 (2021), 1\u20139."},{"key":"e_1_3_2_1_23_1","volume-title":"Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis. Schizophrenia research 216","author":"Parola Alberto","year":"2020","unstructured":"Alberto Parola , Arndis Simonsen , Vibeke Bliksted , and Riccardo Fusaroli . 2020. Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis. Schizophrenia research 216 ( 2020 ), 24\u201340. Alberto Parola, Arndis Simonsen, Vibeke Bliksted, and Riccardo Fusaroli. 2020. Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis. Schizophrenia research 216 (2020), 24\u201340."},{"key":"e_1_3_2_1_24_1","volume-title":"Speaking behavior and speech sound characteristics in acute schizophrenia. Journal of psychiatric research 32 (03","author":"Pueschel Joerg","year":"1998","unstructured":"Joerg Pueschel , Hans Stassen , G. Bomben , Christian Scharfetter , and Daniel Hell . 1998. Speaking behavior and speech sound characteristics in acute schizophrenia. Journal of psychiatric research 32 (03 1998 ), 89\u201397. https:\/\/doi.org\/10.1016\/S0022-3956(98)00046-6 10.1016\/S0022-3956(98)00046-6 Joerg Pueschel, Hans Stassen, G. Bomben, Christian Scharfetter, and Daniel Hell. 1998. Speaking behavior and speech sound characteristics in acute schizophrenia. Journal of psychiatric research 32 (03 1998), 89\u201397. https:\/\/doi.org\/10.1016\/S0022-3956(98)00046-6"},{"key":"e_1_3_2_1_25_1","volume-title":"Speech as a Biomarker: Opportunities, Interpretability, and Challenges. Perspectives of the ASHA Special Interest Groups","author":"Ramanarayanan Vikram","year":"2022","unstructured":"Vikram Ramanarayanan , Adam\u00a0 C Lammert , Hannah\u00a0 P Rowe , Thomas\u00a0 F Quatieri , and Jordan\u00a0 R Green . 2022. Speech as a Biomarker: Opportunities, Interpretability, and Challenges. Perspectives of the ASHA Special Interest Groups ( 2022 ), 1\u20138. Vikram Ramanarayanan, Adam\u00a0C Lammert, Hannah\u00a0P Rowe, Thomas\u00a0F Quatieri, and Jordan\u00a0R Green. 2022. Speech as a Biomarker: Opportunities, Interpretability, and Challenges. Perspectives of the ASHA Special Interest Groups (2022), 1\u20138."},{"key":"e_1_3_2_1_26_1","unstructured":"Vikram Ramanarayanan Oliver Roesler Michael Neumann David Pautler Doug Habberstad Andrew Cornish Hardik Kothare Vignesh Murali Jackson Liscombe Dirk Schnelle-Walka 2020. Toward Remote Patient Monitoring of Speech Video Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology.. In INTERSPEECH. 492\u2013493.  Vikram Ramanarayanan Oliver Roesler Michael Neumann David Pautler Doug Habberstad Andrew Cornish Hardik Kothare Vignesh Murali Jackson Liscombe Dirk Schnelle-Walka 2020. Toward Remote Patient Monitoring of Speech Video Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology.. In INTERSPEECH. 492\u2013493."},{"key":"e_1_3_2_1_27_1","volume-title":"Acoustic and temporal analysis of speech: A potential biomarker for schizophrenia. Medical engineering & physics 32, 9","author":"Rapcan Viliam","year":"2010","unstructured":"Viliam Rapcan , Shona D\u2019Arcy , Sherlyn Yeap , Natasha Afzal , Jogin Thakore , and Richard\u00a0 B Reilly . 2010. Acoustic and temporal analysis of speech: A potential biomarker for schizophrenia. Medical engineering & physics 32, 9 ( 2010 ), 1074\u20131079. Viliam Rapcan, Shona D\u2019Arcy, Sherlyn Yeap, Natasha Afzal, Jogin Thakore, and Richard\u00a0B Reilly. 2010. Acoustic and temporal analysis of speech: A potential biomarker for schizophrenia. Medical engineering & physics 32, 9 (2010), 1074\u20131079."},{"key":"e_1_3_2_1_28_1","volume-title":"Fathi Abd El-Samie, and Ahmed Sedik","author":"Siam Ali","year":"2022","unstructured":"Ali Siam , Naglaa Soliman , Abeer Algarni , Fathi Abd El-Samie, and Ahmed Sedik . 2022 . Deploying Machine Learning Techniques for Human Emotion Detection. Computational Intelligence and Neuroscience 2022 (02 2022). https:\/\/doi.org\/10.1155\/2022\/8032673 10.1155\/2022 Ali Siam, Naglaa Soliman, Abeer Algarni, Fathi Abd El-Samie, and Ahmed Sedik. 2022. Deploying Machine Learning Techniques for Human Emotion Detection. Computational Intelligence and Neuroscience 2022 (02 2022). https:\/\/doi.org\/10.1155\/2022\/8032673"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1600-0447.1970.tb02066.x"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3462244.3479967"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308532.3329415"},{"key":"e_1_3_2_1_32_1","volume-title":"Investigating the diagnostic utility of speech patterns in schizophrenia and their symptom associations. Schizophrenia research 238","author":"Tan J","year":"2021","unstructured":"Eric\u00a0 J Tan , Denny Meyer , Erica Neill , and Susan\u00a0 L Rossell . 2021. Investigating the diagnostic utility of speech patterns in schizophrenia and their symptom associations. Schizophrenia research 238 ( 2021 ), 91\u201398. Eric\u00a0J Tan, Denny Meyer, Erica Neill, and Susan\u00a0L Rossell. 2021. Investigating the diagnostic utility of speech patterns in schizophrenia and their symptom associations. Schizophrenia research 238 (2021), 91\u201398."}],"event":{"name":"ICMI '22: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Bengaluru India","acronym":"ICMI '22"},"container-title":["INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3536220.3558075","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3536220.3558075","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:54Z","timestamp":1750188654000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3536220.3558075"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":33,"alternative-id":["10.1145\/3536220.3558075","10.1145\/3536220"],"URL":"https:\/\/doi.org\/10.1145\/3536220.3558075","relation":{},"subject":[],"published":{"date-parts":[[2022,11,7]]}}}