{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:09:17Z","timestamp":1750219757376,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":56,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,10,9]],"date-time":"2023-10-09T00:00:00Z","timestamp":1696809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Innosuisse","award":["34316.1 IP.ICT"],"award-info":[{"award-number":["34316.1 IP.ICT"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,10,9]]},"DOI":"10.1145\/3577190.3614165","type":"proceedings-article","created":{"date-parts":[[2023,10,7]],"date-time":"2023-10-07T22:30:48Z","timestamp":1696717848000},"page":"51-59","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Annotations from speech and heart rate: impact on multimodal emotion recognition"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-1799-1717","authenticated-orcid":false,"given":"Kaushal","family":"Sharma","sequence":"first","affiliation":[{"name":"SIMS Lab, Computer Science Department, University of Geneva, Switzerland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6184-8924","authenticated-orcid":false,"given":"Guillaume","family":"Chanel","sequence":"additional","affiliation":[{"name":"SIMS Lab, Computer Science Department, University of Geneva, Switzerland"}]}],"member":"320","published-online":{"date-parts":[[2023,10,9]]},"reference":[{"volume-title":"Support Vector Regression","author":"Awad Mariette","key":"e_1_3_2_1_1_1","unstructured":"Mariette Awad and Rahul Khanna . 2015. Support Vector Regression . Apress , Berkeley, CA , 67\u201380. https:\/\/doi.org\/10.1007\/978-1-4302-5990-9_4 10.1007\/978-1-4302-5990-9_4 Mariette Awad and Rahul Khanna. 2015. Support Vector Regression. Apress, Berkeley, CA, 67\u201380. https:\/\/doi.org\/10.1007\/978-1-4302-5990-9_4"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12144-014-9219-4"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2002.01020.x"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2018.8489099"},{"key":"e_1_3_2_1_5_1","volume-title":"The intraclass correlation coefficient as a measure of reliability. Psychological reports 19, 1","author":"Bartko J","year":"1966","unstructured":"John\u00a0 J Bartko . 1966. The intraclass correlation coefficient as a measure of reliability. Psychological reports 19, 1 ( 1966 ), 3\u201311. John\u00a0J Bartko. 1966. The intraclass correlation coefficient as a measure of reliability. Psychological reports 19, 1 (1966), 3\u201311."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2988257.2988264"},{"key":"e_1_3_2_1_7_1","first-page":"2455","article-title":"Emotion Recognition Using Different Sensors","volume":"23","author":"Cai Yujian","year":"2023","unstructured":"Yujian Cai , Xingguang Li , and Jinsong Li . 2023 . Emotion Recognition Using Different Sensors , Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors 23 , 5 (2023), 2455 . Yujian Cai, Xingguang Li, and Jinsong Li. 2023. Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors 23, 5 (2023), 2455.","journal-title":"Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2013.26"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2008.09.026"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG47880.2020.00001"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amc.2006.07.005"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2022-11258"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797610389191"},{"key":"e_1_3_2_1_14_1","volume-title":"To thine own self be true: interoceptive accuracy and interpersonal problems. Borderline personality disorder and emotion dysregulation 9, 1","author":"D\u2019Andrea Wendy","year":"2022","unstructured":"Wendy D\u2019Andrea , Nadia Nieves , and Treva Van\u00a0Cleave . 2022. To thine own self be true: interoceptive accuracy and interpersonal problems. Borderline personality disorder and emotion dysregulation 9, 1 ( 2022 ), 1\u20137. Wendy D\u2019Andrea, Nadia Nieves, and Treva Van\u00a0Cleave. 2022. To thine own self be true: interoceptive accuracy and interpersonal problems. Borderline personality disorder and emotion dysregulation 9, 1 (2022), 1\u20137."},{"key":"e_1_3_2_1_15_1","unstructured":"Asir\u00a0B El L Khadra AH Al-Abbasi 1996. Time-frequency Analysis of Heart Sounds. (1996).  Asir\u00a0B El L Khadra AH Al-Abbasi 1996. Time-frequency Analysis of Heart Sounds. (1996)."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sapharm.2012.04.004"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASRU.2005.1566530"},{"key":"#cr-split#-e_1_3_2_1_18_1.1","doi-asserted-by":"crossref","unstructured":"P. Hamilton. 2002. Open source ECG analysis. In Computers in Cardiology. 101-104. https:\/\/doi.org\/10.1109\/CIC.2002.1166717 10.1109\/CIC.2002.1166717","DOI":"10.1109\/CIC.2002.1166717"},{"key":"#cr-split#-e_1_3_2_1_18_1.2","doi-asserted-by":"crossref","unstructured":"P. Hamilton. 2002. Open source ECG analysis. In Computers in Cardiology. 101-104. https:\/\/doi.org\/10.1109\/CIC.2002.1166717","DOI":"10.1109\/CIC.2002.1166717"},{"key":"e_1_3_2_1_19_1","volume-title":"Surrey audio-visual expressed emotion (savee) database","author":"Jackson Philip","year":"2014","unstructured":"Philip Jackson and SJUo SG Haq . 2014. Surrey audio-visual expressed emotion (savee) database . University of Surrey : Guildford, UK ( 2014 ). Philip Jackson and SJUoSG Haq. 2014. Surrey audio-visual expressed emotion (savee) database. University of Surrey: Guildford, UK (2014)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.4018\/jse.2013010103"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418844"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3155604"},{"key":"e_1_3_2_1_23_1","volume-title":"Deap: A database for emotion analysis","author":"Koelstra Sander","year":"2011","unstructured":"Sander Koelstra , Christian Muhl , Mohammad Soleymani , Jong-Seok Lee , Ashkan Yazdani , Touradj Ebrahimi , Thierry Pun , Anton Nijholt , and Ioannis Patras . 2011 . Deap: A database for emotion analysis ; using physiological signals. IEEE transactions on affective computing 3, 1 (2011), 18\u201331. Sander Koelstra, Christian Muhl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and Ioannis Patras. 2011. Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing 3, 1 (2011), 18\u201331."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-019-01158-4"},{"key":"e_1_3_2_1_25_1","volume-title":"Estimating the reliability, systematic error and random error of interval data. Educational and psychological measurement 30, 1","author":"Krippendorff Klaus","year":"1970","unstructured":"Klaus Krippendorff . 1970. Estimating the reliability, systematic error and random error of interval data. Educational and psychological measurement 30, 1 ( 1970 ), 61\u201370. Klaus Krippendorff. 1970. Estimating the reliability, systematic error and random error of interval data. Educational and psychological measurement 30, 1 (1970), 61\u201370."},{"key":"e_1_3_2_1_26_1","unstructured":"Klaus Krippendorff. 2011. Computing Krippendorff\u2019s alpha-reliability. (2011).  Klaus Krippendorff. 2011. Computing Krippendorff\u2019s alpha-reliability. (2011)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.copsyc.2021.06.008"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.2307\/2532051"},{"key":"e_1_3_2_1_29_1","first-page":"324","article-title":"A note on the concordance correlation coefficient","volume":"56","author":"Kuei Lin Lawrence","year":"2000","unstructured":"Lawrence I- Kuei Lin . 2000 . A note on the concordance correlation coefficient . Biometrics 56 , 1 (2000), 324 \u2013 325 . https:\/\/doi.org\/10.1111\/j.0006-341X.2000.00324.x 10.1111\/j.0006-341X.2000.00324.x Lawrence I-Kuei Lin. 2000. A note on the concordance correlation coefficient. Biometrics 56, 1 (2000), 324\u2013325. https:\/\/doi.org\/10.1111\/j.0006-341X.2000.00324.x","journal-title":"Biometrics"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnbeh.2020.00067"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0196391"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-020-01516-y"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3011549.3011556"},{"key":"#cr-split#-e_1_3_2_1_34_1.1","doi-asserted-by":"crossref","unstructured":"Steven McGee. 2018. Chapter 40 - The First and Second Heart Sounds. In Evidence-Based Physical Diagnosis (Fourth Edition) (fourth edition ed.) Steven McGee (Ed.). Elsevier Philadelphia 333-344.e2. https:\/\/doi.org\/10.1016\/B978-0-323-39276-1.00040-8 10.1016\/B978-0-323-39276-1.00040-8","DOI":"10.1016\/B978-0-323-39276-1.00040-8"},{"key":"#cr-split#-e_1_3_2_1_34_1.2","doi-asserted-by":"crossref","unstructured":"Steven McGee. 2018. Chapter 40 - The First and Second Heart Sounds. In Evidence-Based Physical Diagnosis (Fourth Edition) (fourth edition ed.) Steven McGee (Ed.). Elsevier Philadelphia 333-344.e2. https:\/\/doi.org\/10.1016\/B978-0-323-39276-1.00040-8","DOI":"10.1016\/B978-0-323-39276-1.00040-8"},{"key":"e_1_3_2_1_35_1","volume-title":"A psycho-ethological approach to social signal processing. Cognitive processing 13","author":"Mehu Marc","year":"2012","unstructured":"Marc Mehu and Klaus\u00a0 R Scherer . 2012. A psycho-ethological approach to social signal processing. Cognitive processing 13 ( 2012 ), 397\u2013414. Marc Mehu and Klaus\u00a0R Scherer. 2012. A psycho-ethological approach to social signal processing. Cognitive processing 13 (2012), 397\u2013414."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2019.8925434"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.108580"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2884461"},{"key":"e_1_3_2_1_39_1","volume-title":"Proceedings, Part II. Springer, 730\u2013741","author":"Patania Sabrina","year":"2022","unstructured":"Sabrina Patania , Alessandro D\u2019Amelio , and Raffaella Lanzarotti . 2022 . Exploring Fusion Strategies in Deep Multimodal Affect Prediction. In Image Analysis and Processing\u2013ICIAP 2022: 21st International Conference, Lecce, Italy, May 23\u201327, 2022 , Proceedings, Part II. Springer, 730\u2013741 . Sabrina Patania, Alessandro D\u2019Amelio, and Raffaella Lanzarotti. 2022. Exploring Fusion Strategies in Deep Multimodal Affect Prediction. In Image Analysis and Processing\u2013ICIAP 2022: 21st International Conference, Lecce, Italy, May 23\u201327, 2022, Proceedings, Part II. Springer, 730\u2013741."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3536221.3556584"},{"volume-title":"Emotion Measurement","author":"Pool R","key":"e_1_3_2_1_42_1","unstructured":"Eva\u00a0 R Pool and David Sander . 2021. Emotional learning: measuring how affective values are acquired and updated . In Emotion Measurement . Elsevier , 133\u2013165. Eva\u00a0R Pool and David Sander. 2021. Emotional learning: measuring how affective values are acquired and updated. In Emotion Measurement. Elsevier, 133\u2013165."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2014.11.007"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133953"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2013.6553805"},{"key":"e_1_3_2_1_46_1","volume-title":"RStudio: integrated development for R. Rstudio Team","author":"Team","year":"2020","unstructured":"Team RStudio 2020. RStudio: integrated development for R. Rstudio Team , PBC, Boston, MA URL http:\/\/www. rstudio. com ( 2020 ). Team RStudio 2020. RStudio: integrated development for R. Rstudio Team, PBC, Boston, MA URL http:\/\/www. rstudio. com (2020)."},{"key":"e_1_3_2_1_47_1","volume-title":"A circumplex model of affect.Journal of personality and social psychology 39, 6","author":"Russell A","year":"1980","unstructured":"James\u00a0 A Russell . 1980. A circumplex model of affect.Journal of personality and social psychology 39, 6 ( 1980 ), 1161. James\u00a0A Russell. 1980. A circumplex model of affect.Journal of personality and social psychology 39, 6 (1980), 1161."},{"volume-title":"Appraisal processes in emotion: Theory, methods, research","author":"Scherer R","key":"e_1_3_2_1_48_1","unstructured":"Klaus\u00a0 R Scherer , Angela Schorr , and Tom Johnstone . 2001. Appraisal processes in emotion: Theory, methods, research . Oxford University Press . Klaus\u00a0R Scherer, Angela Schorr, and Tom Johnstone. 2001. Appraisal processes in emotion: Theory, methods, research. Oxford University Press."},{"key":"e_1_3_2_1_49_1","volume-title":"Do interoceptive accuracy and interoceptive sensibility predict emotion regulation?Psychological Research 85","author":"Schuette A","year":"2021","unstructured":"Stephanie\u00a0 A Schuette , Nancy\u00a0 L Zucker , and Moria\u00a0 J Smoski . 2021. Do interoceptive accuracy and interoceptive sensibility predict emotion regulation?Psychological Research 85 ( 2021 ), 1894\u20131908. Stephanie\u00a0A Schuette, Nancy\u00a0L Zucker, and Moria\u00a0J Smoski. 2021. Do interoceptive accuracy and interoceptive sensibility predict emotion regulation?Psychological Research 85 (2021), 1894\u20131908."},{"key":"e_1_3_2_1_50_1","first-page":"6","article-title":"A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database","volume":"6","author":"Faridul\u00a0Haque Siddiqui Mohammad","year":"2022","unstructured":"Mohammad Faridul\u00a0Haque Siddiqui , Parashar Dhakal , Xiaoli Yang , and Ahmad\u00a0 Y Javaid . 2022 . A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database . Multimodal Technologies and Interaction 6 , 6 (June 2022), 47. Mohammad Faridul\u00a0Haque Siddiqui, Parashar Dhakal, Xiaoli Yang, and Ahmad\u00a0Y Javaid. 2022. A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database. Multimodal Technologies and Interaction 6, 6 (June 2022), 47.","journal-title":"Multimodal Technologies and Interaction"},{"key":"e_1_3_2_1_51_1","volume-title":"Multisensory perception of the six basic emotions is modulated by attentional instruction and unattended modality. Frontiers in Integrative Neuroscience 9 (2","author":"Takagi Sachiko","year":"2015","unstructured":"Sachiko Takagi , Saori Hiramatsu , Ken\u00a0Ichi Tabei , and Akihiro Tanaka . 2015. Multisensory perception of the six basic emotions is modulated by attentional instruction and unattended modality. Frontiers in Integrative Neuroscience 9 (2 2015 ). Issue FEB. https:\/\/doi.org\/10.3389\/fnint.2015.00001 10.3389\/fnint.2015.00001 Sachiko Takagi, Saori Hiramatsu, Ken\u00a0Ichi Tabei, and Akihiro Tanaka. 2015. Multisensory perception of the six basic emotions is modulated by attentional instruction and unattended modality. Frontiers in Integrative Neuroscience 9 (2 2015). Issue FEB. https:\/\/doi.org\/10.3389\/fnint.2015.00001"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2988257.2988258"},{"key":"e_1_3_2_1_53_1","volume-title":"Dawn of the transformer era in speech emotion recognition: closing the valence gap","author":"Wagner Johannes","year":"2023","unstructured":"Johannes Wagner , Andreas Triantafyllopoulos , Hagen Wierstorf , Maximilian Schmitt , Felix Burkhardt , Florian Eyben , and Bj\u00f6rn\u00a0 W Schuller . 2023. Dawn of the transformer era in speech emotion recognition: closing the valence gap . IEEE Transactions on Pattern Analysis and Machine Intelligence ( 2023 ). Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Bj\u00f6rn\u00a0W Schuller. 2023. Dawn of the transformer era in speech emotion recognition: closing the valence gap. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)."},{"key":"e_1_3_2_1_54_1","volume-title":"Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259","author":"Zadeh Amir","year":"2016","unstructured":"Amir Zadeh , Rowan Zellers , Eli Pincus , and Louis-Philippe Morency . 2016. Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259 ( 2016 ). Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. 2016. Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259 (2016)."},{"key":"e_1_3_2_1_55_1","volume-title":"Multimodal affective states recognition based on multiscale cnns and biologically inspired decision fusion model","author":"Zhao Yuxuan","year":"2021","unstructured":"Yuxuan Zhao , Xinyan Cao , Jinlong Lin , Dunshan Yu , and Xixin Cao . 2021. Multimodal affective states recognition based on multiscale cnns and biologically inspired decision fusion model . IEEE Transactions on Affective Computing ( 2021 ). Yuxuan Zhao, Xinyan Cao, Jinlong Lin, Dunshan Yu, and Xixin Cao. 2021. Multimodal affective states recognition based on multiscale cnns and biologically inspired decision fusion model. IEEE Transactions on Affective Computing (2021)."}],"event":{"name":"ICMI '23: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Paris France","acronym":"ICMI '23"},"container-title":["INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3577190.3614165","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3577190.3614165","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:02Z","timestamp":1750178222000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3577190.3614165"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,9]]},"references-count":56,"alternative-id":["10.1145\/3577190.3614165","10.1145\/3577190"],"URL":"https:\/\/doi.org\/10.1145\/3577190.3614165","relation":{},"subject":[],"published":{"date-parts":[[2023,10,9]]},"assertion":[{"value":"2023-10-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}