{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T01:02:30Z","timestamp":1790989350962,"version":"4.1.0"},"reference-count":210,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2023,11,25]],"date-time":"2023-11-25T00:00:00Z","timestamp":1700870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100014013","name":"UKRI","doi-asserted-by":"crossref","award":["EP\/S02266X\/1"],"award-info":[{"award-number":["EP\/S02266X\/1"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000266","name":"EPSRC","doi-asserted-by":"crossref","award":["EP\/N035305\/1"],"award-info":[{"award-number":["EP\/N035305\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Automated co-located human\u2013human interaction analysis has been addressed by the use of nonverbal communication as measurable evidence of social and psychological phenomena. We survey the computing studies (since 2010) detecting phenomena related to social traits (e.g., leadership, dominance, and personality traits), social roles\/relations, and interaction dynamics (e.g., group cohesion, engagement, and rapport). Our target is to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings (free-standing conversations, meetings, dyads, and crowds). We also present a comprehensive summary of the related datasets and outline future research directions, which are regarding the implementation of artificial intelligence, dataset curation, and privacy-preserving interaction analysis. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3\u20134 persons equipped with microphones and cameras, respectively; multimodal features are prominently performing better; deep learning architectures showed improved performance in overall, but there exist many phenomena whose detection has never been implemented through deep models. We also identified several limitations such as the lack of scalable benchmarks, annotation reliability tests, cross-dataset experiments, and explainability analysis.<\/jats:p>","DOI":"10.1145\/3626516","type":"journal-article","created":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T11:44:44Z","timestamp":1696592684000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Co-Located Human\u2013Human Interaction Analysis Using Nonverbal Cues: A Survey"],"prefix":"10.1145","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9583-0087","authenticated-orcid":false,"given":"Cigdem","family":"Beyan","sequence":"first","affiliation":[{"name":"Dept. of Management, Information and Production Engineering, University of Bergamo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9048-0524","authenticated-orcid":false,"given":"Alessandro","family":"Vinciarelli","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2262-4872","authenticated-orcid":false,"given":"Alessio Del","family":"Bue","sequence":"additional","affiliation":[{"name":"Pattern Analysis and Computer Vision, Istituto Italiano di Tecnologia, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,25]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2016.7900087"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2019.8803634"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806238"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2015.06.006"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2014.91"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP43922.2022.9746426"},{"key":"e_1_3_1_8_2","first-page":"3687","volume-title":"Proceedings of the ICPR","author":"Aran Oya","year":"2010","unstructured":"Oya Aran and Daniel Gatica-Perez. 2010. Fusing audio-visual nonverbal cues to detect dominant people in small group conversations. In Proceedings of the ICPR. 3687\u20133690."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2522858"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2522859"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1145\/2666242.2666246","volume-title":"Workshop on Understanding Modeling Multiparty Multimodal Interactions","author":"Avci Umut","year":"2014","unstructured":"Umut Avci and Oya Aran. 2014. Effect of nonverbal behavioral patterns on the performance of small groups. In Workshop on Understanding Modeling Multiparty Multimodal Interactions. 9\u201314."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2521348"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/645"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477553"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2850284"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2017.274"},{"key":"e_1_3_1_17_2","first-page":"1","volume-title":"Proc. Workshop on Eye Gaze in Intelligent Human Machine Interaction","author":"Bednarik Roman","year":"2012","unstructured":"Roman Bednarik, Shahram Eivazi, and Michal Hradis. 2012. Gaze and conversational engagement in multiparty video conversation: An annotation scheme and classification of high and low levels of engagement. In Proc. Workshop on Eye Gaze in Intelligent Human Machine Interaction. 1\u20136."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418876"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3005467.3005469"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2740062"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993175"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123404"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2895505"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240685"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2019.2944614"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3304109.3325816"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3243022"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418843"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2015.03.005"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.331"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3279981.3279985"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2499621"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2848914"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993170"},{"key":"e_1_3_1_35_2","article-title":"OpenPose: Realtime multi-person 2D pose estimation using part affinity fields","author":"Cao Zhe","year":"2019","unstructured":"Zhe Cao, Gines Hidalgo Martinez, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2019. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on PAMI (2019).","journal-title":"IEEE Transactions on PAMI"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1093\/iwc\/iwv013"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"J. P. Chang C. Chiam L. Fu A. Z. Wang J. Zhang and C. Danescu-Niculescu-Mizil. 2020. Convokit: A toolkit for the analysis of conversations. arXiv preprint arXiv:2005.04246.","DOI":"10.18653\/v1\/2020.sigdial-1.8"},{"key":"e_1_3_1_38_2","first-page":"817","volume-title":"Proceedings of the IEEE ICASSP","year":"2015","unstructured":"Theodora Chaspari, Brian Baucom, Adela C. Timmons, Andreas Tsiartas, Larissa Borofsky Del Piero, Katherine J. W. Baucom, Panayiotis Georgiou, Gayla Margolin, and Shrikanth S. Narayanan. 2015. Quantifying EDA synchrony through joint sparse representation: A case-study of couples\u2019 interactions. In Proceedings of the IEEE ICASSP. 817\u2013821."},{"key":"e_1_3_1_39_2","first-page":"267","volume-title":"Proceedings of the ACII","year":"2017","unstructured":"Theodora Chaspari, Adela C. Timmons, Brian R. Baucom, Laura Perrone, Katherine J. W. Baucom, Panayiotis Georgiou, Gayla Margolin, and Shrikanth S. Narayanan. 2017. Exploring sparse representation measures of physiological synchrony for romantic couples. In Proceedings of the ACII. 267\u2013272."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2011.5771376"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.5244\/C.25.23"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/PASSAT\/SocialCom.2011.32"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2011.12.038"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2012.2225039"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG47880.2020.00024"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","unstructured":"Dario Dotti Mirela Popa and Stylianos Asteriadis. 2020. Being the center of attention: A person-context CNN framework for personality recognition. ACM Trans. Interact. Intell. Syst. 10 3 (2020) 1\u201320.","DOI":"10.1145\/3338245"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-021-00364-0"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300269"},{"key":"e_1_3_1_49_2","volume-title":"Human Activity Recognition and Prediction","author":"(Ed.) Yun Fu","year":"2016","unstructured":"Yun Fu (Ed.). 2016. Human Activity Recognition and Prediction. Springer."},{"key":"e_1_3_1_50_2","first-page":"1","volume-title":"EURASIP Journal on Advances in Signal Processing","author":"Escalera Sergio","year":"2010","unstructured":"Sergio Escalera, Oriol Pujol, Petia Radeva, Jordi Vitria, and M. Teresa Anguera. 2010. Automatic detection of dominance and expected interest. In EURASIP Journal on Advances in Signal Processing. 1\u201312."},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2457417"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502224"},{"key":"e_1_3_1_53_2","first-page":"961","volume-title":"Proceedings of the CVPR","author":"Escorcia Bernard Ghanem Fabian Caba Heilbron, Victor","year":"2015","unstructured":"Bernard Ghanem Fabian Caba Heilbron, Victor Escorcia and Juan Carlos Niebles. 2015. ActivityNet: A large-scale video benchmark for human activity understanding. In Proceedings of the CVPR. 961\u2013970."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993201"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6247805"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00138-018-0979-y"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","unstructured":"S. Feese B. Arnrich G. Troster B. Meyer and K. Jonas. 2011. Detecting posture mirroring in social interactions with wearable sensors. In 15th Annual International Symposium on Wearable Computers San Francisco CA 119\u2013120. DOI:10.1109\/ISWC.2011.31","DOI":"10.1109\/ISWC.2011.31"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom-PASSAT.2012.48"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/PASSAT\/SocialCom.2011.209"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2018-1654"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993164"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2016-202"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR56361.2022.9956589"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502096"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2009.01.004"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287041"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13229-019-0305-1"},{"key":"e_1_3_1_68_2","first-page":"7","volume-title":"ACM Workshop on Human Gesture and Behavior Understanding","author":"Georgiou Panayiotis G.","year":"2011","unstructured":"Panayiotis G. Georgiou, Matthew P. Black, and Shrikanth S. Narayanan. 2011. Behavioral signal processing for understanding (distressed) dyadic interactions: Some recent developments. In ACM Workshop on Human Gesture and Behavior Understanding. 7\u201312."},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2019.8852184"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0144610"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2020.101198"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom.2010.11"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-46416-0"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/PASSAT\/SocialCom.2011.143"},{"key":"e_1_3_1_75_2","first-page":"2374","volume-title":"Proceedings of the Interspeech","author":"Haider Fasih","year":"2019","unstructured":"Fasih Haider and Saturnino Luz. 2019. A system for real-time privacy preserving data collection for ambient assisted living. In Proceedings of the Interspeech. 2374\u20132375."},{"key":"e_1_3_1_76_2","doi-asserted-by":"crossref","first-page":"3359","DOI":"10.21437\/Interspeech.2017-1496","volume-title":"INTERSPEECH","author":"Huang Yuyun","year":"2017","unstructured":"Yuyun Huang, Emer Gilmartin, and Nick Campbell. 2017. Speaker dependency analysis, audiovisual fusion cues and a multimodal BLSTM for conversational engagement recognition. In INTERSPEECH. 3359\u20133363."},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/1873951.1874102"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2010.2055233"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2010.2066267"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3419748"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/SITIS.2016.89"},{"key":"e_1_3_1_82_2","first-page":"18","volume-title":"ACM ICMI","author":"Ishii Ryo","year":"2014","unstructured":"Ryo Ishii, Kazuhiro Otsuka, Shiro Kumano, and Junji Yamato. 2014. Analysis of respiration for prediction of \u201cwho will be next speaker and when?\u201d In multi-party meetings. In ACM ICMI. 18\u201325."},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-011-0243-z"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2010.2065218"},{"key":"e_1_3_1_85_2","first-page":"312","volume-title":"Proceedings of the ICIS","author":"Jie Cao","year":"2010","unstructured":"Cao Jie and Pan Peng. 2010. Recognize the most dominant person in multi-party meetings using nontraditional features. In Proceedings of the ICIS. 312\u2013316."},{"key":"e_1_3_1_86_2","first-page":"273","volume-title":"Companion Publication of ACM ICMI","author":"Johnson David","year":"2021","unstructured":"David Johnson and Gabriel Murray. 2021. Clustering and multimodal analysis of participants in task-based discussions. In Companion Publication of ACM ICMI. 273\u2013277."},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2782743"},{"key":"e_1_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388683"},{"key":"e_1_3_1_89_2","first-page":"124","article-title":"Automatic modeling of dominance effects using granger causality","volume":"7065","author":"Kalimeri Kyriaki","year":"2011","unstructured":"Kyriaki Kalimeri, Bruno Lepri, Taemie Kim, Fabio Pianesi, and Alex Sandy Pentland. 2011. Automatic modeling of dominance effects using granger causality. Human Behavior Understanding, Lecture Notes in Computer Science, A. Salah, Lepri, B., Eds., Springer, Berlin\/Heidelberg 7065 (2011), 124\u2013133.","journal-title":"Human Behavior Understanding, Lecture Notes in Computer Science, A. Salah, Lepri, B., Eds., Springer, Berlin\/Heidelberg"},{"key":"e_1_3_1_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/1878116.1878130"},{"key":"e_1_3_1_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132711.3132714"},{"key":"e_1_3_1_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACIIW.2019.8925137"},{"key":"e_1_3_1_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/2968219.2968281"},{"key":"e_1_3_1_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/3362743.3362959"},{"key":"e_1_3_1_95_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2013-457"},{"key":"e_1_3_1_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigMM50055.2020.00024"},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2010.7"},{"key":"e_1_3_1_98_2","first-page":"1","article-title":"Multi-domain and multi-task prediction of extraversion and leadership from meeting videos","volume":"77","author":"Kindiroglu Ahmet Alp","year":"2017","unstructured":"Ahmet Alp Kindiroglu, Lale Akarun, and Oya Aran. 2017. Multi-domain and multi-task prediction of extraversion and leadership from meeting videos. EURASIP JIVP 77, 1 (2017), 1\u201314.","journal-title":"EURASIP JIVP"},{"key":"e_1_3_1_99_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2013.10.007"},{"key":"e_1_3_1_100_2","doi-asserted-by":"publisher","DOI":"10.3390\/brainsci10050278"},{"key":"e_1_3_1_101_2","unstructured":"P. Koromilas and T. Giannakopoulos. 2021. Unsupervised multimodal language representations using convolutional autoencoders. arXiv preprint arXiv:2110.03007."},{"key":"e_1_3_1_102_2","volume-title":"Proceedings of the LREC","author":"Koutsombogera M.","year":"2018","unstructured":"M. Koutsombogera and C. Vogel. 2018. Modeling collaborative multimodal behavior in group dialogues: The MULTISIMO corpus. In Proceedings of the LREC."},{"key":"e_1_3_1_103_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2019-3062"},{"key":"e_1_3_1_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2015.7284861"},{"key":"e_1_3_1_105_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.191815"},{"key":"e_1_3_1_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3279810.3279840"},{"key":"e_1_3_1_107_2","volume-title":"Proceedings of the ISCA","year":"2010","unstructured":"Chi-Chun Lee, Matthew Black, Athanasios Katsamanis, Adam C. Lammert, Brian R. Baucom, Andrew Christensen, Panayiotis G. Georgiou, and Shrikanth S. Narayanan. 2010. Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples. In Proceedings of the ISCA."},{"key":"e_1_3_1_108_2","first-page":"1","volume-title":"Proceedings of the APSIPA","author":"Lee Chi-Chun","year":"2012","unstructured":"Chi-Chun Lee, Athanasios Katsamanis, Brian R. Baucom, Panayiotis G. Georgiou, and Shrikanth S. Narayanan. 2012. Using measures of vocal entrainment to inform outcome-related behaviors in marital conflicts. In Proceedings of the APSIPA. 1\u20135."},{"key":"e_1_3_1_109_2","first-page":"140","article-title":"Honest signals and their contribution to the automatic analysis of personality traits a comparative study","volume":"6219","author":"Lepri Bruno","year":"2010","unstructured":"Bruno Lepri, Kyriaki Kalimeri, and Fabio Pianesi. 2010. Honest signals and their contribution to the automatic analysis of personality traits a comparative study. Human Behavior Understanding, ser. Lecture Notes in Computer Science, A. Salah, T. Gevers, N. Sebe, and A. Vinciarelli, Eds. Springer, Berlin 6219 (2010), 140\u2013150.","journal-title":"Human Behavior Understanding, ser. Lecture Notes in Computer Science, A. Salah, T. Gevers, N. Sebe, and A. Vinciarelli, Eds. Springer, Berlin"},{"key":"e_1_3_1_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom-PASSAT.2012.71"},{"key":"e_1_3_1_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2012.17"},{"key":"e_1_3_1_112_2","volume-title":"Proceedings of the ACM ICMI-MLMI","author":"Lepri Bruno","year":"2013","unstructured":"Bruno Lepri, Ramanathan Subramanian, Kyriaki Kalimeri, Jacopo Staiano, Fabio Pianesi, and Nicu Sebe. 2013. Employing social gaze and speaking activity for automatic determination of the extraversion trait. In Proceedings of the ACM ICMI-MLMI."},{"key":"e_1_3_1_113_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2020.102468"},{"key":"e_1_3_1_114_2","first-page":"163","volume-title":"Proceedings of the ACM ICMI","author":"Lin Yun-Shao","year":"2016","unstructured":"Yun-Shao Lin and Chi-Chun Lee. 2016. Using interlocutor-modulated attention BLSTM to predict personality traits in small group interaction. In Proceedings of the ACM ICMI. 163\u2013169."},{"key":"e_1_3_1_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9053308"},{"key":"e_1_3_1_116_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00368"},{"key":"e_1_3_1_117_2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/2666633.2666635","volume-title":"Proc. ACM Multimodal Learning Analytics Workshop and Grand Challenge","author":"Lubold Nichola","year":"2014","unstructured":"Nichola Lubold and Heather Pon-Barry. 2014. Acoustic-prosodic entrainment and rapport in collaborative learning dialogues. In Proc. ACM Multimodal Learning Analytics Workshop and Grand Challenge. 5\u201312."},{"key":"e_1_3_1_118_2","volume-title":"Advances in Neural Information Processing Systems","author":"Lundberg Scott M","year":"2017","unstructured":"Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30."},{"key":"e_1_3_1_119_2","first-page":"355","volume-title":"MultiMedia Modeling","author":"Lv Jinna","year":"2018","unstructured":"Jinna Lv, Wu Liu, Lili Zhou, Bin Wu, and Huadong Ma. 2018. Multi-stream fusion model for social relation recognition from videos. In MultiMedia Modeling. Cham, 355\u2013368."},{"key":"e_1_3_1_120_2","first-page":"390","volume-title":"MultiMedia Modeling","author":"Lv Jinna","year":"2019","unstructured":"Jinna Lv and Bin Wu. 2019. Spatio-temporal attention model based on multi-view for social relation understanding. In MultiMedia Modeling. Cham, 390\u2013401."},{"key":"e_1_3_1_121_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3005719"},{"key":"e_1_3_1_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380121"},{"key":"e_1_3_1_123_2","doi-asserted-by":"publisher","DOI":"10.1145\/2638728.2641287"},{"key":"e_1_3_1_124_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-019-09770-z"},{"key":"e_1_3_1_125_2","doi-asserted-by":"publisher","DOI":"10.1109\/eScience.2019.00029"},{"key":"e_1_3_1_126_2","article-title":"AMIGOS: A dataset for affect, personality and mood research on individuals and groups","author":"Correa Juan Abdon Miranda","year":"2018","unstructured":"Juan Abdon Miranda Correa, Mojtaba Khomami Abadi, Nicu Sebe, and Ioannis Patras. 2018. AMIGOS: A dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Comput. (2018).","journal-title":"IEEE Transactions on Affective Comput."},{"key":"e_1_3_1_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340555.3353729"},{"key":"e_1_3_1_128_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2740923"},{"key":"e_1_3_1_129_2","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172969"},{"key":"e_1_3_1_130_2","doi-asserted-by":"publisher","unstructured":"Philipp Matthias Muller and Andreas Bulling. 2019. Emergent leadership detection across datasets. In 2019 International Conference on Multimodal Interaction (ICMI\u201919) . Association for Computing Machinery New York NY 274\u2013278. 10.1145\/3340555.3353721","DOI":"10.1145\/3340555.3353721"},{"issue":"3","key":"e_1_3_1_131_2","article-title":"A tale of two interactions: Inferring performance in hospitality encounters from cross-situation social sensing","volume":"2","author":"Muralidhar Skanda","year":"2018","unstructured":"Skanda Muralidhar, Marianne Schmid Mast, and Daniel Gatica-Perez. 2018. A tale of two interactions: Inferring performance in hospitality encounters from cross-situation social sensing. ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1\u201324.","journal-title":"ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_3_1_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3243027"},{"key":"e_1_3_1_133_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2015.7163127"},{"key":"e_1_3_1_134_2","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3136811"},{"key":"e_1_3_1_135_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2307169"},{"key":"e_1_3_1_136_2","doi-asserted-by":"publisher","DOI":"10.1145\/3212721.3212816"},{"key":"e_1_3_1_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/2663204.2663248"},{"key":"e_1_3_1_138_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-012-0108-6"},{"key":"e_1_3_1_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820759"},{"key":"e_1_3_1_140_2","doi-asserted-by":"crossref","unstructured":"Catharine Oertel Kenneth A. Funes Mora Samira Sheikhi Jean-Marc Odobez and Joakim Gustafson. 2014. Who will get the grant? A multimodal corpus for the analysis of conversational behaviours in group interviews. 27\u201332.","DOI":"10.1145\/2666242.2666251"},{"key":"e_1_3_1_141_2","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2522865"},{"key":"e_1_3_1_142_2","volume-title":"In Multimodal-Corpora:Beyond Audio and Gaze, 2013","author":"Oertel Catharine","unstructured":"Catharine Oertel, Giampiero Salvi, Jana Gotze, Jens Edlund, Joakim Gustafson, and Mattias Heldner. [n.d.]. The KTH games corpora: How to catch a werewolf. In In Multimodal-Corpora:Beyond Audio and Gaze, 2013."},{"key":"e_1_3_1_143_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1007\/978-3-319-58077-7_28","volume-title":"Human\u2013Computer Interaction. Interaction Contexts","author":"Ohnishi Ayumi","year":"2017","unstructured":"Ayumi Ohnishi, Kaoru Saito, Tsutomu Terada, and Masahiko Tsukamoto. 2017. Toward interest estimation from head motion using wearable sensors: A case study in story time for children. In Human\u2013Computer Interaction. Interaction Contexts. 353\u2013363."},{"key":"e_1_3_1_144_2","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820757"},{"key":"e_1_3_1_145_2","doi-asserted-by":"publisher","DOI":"10.1145\/3265754"},{"key":"e_1_3_1_146_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418868"},{"key":"e_1_3_1_147_2","doi-asserted-by":"publisher","DOI":"10.3390\/s19122786"},{"key":"e_1_3_1_148_2","doi-asserted-by":"publisher","DOI":"10.1145\/2893487"},{"key":"e_1_3_1_149_2","first-page":"1","volume-title":"Proceedings of the IEEE WACVw","year":"2021","unstructured":"Cristina Palmero, Javier Selva, Sorina Smeureanu, Julio C. S. Jacques Junior, Albert Clapes, Alexa Mosegui, Zejian Zhang, David Gallardo, Georgina Guilera, David Leiva, and Sergio Escalera. 2021. Context-aware personality inference in dyadic scenarios: Introducing the UDIVA dataset. In Proceedings of the IEEE WACVw. 1\u201312."},{"key":"e_1_3_1_150_2","doi-asserted-by":"publisher","unstructured":"G. Paoletti J. Cavazza C. Beyan and A. Del Bue. 2021. Subspace clustering for action recognition with covariance representations and temporal pruning. In 25th International Conference on Pattern Recognition (ICPR\u201921) . Milan 6035\u20136042. DOI:10.1109\/ICPR48806.2021.9412060","DOI":"10.1109\/ICPR48806.2021.9412060"},{"key":"e_1_3_1_151_2","volume-title":"Proceedings of the BMVC","author":"Paoletti Giancarlo","year":"2021","unstructured":"Giancarlo Paoletti, Jacopo Cavazza, Cigdem Beyan, and Alessio Del Bue. 2021. Unsupervised human action recognition with skeletal graph laplacian and self-supervised viewpoints invariance. In Proceedings of the BMVC."},{"key":"e_1_3_1_152_2","first-page":"19","volume-title":"Proc. ACM ICMI","author":"Park Sunghyun","year":"2012","unstructured":"Sunghyun Park, Jonathan Gratch, and Louis-Philippe Morency. 2012. I already know your answer: Using nonverbal behaviors to predict immediate outcomes in a dyadic negotiation. In Proc. ACM ICMI. 19\u201322."},{"key":"e_1_3_1_153_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2013.76"},{"key":"e_1_3_1_154_2","doi-asserted-by":"publisher","DOI":"10.1155\/2008\/324626"},{"key":"e_1_3_1_155_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2007.4286569"},{"key":"e_1_3_1_156_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.320"},{"key":"e_1_3_1_157_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-019-08561-6"},{"key":"e_1_3_1_158_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.529"},{"key":"e_1_3_1_159_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-014-0251-1"},{"key":"e_1_3_1_160_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACIIW.2019.8925179"},{"key":"e_1_3_1_161_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0092976"},{"key":"e_1_3_1_162_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2010-562"},{"key":"e_1_3_1_163_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_1_164_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460120.3484565"},{"key":"e_1_3_1_165_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-012-0101-0"},{"key":"e_1_3_1_166_2","doi-asserted-by":"publisher","DOI":"10.1145\/1891903.1891953"},{"key":"e_1_3_1_167_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2011.2181941"},{"key":"e_1_3_1_168_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2013-309"},{"key":"e_1_3_1_169_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2408437"},{"key":"e_1_3_1_170_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_1_171_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2013.6738732"},{"key":"e_1_3_1_172_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00238"},{"key":"e_1_3_1_173_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACIIW.2019.8925231"},{"key":"e_1_3_1_174_2","first-page":"1","article-title":"First impressions: A survey on vision-based apparent personality trait analysis","year":"2019","unstructured":"Julio C. S. Jacques Junior, Ya\u01e7mur G\u00fc\u00e7l\u00fct\u00fcrk, Marc P\u00e9rez, Umut G\u00fc\u00e7l\u00fc, Carlos Andujar, Xavier Bar\u00f3, Hugo Jair Escalante, Isabelle Guyon, Marcel A. J. van Gerven, Rob van Lier, and Sergio Escalera. 2019. First impressions: A survey on vision-based apparent personality trait analysis. IEEE Transactions on Affective Comput. (2019), 1\u20131.","journal-title":"IEEE Transactions on Affective Comput."},{"key":"e_1_3_1_175_2","first-page":"43","volume-title":"IEEE CVPRw","author":"Solera Francesco","year":"2015","unstructured":"Francesco Solera, Simone Calderara, and Rita Cucchiara. 2015. Learning to identify leaders in crowd. In IEEE CVPRw. 43\u201348."},{"key":"e_1_3_1_176_2","first-page":"27","volume-title":"Proc. ACM ICMI","author":"Song Yale","year":"2012","unstructured":"Yale Song, Louis-Philippe Morency, and Randall Davis. 2012. Multimodal human behavior analysis: Learning correlation and interaction across modalities. In Proc. ACM ICMI. 27\u201330."},{"key":"e_1_3_1_177_2","first-page":"329","volume-title":"Proceedings of the ACMMM","author":"Srivastava Ruchir","year":"2012","unstructured":"Ruchir Srivastava, Jiashi Feng, Sujoy Roy, Shuicheng Yan, and Terence Sim. 2012. Don\u2019t ask me what i\u2019m like, just watch and listen. In Proceedings of the ACMMM. New York, NY, USA, 329\u2013338."},{"key":"e_1_3_1_178_2","doi-asserted-by":"publisher","DOI":"10.1109\/PASSAT\/SocialCom.2011.28"},{"key":"e_1_3_1_179_2","doi-asserted-by":"publisher","DOI":"10.1145\/2072298.2071920"},{"key":"e_1_3_1_180_2","article-title":"Analyzing human\u2013human interactions: A survey","volume":"188","author":"Stergiou Alexandros","year":"2019","unstructured":"Alexandros Stergiou and Ronald Poppe. 2019. Analyzing human\u2013human interactions: A survey. CVIU 188 (2019).","journal-title":"CVIU"},{"key":"e_1_3_1_181_2","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806343"},{"key":"e_1_3_1_182_2","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2522862"},{"key":"e_1_3_1_183_2","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1007\/978-3-642-39226-9_40","volume-title":"Human Interface and the Management of Information. Information and Interaction for Learning, Culture, Collaboration and Business,","author":"Suzuki Noriko","year":"2013","unstructured":"Noriko Suzuki, Tosirou Kamiya, Ichiro Umata, Sadanori Ito, Shoichiro Iwasawa, Mamiko Sakata, and Katsunori Shimohara. 2013. Detection of division of labor in multiparty collaboration. In Human Interface and the Management of Information. Information and Interaction for Learning, Culture, Collaboration and Business,. 362\u2013371."},{"key":"e_1_3_1_184_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2370044"},{"key":"e_1_3_1_185_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2015.05.131"},{"key":"e_1_3_1_186_2","first-page":"717","volume-title":"ACM ICMI","author":"Thakur Sanket Kumar","year":"2021","unstructured":"Sanket Kumar Thakur, Cigdem Beyan, Pietro Morerio, and Alessio Del Bue. 2021. Predicting gaze from egocentric social interaction videos and IMU data. In ACM ICMI. 717\u2013722."},{"key":"e_1_3_1_187_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33863-2_36"},{"key":"e_1_3_1_188_2","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2014.2326873"},{"key":"e_1_3_1_189_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2012-125"},{"key":"e_1_3_1_190_2","first-page":"410","article-title":"Joint estimation of human pose and conversational groups from social scenes","volume":"126","author":"Varadarajan Jagannadan","year":"2018","unstructured":"Jagannadan Varadarajan, Ramanathan Subramanian, Samuel Rota Bul\u00f2, Narendra Ahuja, Oswald Lanz, and Elisa Ricci. 2018. Joint estimation of human pose and conversational groups from social scenes. Proceedings of the ICCV 126, 2\u20134 (2018), 410\u2013429.","journal-title":"Proceedings of the ICCV"},{"key":"e_1_3_1_191_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2011.6130339"},{"key":"e_1_3_1_192_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2330816"},{"key":"e_1_3_1_193_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2008.11.007"},{"key":"e_1_3_1_194_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSMC.2011.6083694"},{"key":"e_1_3_1_195_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"e_1_3_1_196_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2012.03.004"},{"key":"e_1_3_1_197_2","volume-title":"Proceedings of the ACM SIGKDD Int. Conf. on KDDM MLG Workshop","author":"Wang Yanbang","year":"2020","unstructured":"Yanbang Wang, Pan Li, Chongyang Bai, VS Subrahmanian, and Jure Leskovec. 2020. Generic representation learning for dynamic social interaction. In Proceedings of the ACM SIGKDD Int. Conf. on KDDM MLG Workshop."},{"key":"e_1_3_1_198_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2014-55"},{"key":"e_1_3_1_199_2","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388707"},{"key":"e_1_3_1_200_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2013-251"},{"key":"e_1_3_1_201_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2015-537"},{"key":"e_1_3_1_202_2","volume-title":"Proceedings of the ACM ICMI-MLMI","author":"Yasuyuki Sumi","year":"2010","unstructured":"Sumi Yasuyuki, Yano Masaharu, and Nishida Toyoaki. 2010. Analysis environment of conversational structure with nonverbal multimodal data. In Proceedings of the ACM ICMI-MLMI."},{"key":"e_1_3_1_203_2","volume-title":"Proceedings of the ACL","author":"Zadeh A.","year":"2018","unstructured":"A. Zadeh and P. Pu. 2018. Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. In Proceedings of the ACL."},{"key":"e_1_3_1_204_2","doi-asserted-by":"publisher","DOI":"10.1145\/1878039.1878048"},{"key":"e_1_3_1_205_2","volume-title":"Proceedings of the IEEE WACV","year":"2020","unstructured":"Lingyu Zhang, Indrani Bhattacharya, Mallory Morgan, Michael Foley, Christoph Riedl, Brooke Welles, and Richard Radke. 2020. Multiparty visual co-occurrences for estimating personality traits in group meetings. In Proceedings of the IEEE WACV."},{"key":"e_1_3_1_206_2","first-page":"385","volume-title":"Proceedings of the ACM ICMI","year":"2019","unstructured":"Lingyu Zhang, Mallory Morgan, Indrani Bhattacharya, Michael Foley, Jonas Braasch, Christoph Riedl, Brooke Foucault Welles, and Richard J. Radke. 2019. Improved visual focus of attention estimation and prosodic features for analyzing group interactions. In Proceedings of the ACM ICMI. 385\u2013394."},{"key":"e_1_3_1_207_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2020.2992394"},{"key":"e_1_3_1_208_2","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172997"},{"key":"e_1_3_1_209_2","doi-asserted-by":"publisher","DOI":"10.1145\/3264960"},{"key":"e_1_3_1_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/2671188.2749305"},{"key":"e_1_3_1_211_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2019-2087"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626516","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626516","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:50:15Z","timestamp":1750272615000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626516"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,25]]},"references-count":210,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3626516"],"URL":"https:\/\/doi.org\/10.1145\/3626516","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,25]]},"assertion":[{"value":"2021-06-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}