{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T16:00:37Z","timestamp":1772553637834,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2019,5,31]],"date-time":"2019-05-31T00:00:00Z","timestamp":1559260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"CSC\/Queen Mary joint PhD scholarship"},{"name":"Nvidia corporation with the donation of a TitanX GPU"},{"name":"EPSRC under its IDEAS Factory Sandpits call on Digital Personhood","award":["EP\/L00416X\/1"],"award-info":[{"award-number":["EP\/L00416X\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,5,31]]},"abstract":"<jats:p>Recognition and analysis of human affect has been researched extensively within the field of computer science in the past two decades. However, most of the past research in automatic analysis of human affect has focused on the recognition of affect displayed by people in individual settings and little attention has been paid to the analysis of the affect expressed in group settings. In this article, we first analyze the affect expressed by each individual in terms of arousal and valence dimensions in both individual and group videos and then propose methods to recognize the contextual information, i.e., whether a person is alone or in-a-group by analyzing their face and body behavioral cues. For affect analysis, we first devise affect recognition models separately in individual and group videos and then introduce a cross-condition affect recognition model that is trained by combining the two different types of data. We conduct a set of experiments on two datasets that contain both individual and group videos. Our experiments show that (1) the proposed Volume Quantized Local Zernike Moments Fisher Vector outperforms other unimodal features in affect analysis; (2) the temporal learning model, Long-Short Term Memory Networks, works better than the static learning model, Support Vector Machine; (3) decision fusion helps to improve affect recognition, indicating that body behaviors carry emotional information that is complementary rather than redundant to the emotion content in facial behaviors; and (4) it is possible to predict the context, i.e., whether a person is alone or in-a-group, using their non-verbal behavioral cues.<\/jats:p>","DOI":"10.1145\/3321509","type":"journal-article","created":{"date-parts":[[2019,6,11]],"date-time":"2019-06-11T13:28:16Z","timestamp":1560259696000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Alone versus In-a-group"],"prefix":"10.1145","volume":"15","author":[{"given":"Wenxuan","family":"Mou","sequence":"first","affiliation":[{"name":"Queen Mary University of London"}]},{"given":"Hatice","family":"Gunes","sequence":"additional","affiliation":[{"name":"University of Cambridge"}]},{"given":"Ioannis","family":"Patras","sequence":"additional","affiliation":[{"name":"Queen Mary University of London"}]}],"member":"320","published-online":{"date-parts":[[2019,6,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0025827"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1177\/0963721412438352"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2014.7025852"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2513401"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133949"},{"key":"e_1_2_1_7_1","volume-title":"Cohn and Fernando De la Torre","author":"Jeffrey","year":"2014","unstructured":"Jeffrey F. Cohn and Fernando De la Torre . 2014 . Automated face analysis for affective. Oxford Handbook Affect. Comput . (2014), 131. Jeffrey F. Cohn and Fernando De la Torre. 2014. Automated face analysis for affective. Oxford Handbook Affect. Comput. (2014), 131."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2515606"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310555"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2006.2004"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2012.26"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2015.7344580"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2397456"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3143004"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2997638"},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Abhinav Dhall Jyoti Joshi Ibrahim Radwan and Roland Goecke. 2012. Finding happiest moments in a social context. http:\/\/users.cecs.anu.edu.au\/&sim;adhall\/Dhall_Joshi_Radwan_Goecke_ACCV_2012.pdf.  Abhinav Dhall Jyoti Joshi Ibrahim Radwan and Roland Goecke. 2012. Finding happiest moments in a social context. http:\/\/users.cecs.anu.edu.au\/&sim;adhall\/Dhall_Joshi_Radwan_Goecke_ACCV_2012.pdf.","DOI":"10.1007\/978-3-642-37444-9_48"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2015.7163151"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2997632"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206828"},{"key":"e_1_2_1_20_1","volume-title":"Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, and Dong Hyun Lee.","author":"Goodfellow Ian J.","year":"2013","unstructured":"Ian J. Goodfellow , Dumitru Erhan , Pierre Luc Carrier , Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, and Dong Hyun Lee. 2013 . Challenges in representation learning: A report on three machine learning contests. Neural Netw . (2013). Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, and Dong Hyun Lee. 2013. Challenges in representation learning: A report on three machine learning contests. Neural Netw. (2013)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.4018\/jse.2010101605"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2011.5771357"},{"key":"e_1_2_1_23_1","volume-title":"Bodily expression for automatic affect recognition. Emotion Recogn. Pattern Anal. Approach","author":"Gunes Hatice","year":"2015","unstructured":"Hatice Gunes , Caifeng Shan , Shizhi Chen , and YingLi Tian . 2015. Bodily expression for automatic affect recognition. Emotion Recogn. Pattern Anal. Approach ( 2015 ). Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian. 2015. Bodily expression for automatic affect recognition. Emotion Recogn. Pattern Anal. Approach (2015)."},{"key":"e_1_2_1_24_1","volume-title":"Picard","author":"Hernandez Javier","year":"2012","unstructured":"Javier Hernandez , Mohammed Hoque , Will Drevo , and Rosalind W . Picard . 2012 . Mood meter: Counting smiles in the wild. https:\/\/affect.media.mit.edu\/pdfs\/12.Hernandez-Hoque-Drevo-Picard-MoodMeter-Ubicomp.pdf. Javier Hernandez, Mohammed Hoque, Will Drevo, and Rosalind W. Picard. 2012. Mood meter: Counting smiles in the wild. https:\/\/affect.media.mit.edu\/pdfs\/12.Hernandez-Hoque-Drevo-Picard-MoodMeter-Ubicomp.pdf."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_26_1","volume-title":"Multi-modal framework for analyzing the affect of a group of people","author":"Huang Xiaohua","year":"2018","unstructured":"Xiaohua Huang , Abhinav Dhall , Roland Goecke , Matti Pietikainen , and Guoying Zhao . 2018. Multi-modal framework for analyzing the affect of a group of people . IEEE Trans. Multimedia ( 2018 ). Xiaohua Huang, Abhinav Dhall, Roland Goecke, Matti Pietikainen, and Guoying Zhao. 2018. Multi-modal framework for analyzing the affect of a group of people. IEEE Trans. Multimedia (2018)."},{"key":"e_1_2_1_27_1","unstructured":"Xiaohua Huang Abhinav Dhall Xin Liu Guoying Zhao Jingang Shi Roland Goecke and Matti Pietikainen. 2016. Analyzing the affect of a group of people using multi-modal framework. arXiv preprint arXiv:1610.03640.  Xiaohua Huang Abhinav Dhall Xin Liu Guoying Zhao Jingang Shi Roland Goecke and Matti Pietikainen. 2016. Analyzing the affect of a group of people using multi-modal framework. arXiv preprint arXiv:1610.03640."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.29.34"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661806.2661817"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33191-6_36"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2013.29"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2012.16"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.15"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2012.10.002"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2997636"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.817413"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0507650102"},{"key":"e_1_2_1_38_1","volume-title":"Nicu Sebe, and Ioannis Patras.","author":"Miranda-Correa Juan Abdon","year":"2017","unstructured":"Juan Abdon Miranda-Correa , Mojtaba Khomami Abadi , Nicu Sebe, and Ioannis Patras. 2017 . AMIGOS : A dataset for mood, personality and affect research on Individuals and GrOupS. arXiv preprint arXiv:1702.02510. Juan Abdon Miranda-Correa, Mojtaba Khomami Abadi, Nicu Sebe, and Ioannis Patras. 2017. AMIGOS: A dataset for mood, personality and affect research on Individuals and GrOupS. arXiv preprint arXiv:1702.02510."},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Louis-Philippe Morency. 2013. The role of context in affective behavior understanding. Soc. Emot. Nature Artif. (2013).  Louis-Philippe Morency. 2013. The role of context in affective behavior understanding. Soc. Emot. Nature Artif. (2013).","DOI":"10.1093\/acprof:oso\/9780195387643.003.0009"},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG\u201915)","author":"Mou Wenxuan","year":"2015","unstructured":"Wenxuan Mou , Oya Celiktutan , and Hatice Gunes . 2015 . Group-level arousal and valence recognition in static images: Face, body and context . In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG\u201915) . Wenxuan Mou, Oya Celiktutan, and Hatice Gunes. 2015. Group-level arousal and valence recognition in static images: Face, body and context. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG\u201915)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2967276"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2016.185"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2927006.2927012"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2005.859075"},{"key":"e_1_2_1_45_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. https:\/\/openreview.net\/pdf?id=BJJsrmfCZ.  Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. https:\/\/openreview.net\/pdf?id=BJJsrmfCZ."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133953"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2808196.2811642"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2013.6553805"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0636-x"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2366127"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.27.108"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2531741"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.25"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3143008"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.908962"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2988257.2988258"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1037\/1528-3542.7.3.487"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2666242.2666245"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995407"},{"key":"e_1_2_1_60_1","volume-title":"Int. J. Comput. Vis.","author":"Wang Heng","year":"2013","unstructured":"Heng Wang , Alexander Kl\u00e4ser , Cordelia Schmid , and Cheng-Lin Liu . 2013. Dense trajectories and motion boundary descriptors for action recognition . Int. J. Comput. Vis. ( 2013 ). Heng Wang, Alexander Kl\u00e4ser, Cordelia Schmid, and Cheng-Lin Liu. 2013. Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vis. (2013)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.441"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.75"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the British Machine and Vision Conference (BMVC\u201915)","author":"Yang Heng","year":"2015","unstructured":"Heng Yang , Wenxuan Mou , Yichi Zhang , Ioannis Patras , Hatice Gunes , and Peter Robinson . 2015 . Pose-invariant 3D face alignment . Proceedings of the British Machine and Vision Conference (BMVC\u201915) . Heng Yang, Wenxuan Mou, Yichi Zhang, Ioannis Patras, Hatice Gunes, and Peter Robinson. 2015. Pose-invariant 3D face alignment. Proceedings of the British Machine and Vision Conference (BMVC\u201915)."},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the 19th Conference on Human Vision and Electronic Imaging.","author":"Zhu Yi","unstructured":"Yi Zhu , Ingrid Heynderickx , and Judith A. Redi . 2014. Alone or together: Measuring users\u2019 viewing experience in different social contexts . In Proceedings of the 19th Conference on Human Vision and Electronic Imaging. Yi Zhu, Ingrid Heynderickx, and Judith A. Redi. 2014. Alone or together: Measuring users\u2019 viewing experience in different social contexts. In Proceedings of the 19th Conference on Human Vision and Electronic Imaging."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321509","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3321509","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:54:38Z","timestamp":1750204478000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321509"}},"subtitle":["A Multi-modal Framework for Automatic Affect Recognition"],"short-title":[],"issued":{"date-parts":[[2019,5,31]]},"references-count":64,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,5,31]]}},"alternative-id":["10.1145\/3321509"],"URL":"https:\/\/doi.org\/10.1145\/3321509","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,31]]},"assertion":[{"value":"2017-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-06-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}