{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T02:12:19Z","timestamp":1775009539233,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"1s","license":[{"start":{"date-parts":[[2019,1,24]],"date-time":"2019-01-24T00:00:00Z","timestamp":1548288000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"UBImpressed Sinergia project","award":["CRSII2 147611"],"award-info":[{"award-number":["CRSII2 147611"]}]},{"name":"Japan Society for the Promotion of Science(JSPS) KAK-ENHI","award":["25730132, 15K00300, 25280076"],"award-info":[{"award-number":["25730132, 15K00300, 25280076"]}]},{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]},{"name":"SOBE Ambizione Fellowship","award":["PZ00P2 136811"],"award-info":[{"award-number":["PZ00P2 136811"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,1,31]]},"abstract":"<jats:p>This article proposes a novel feature-extraction framework for inferring impression personality traits, emergent leadership skills, communicative competence, and hiring decisions. The proposed framework extracts multimodal features, describing each participant\u2019s nonverbal activities. It captures intermodal and interperson relationships in interactions and captures how the target interactor generates nonverbal behavior when other interactors also generate nonverbal behavior. The intermodal and interperson patterns are identified as frequent co-occurring events based on clustering from multimodal sequences. The proposed framework is applied to the SONVB corpus, which is an audiovisual dataset collected from dyadic job interviews, and the ELEA audiovisual data corpus, which is a dataset collected from group meetings. We evaluate the framework on a binary classification task involving 15 impression variables from the two data corpora. The experimental results show that the model trained with co-occurrence features is more accurate than previous models for 14 out of 15 traits.<\/jats:p>","DOI":"10.1145\/3265754","type":"journal-article","created":{"date-parts":[[2019,1,28]],"date-time":"2019-01-28T14:01:39Z","timestamp":1548684099000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Modeling Dyadic and Group Impressions with Intermodal and Interperson Features"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9260-0403","authenticated-orcid":false,"given":"Shogo","family":"Okada","sequence":"first","affiliation":[{"name":"Japan Advanced Institute of Science and Technology (JAIST) and RIKEN Center for Advanced Intelligence Project (AIP), Japan"}]},{"given":"Laurent Son","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Idiap Research Institute, Martigny, Switzerland"}]},{"given":"Oya","family":"Aran","sequence":"additional","affiliation":[{"name":"De La Salle University Manila, Philippines"}]},{"given":"Daniel","family":"Gatica-Perez","sequence":"additional","affiliation":[{"name":"Idiap Research Institute and Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland"}]}],"member":"320","published-online":{"date-parts":[[2019,1,24]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2496269"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806238"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/182.358434"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2013.2284893"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2522848.2522859"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA\u201999)","author":"Aslam Javed","year":"1999","unstructured":"Javed Aslam , Katya Pelekhov , and Daniela Rus . 1999 . A practical clustering algorithm for static and dynamic information organization . In Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA\u201999) . Society for Industrial and Applied Mathematics, 51--60. https:\/\/dl.acm.org\/citation.cfm?id&equals;314530. Javed Aslam, Katya Pelekhov, and Daniela Rus. 1999. A practical clustering algorithm for static and dynamic information organization. In Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA\u201999). Society for Industrial and Applied Mathematics, 51--60. https:\/\/dl.acm.org\/citation.cfm?id&equals;314530."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2521348"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2522763"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/212094.212141"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388689"},{"key":"e_1_2_1_11_1","first-page":"993","article-title":"Latent Dirichlet allocation","author":"Blei David M.","year":"2003","unstructured":"David M. Blei , Andrew Y. Ng , and Michael I. Jordan . 2003 . Latent Dirichlet allocation . Journal of Machine Learning Research 3 , Jan (2003), 993 -- 1022 . David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research 3, Jan (2003), 993--1022.","journal-title":"Journal of Machine Learning Research 3"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820747"},{"key":"e_1_2_1_14_1","volume-title":"Curhan and Alex Pentland","author":"Jared","year":"2007","unstructured":"Jared R. Curhan and Alex Pentland . 2007 . Thin slices of negotiation: Predicting outcomes from conversational dynamics within the first 5 minutes.Journal of Applied Psychology 92, 3 (2007), 802. Jared R. Curhan and Alex Pentland. 2007. Thin slices of negotiation: Predicting outcomes from conversational dynamics within the first 5 minutes.Journal of Applied Psychology 92, 3 (2007), 802."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2009.01.004"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0092-6566(03)00046-1"},{"key":"e_1_2_1_17_1","volume-title":"Salakhutdinov","author":"Hinton Geoffrey E.","year":"2006","unstructured":"Geoffrey E. Hinton and Ruslan R . Salakhutdinov . 2006 . Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504--507. Geoffrey E. Hinton and Ruslan R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504--507."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/28.3-4.321"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070481.2070525"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388772"},{"key":"e_1_2_1_22_1","volume-title":"John and Sanjay Srivastava","author":"Oliver","year":"1999","unstructured":"Oliver P. John and Sanjay Srivastava . 1999 . The big five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of Personality: Theory and Research 2, 1999 (1999), 102--138. Oliver P. John and Sanjay Srivastava. 1999. The big five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of Personality: Theory and Research 2, 1999 (1999), 102--138."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.231"},{"key":"e_1_2_1_24_1","volume-title":"The Handbook of Brain Theory and Neural Networks, Michael A","author":"LeCun Yann","unstructured":"Yann LeCun and Yoshua Bengio . 1998. Convolutional Networks for Images, Speech, and Time Series . In The Handbook of Brain Theory and Neural Networks, Michael A . Arbib (Ed.). MIT Press , 255--258. Yann LeCun and Yoshua Bengio. 1998. Convolutional Networks for Images, Speech, and Time Series. In The Handbook of Brain Theory and Neural Networks, Michael A. Arbib (Ed.). MIT Press, 255--258."},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of International Conference on Augmented Cognition. Lawrence Erlbaum Associates, Inc (Acquired by CRC Press, Taylor 8 Francis Group), 6 pages.","author":"Madan Anmol","year":"2004","unstructured":"Anmol Madan , Ron Caneel , and Alex Pentland . 2004 . Voices of attraction . In Proceedings of International Conference on Augmented Cognition. Lawrence Erlbaum Associates, Inc (Acquired by CRC Press, Taylor 8 Francis Group), 6 pages. Anmol Madan, Ron Caneel, and Alex Pentland. 2004. Voices of attraction. In Proceedings of International Conference on Augmented Cognition. Lawrence Erlbaum Associates, Inc (Acquired by CRC Press, Taylor 8 Francis Group), 6 pages."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070481.2070485"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2663204.2663236"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388702"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915)","volume":"1","author":"Naim I.","unstructured":"I. Naim , M. I. Tanveer , D. Gildea , and M. E. Hoque . 2015. Automated prediction and analysis of job interview performance: The role of what you say and how you say it . In Proceedings of 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915) , Vol. 1 . IEEE, 1--6. I. Naim, M. I. Tanveer, D. Gildea, and M. E. Hoque. 2015. Automated prediction and analysis of job interview performance: The role of what you say and how you say it. In Proceedings of 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915), Vol. 1. IEEE, 1--6."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820764"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2307169"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388734"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2663204.2663248"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1006\/jvci.1995.1029"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820757"},{"key":"e_1_2_1_36_1","volume-title":"Predicting performance of collaborative storytelling using multimodal analysis. IEICE Transactions 99-D, 6","author":"Okada Shogo","year":"2016","unstructured":"Shogo Okada , Mi Hang , and Katsumi Nitta . 2016. Predicting performance of collaborative storytelling using multimodal analysis. IEICE Transactions 99-D, 6 ( 2016 ), 1462--1473. Shogo Okada, Mi Hang, and Katsumi Nitta. 2016. Predicting performance of collaborative storytelling using multimodal analysis. IEICE Transactions 99-D, 6 (2016), 1462--1473."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2396079"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1452392.1452404"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2638207"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/11677482_7"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-012-0101-0"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2011.2181941"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201913)","author":"Scherer S.","unstructured":"S. Scherer , G. Stratou , M. Mahmoud , J. Boberg , J. Gratch , A. Rizzo , and L. P. Morency . 2013. Automatic behavior descriptors for psychological disorder analysis . In Proceedings of 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201913) . IEEE, 1--8. S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, J. Gratch, A. Rizzo, and L. P. Morency. 2013. Automatic behavior descriptors for psychological disorder analysis. In Proceedings of 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201913). IEEE, 1--8."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388684"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2370044"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.5555\/1661445.1661647"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2007.902882"},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1--4. https:\/\/ieeexplore.ieee.org\/document\/5349466","author":"Vinciarelli A.","unstructured":"A. Vinciarelli , A. Dielmann , S. Favre , and H. Salamin . 2009. Canal9: A database of political debates for analysis of social interactions . In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1--4. https:\/\/ieeexplore.ieee.org\/document\/5349466 . A. Vinciarelli, A. Dielmann, S. Favre, and H. Salamin. 2009. Canal9: A database of political debates for analysis of social interactions. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1--4. https:\/\/ieeexplore.ieee.org\/document\/5349466."},{"key":"e_1_2_1_49_1","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 25--32","author":"Wu Zhong","year":"2009","unstructured":"Zhong Wu , Qifa Ke , M. Isard , and Jian Sun . 2009 . Bundling features for large scale partial-duplicate web image search . In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 25--32 . Zhong Wu, Qifa Ke, M. Isard, and Jian Sun. 2009. Bundling features for large scale partial-duplicate web image search. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 25--32."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/2832747.2832806"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2411433"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1180995.1181003"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2011.2128333"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2417573"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2016.2607172"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3265754","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3265754","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:49Z","timestamp":1750210789000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3265754"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,24]]},"references-count":55,"journal-issue":{"issue":"1s","published-print":{"date-parts":[[2019,1,31]]}},"alternative-id":["10.1145\/3265754"],"URL":"https:\/\/doi.org\/10.1145\/3265754","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,24]]},"assertion":[{"value":"2017-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}