{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T08:19:44Z","timestamp":1768033184391,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":45,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,18]],"date-time":"2021-10-18T00:00:00Z","timestamp":1634515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,18]]},"DOI":"10.1145\/3461615.3485408","type":"proceedings-article","created":{"date-parts":[[2021,12,18]],"date-time":"2021-12-18T04:57:40Z","timestamp":1639803460000},"page":"202-210","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Crossmodal Clustered Contrastive Learning: Grounding of Spoken Language to Gesture"],"prefix":"10.1145","author":[{"given":"Dong Won","family":"Lee","sequence":"first","affiliation":[{"name":"Machine Learning Department, Carnegie Mellon University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaitanya","family":"Ahuja","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Louis-Philippe","family":"Morency","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,12,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.170"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58523-5_15"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340555.3353725"},{"key":"e_1_3_2_1_4_1","unstructured":"Alexei Baevski Henry Zhou Abdelrahman Mohamed and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. arXiv preprint arXiv:2006.11477(2020).  Alexei Baevski Henry Zhou Abdelrahman Mohamed and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. arXiv preprint arXiv:2006.11477(2020)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1162\/pres.15.4.359"},{"key":"e_1_3_2_1_6_1","volume-title":"BEAT: the Behavior Expression Animation Toolkit. In the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH \u201901). 477\u2013486. https:\/\/doi.org\/10.1145\/383259.383315","author":"Cassell Justine","unstructured":"Justine Cassell , Hannes\u00a0H\u00f6gni Vilhj\u00e1lmsson , and Timothy Bickmore . 2001. BEAT: the Behavior Expression Animation Toolkit. In the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH \u201901). 477\u2013486. https:\/\/doi.org\/10.1145\/383259.383315 Justine Cassell, Hannes\u00a0H\u00f6gni Vilhj\u00e1lmsson, and Timothy Bickmore. 2001. BEAT: the Behavior Expression Animation Toolkit. In the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH \u201901). 477\u2013486. https:\/\/doi.org\/10.1145\/383259.383315"},{"key":"e_1_3_2_1_7_1","unstructured":"Ting Chen Simon Kornblith Mohammad Norouzi and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709(2020).  Ting Chen Simon Kornblith Mohammad Norouzi and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709(2020)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/2615731.2615857"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-21996-7_17"},{"key":"e_1_3_2_1_10_1","unstructured":"Marco Cuturi and Mathieu Blondel. 2017. Soft-DTW: a differentiable loss function for time-series. arXiv preprint arXiv:1703.01541(2017).  Marco Cuturi and Mathieu Blondel. 2017. Soft-DTW: a differentiable loss function for time-series. arXiv preprint arXiv:1703.01541(2017)."},{"key":"e_1_3_2_1_11_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018)."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2957365"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Ylva Ferstl Michael Neff and Rachel McDonnell. 2019. Multi-objective adversarial gesture generation. (2019).  Ylva Ferstl Michael Neff and Rachel McDonnell. 2019. Multi-objective adversarial gesture generation. (2019).","DOI":"10.1145\/3359566.3360053"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00361"},{"key":"e_1_3_2_1_15_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672\u20132680.  Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672\u20132680."},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 297\u2013304","author":"Gutmann Michael","year":"2010","unstructured":"Michael Gutmann and Aapo Hyv\u00e4rinen . 2010 . Noise-contrastive estimation: A new estimation principle for unnormalized statistical models . In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 297\u2013304 . Michael Gutmann and Aapo Hyv\u00e4rinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 297\u2013304."},{"key":"e_1_3_2_1_17_1","volume-title":"Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics.Journal of Machine Learning Research 13, 2","author":"Gutmann U","year":"2012","unstructured":"Michael\u00a0 U Gutmann and Aapo Hyv\u00e4rinen . 2012. Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics.Journal of Machine Learning Research 13, 2 ( 2012 ). Michael\u00a0U Gutmann and Aapo Hyv\u00e4rinen. 2012. Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics.Journal of Machine Learning Research 13, 2 (2012)."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267851.3267878"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9414460"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Adam Kendon. 1980. Gesture and speech: two aspects of the process of utterance. In Nonverbal Communication and Language M. R. Key (Ed.). 207\u2013227.  Adam Kendon. 1980. Gesture and speech: two aspects of the process of utterance. In Nonverbal Communication and Language M. R. Key (Ed.). 207\u2013227.","DOI":"10.1515\/9783110813098.207"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Taras Kucherenko Dai Hasegawa Gustav\u00a0Eje Henter Naoshi Kaneko and Hedvig Kjellstr\u00f6m. 2019. Analyzing Input and Output Representations for Speech-Driven Gesture Generation. arXiv preprint arXiv:1903.03369(2019).  Taras Kucherenko Dai Hasegawa Gustav\u00a0Eje Henter Naoshi Kaneko and Hedvig Kjellstr\u00f6m. 2019. Analyzing Input and Output Representations for Speech-Driven Gesture Generation. arXiv preprint arXiv:1903.03369(2019).","DOI":"10.1145\/3308532.3329472"},{"key":"e_1_3_2_1_23_1","volume-title":"Gesticulator: A framework for semantically-aware speech-driven gesture generation. arXiv preprint arXiv:2001.09326(2020).","author":"Kucherenko Taras","year":"2020","unstructured":"Taras Kucherenko , Patrik Jonell , Sanne van Waveren , Gustav\u00a0Eje Henter , Simon Alexanderson , Iolanda Leite , and Hedvig Kjellstr\u00f6m . 2020 . Gesticulator: A framework for semantically-aware speech-driven gesture generation. arXiv preprint arXiv:2001.09326(2020). Taras Kucherenko, Patrik Jonell, Sanne van Waveren, Gustav\u00a0Eje Henter, Simon Alexanderson, Iolanda Leite, and Hedvig Kjellstr\u00f6m. 2020. Gesticulator: A framework for semantically-aware speech-driven gesture generation. arXiv preprint arXiv:2001.09326(2020)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/11821830_20"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778861"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618518"},{"key":"e_1_3_2_1_27_1","volume-title":"From embodied metaphors to metaphoric gestures. CogSci","author":"Lhommet Margot","year":"2016","unstructured":"Margot Lhommet and Stacy Marsella . 2016. From embodied metaphors to metaphoric gestures. CogSci ( 2016 ), 788\u2013793. Margot Lhommet and Stacy Marsella. 2016. From embodied metaphors to metaphoric gestures. CogSci (2016), 788\u2013793."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9778"},{"key":"e_1_3_2_1_29_1","unstructured":"Junnan Li Pan Zhou Caiming Xiong Richard Socher and Steven\u00a0CH Hoi. 2020. Prototypical contrastive learning of unsupervised representations. arXiv preprint arXiv:2005.04966(2020).  Junnan Li Pan Zhou Caiming Xiong Richard Socher and Steven\u00a0CH Hoi. 2020. Prototypical contrastive learning of unsupervised representations. arXiv preprint arXiv:2005.04966(2020)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2485895.2485900"},{"key":"e_1_3_2_1_31_1","volume-title":"Hand and mind: What gestures reveal about thought","author":"McNeill David","unstructured":"David McNeill . 1992. Hand and mind: What gestures reveal about thought . University of Chicago Press. David McNeill. 1992. Hand and mind: What gestures reveal about thought. University of Chicago Press."},{"key":"e_1_3_2_1_32_1","unstructured":"Aaron van\u00a0den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748(2018).  Aaron van\u00a0den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748(2018)."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58545-7_19"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2008.04.009"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40480-1_16"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Najmeh Sadoughi and Carlos Busso. 2018. Novel Realizations of Speech-Driven Head Movements with Generative Adversarial Networks. 6169\u20136173. https:\/\/doi.org\/10.1109\/ICASSP.2018.8461967  Najmeh Sadoughi and Carlos Busso. 2018. Novel Realizations of Speech-Driven Head Movements with Generative Adversarial Networks. 6169\u20136173. https:\/\/doi.org\/10.1109\/ICASSP.2018.8461967","DOI":"10.1109\/ICASSP.2018.8461967"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.70797"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Steffen Schneider Alexei Baevski Ronan Collobert and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862(2019).  Steffen Schneider Alexei Baevski Ronan Collobert and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862(2019).","DOI":"10.21437\/Interspeech.2019-1873"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00790"},{"key":"e_1_3_2_1_41_1","volume-title":"Cobra: Contrastive bi-modal representation algorithm. arXiv preprint arXiv:2005.03687(2020).","author":"Udandarao Vishaal","year":"2020","unstructured":"Vishaal Udandarao , Abhishek Maiti , Deepak Srivatsav , Suryatej\u00a0Reddy Vyalla , Yifang Yin , and Rajiv\u00a0Ratn Shah . 2020 . Cobra: Contrastive bi-modal representation algorithm. arXiv preprint arXiv:2005.03687(2020). Vishaal Udandarao, Abhishek Maiti, Deepak Srivatsav, Suryatej\u00a0Reddy Vyalla, Yifang Yin, and Rajiv\u00a0Ratn Shah. 2020. Cobra: Contrastive bi-modal representation algorithm. arXiv preprint arXiv:2005.03687(2020)."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Petra Wagner Zofia Malisz and Stefan Kopp. 2014. Gesture and speech in interaction: An overview.  Petra Wagner Zofia Malisz and Stefan Kopp. 2014. Gesture and speech in interaction: An overview.","DOI":"10.1016\/j.specom.2013.09.008"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Yuyu Xu Catherine Pelachaud and Stacy Marsella. 2014. Compound gesture generation: A model based on ideational units. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8637 LNAI (2014) 477\u2013491. https:\/\/doi.org\/10.1007\/978-3-319-09767-1_58  Yuyu Xu Catherine Pelachaud and Stacy Marsella. 2014. Compound gesture generation: A model based on ideational units. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8637 LNAI (2014) 477\u2013491. https:\/\/doi.org\/10.1007\/978-3-319-09767-1_58","DOI":"10.1007\/978-3-319-09767-1_58"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417838"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793720"}],"event":{"name":"ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","location":"Montreal QC Canada","acronym":"ICMI '21","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Companion Publication of the 2021 International Conference on Multimodal Interaction"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461615.3485408","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461615.3485408","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:45:10Z","timestamp":1750268710000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461615.3485408"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,18]]},"references-count":45,"alternative-id":["10.1145\/3461615.3485408","10.1145\/3461615"],"URL":"https:\/\/doi.org\/10.1145\/3461615.3485408","relation":{},"subject":[],"published":{"date-parts":[[2021,10,18]]},"assertion":[{"value":"2021-12-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}