{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T05:55:35Z","timestamp":1788501335745,"version":"build-2803163510"},"reference-count":81,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T00:00:00Z","timestamp":1680134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>One of the main goals of robotics and intelligent agent research is to enable them to communicate with humans in physically situated settings. Human communication consists of both verbal and non-verbal modes. Recent studies in enabling communication for intelligent agents have focused on verbal modes, i.e., language and speech. However, in a situated setting the non-verbal mode is crucial for an agent to adapt flexible communication strategies. In this work, we focus on learning to generate non-verbal communicative expressions in situated embodied interactive agents. Specifically, we show that an agent can learn pointing gestures in a physically simulated environment through a combination of imitation and reinforcement learning that achieves high motion naturalness and high referential accuracy. We compared our proposed system against several baselines in both subjective and objective evaluations. The subjective evaluation is done in a virtual reality setting where an embodied referential game is played between the user and the agent in a shared 3D space, a setup that fully assesses the communicative capabilities of the generated gestures. The evaluations show that our model achieves a higher level of referential accuracy and motion naturalness compared to a state-of-the-art supervised learning motion synthesis model, showing the promise of our proposed system that combines imitation and reinforcement learning for generating communicative gestures. Additionally, our system is robust in a physically-simulated environment thus has the potential of being applied to robots.<\/jats:p>","DOI":"10.3389\/frobt.2023.1110534","type":"journal-article","created":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T01:17:13Z","timestamp":1680139033000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Learning to generate pointing gestures in situated embodied conversational agents"],"prefix":"10.3389","volume":"10","author":[{"given":"Anna","family":"Deichler","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siyang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon","family":"Alexanderson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonas","family":"Beskow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,3,30]]},"reference":[{"key":"B1","first-page":"67","article-title":"Human pointing as a robot directive","author":"Abidi","year":"2013"},{"key":"B2","first-page":"1884","volume-title":"No gestures left behind: Learning relationships between spoken language and freeform gestures","author":"Ahuja","year":"2020"},{"key":"B3","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1111\/cgf.13946","article-title":"Style-controllable speech-driven gesture synthesis using normalising flows","volume":"39","author":"Alexanderson","year":"2020","journal-title":"Comput. Graph. Forum"},{"key":"B4","first-page":"2571","article-title":"Looking coordinated: Bidirectional gaze mechanisms for collaborative interaction with virtual characters","author":"Andrist","year":"2017"},{"key":"B5","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1016\/j.specom.2010.02.015","article-title":"Gaze, conversational agents and face-to-face communication","volume":"52","author":"Bailly","year":"2010","journal-title":"Speech Commun."},{"key":"B6","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1111\/j.0956-7976.2004.00694.x","article-title":"Using pointing and describing to achieve joint focus of attention in dialogue","volume":"15","author":"Bangerter","year":"2004","journal-title":"Psychol. Sci."},{"key":"B7","volume-title":"Sprachtheorie","author":"B\u00fchler","year":"1934"},{"key":"B8","volume-title":"Exploring zero-shot emergent communication in embodied multi-agent populations","author":"Bullard","year":"2020"},{"key":"B9","first-page":"413","volume-title":"Animated conversation: Rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents","author":"Cassell","year":"1994"},{"key":"B10","first-page":"477","volume-title":"Beat: The behavior expression animation toolkit","author":"Cassell","year":"2001"},{"key":"B11","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.ins.2014.03.104","article-title":"A developmental approach to robotic pointing via human\u2013robot interaction","volume":"283","author":"Chao","year":"2014","journal-title":"Inf. Sci."},{"key":"B12","first-page":"1385","article-title":"Yourefit: Embodied reference understanding with language and gesture","author":"Chen","year":"2021"},{"key":"B13","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1057\/9780230524125_2","article-title":"Changing ideas about reference","volume-title":"Experimental pragmatics","author":"Clark","year":"2004"},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.31234\/osf.io\/2vxft","volume-title":"Fifteen ways of looking at a pointing gesture","author":"Cooperrider","year":"2020"},{"key":"B15","article-title":"Towards context-aware human-like pointing gestures with rl motion imitation","author":"Deichler","year":"2022"},{"key":"B16","doi-asserted-by":"publisher","first-page":"555265","DOI":"10.3389\/fpsyg.2020.555265","article-title":"Demonstratives in spatial language and social interaction: An interdisciplinary review","volume":"11","author":"Diessel","year":"2020","journal-title":"Front. Psychol."},{"key":"B17","volume-title":"Nice: Non-linear independent components estimation","author":"Dinh","year":"2014"},{"key":"B18","volume-title":"Density estimation using real nvp","author":"Dinh","year":"2016"},{"key":"B19","first-page":"1329","article-title":"Benchmarking deep reinforcement learning for continuous control","author":"Duan","year":"2016"},{"key":"B20","doi-asserted-by":"crossref","DOI":"10.1109\/IROS47612.2022.9981973","volume-title":"Adversarial motion priors make good substitutes for complex reward functions","author":"Escontrela","year":"2022"},{"key":"B21","first-page":"271","article-title":"Embodied collaborative referring expression generation in situated human-robot interaction","author":"Fang","year":"2015"},{"key":"B22","first-page":"93","article-title":"Investigating the use of recurrent motion modelling for speech gesture generation","author":"Ferstl","year":"2018"},{"key":"B23","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/j.cag.2020.04.007","article-title":"Adversarial gesture generation with realistic gesture phasing","volume":"89","author":"Ferstl","year":"2020","journal-title":"Comput. Graph."},{"key":"B24","volume-title":"Dialfred: Dialogue-enabled agents for embodied instruction following","author":"Gao","year":"2022"},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2019.00361","volume-title":"Learning individual styles of conversational gesture","author":"Ginosar","year":"2019"},{"key":"B26","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1007\/978-3-642-34103-8_38","article-title":"Studies on grounding with gaze and pointing gestures in human-robot-interaction","volume-title":"International conference on social robotics","author":"H\u00e4ring","year":"2012"},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1145\/3267851.3267878","volume-title":"Evaluation of speech-to-gesture generation using bi-directional LSTM network","author":"Hasegawa","year":"2018"},{"key":"B28","volume-title":"Emergence of locomotion behaviours in rich environments","author":"Heess","year":"2017"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3414685.3417836","article-title":"Moglow: Probabilistic and controllable motion synthesis using normalising flows","volume":"39","author":"Henter","year":"2020","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B30","article-title":"Generative adversarial imitation learning","volume-title":"Advances in neural information processing systems 29","author":"Ho","year":"2016"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3072959.3073663","article-title":"Phase-functioned neural networks for character control","volume":"36","author":"Holden","year":"2017","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B32","first-page":"217","article-title":"Legible robot pointing","author":"Holladay","year":"2014"},{"key":"B33","first-page":"57","article-title":"Learning-based modeling of multimodal behaviors for humanlike robots","author":"Huang","year":"2014"},{"key":"B34","article-title":"Caesar: An embodied simulator for generating multimodal referring expression datasets","author":"Islam","year":"2022"},{"key":"B35","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780199215805.001.0001","volume-title":"Reference in discourse","author":"Kibrik","year":"2011"},{"key":"B36","article-title":"Glow: Generative flow with invertible 1x1 convolutions","volume-title":"Advances in neural information processing systems 31","author":"Kingma","year":"2018"},{"key":"B37","doi-asserted-by":"crossref","first-page":"9","DOI":"10.4324\/9781410607744-5","article-title":"Pointing: A foundational building block of human communication","volume-title":"Pointing","author":"Kita","year":"2003"},{"key":"B38","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1002\/cav.6","article-title":"Synthesizing multimodal utterances for conversational agents","volume":"15","author":"Kopp","year":"2004","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"B39","article-title":"The finemotion entry to the genea challenge 2020","volume-title":"Proc. GENEA workshop","author":"Korzun","year":"2020"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1300","DOI":"10.1080\/10447318.2021.1883883","article-title":"Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation","volume":"37","author":"Kucherenko","year":"","journal-title":"Int. J. Human\u2013Computer Interact."},{"key":"B41","first-page":"242","article-title":"Gesticulator: A framework for semantically-aware speech-driven gesture generation","author":"Kucherenko","year":"2020"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.1145\/3397481.3450692","article-title":"A large, crowdsourced evaluation of gesture generation systems on common data: The genea challenge 2020","author":"Kucherenko","year":""},{"key":"B43","article-title":"Emergence of linguistic communication from referential games with symbolic and pixel input","volume-title":"Iclr","author":"Lazaridou","year":"2018"},{"key":"B44","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1080\/088395199117324","article-title":"Deictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents","volume":"13","author":"Lester","year":"1999","journal-title":"Appl. Artif. Intell."},{"key":"B45","volume-title":"Convention: A philosophical study","author":"Lewis","year":"1969"},{"key":"B46","first-page":"31","article-title":"Speech-based gesture generation for robots and embodied agents: A scoping review","author":"Liu","year":"2021"},{"key":"B47","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.pragma.2014.12.013","article-title":"Pointing and reference reconsidered","volume":"77","author":"L\u00fccking","year":"2015","journal-title":"J. Pragmat."},{"key":"B48","doi-asserted-by":"crossref","DOI":"10.24963\/ijcai.2019\/880","volume-title":"A survey of reinforcement learning informed by natural language","author":"Luketina","year":"2019"},{"key":"B49","volume-title":"Interactive language: Talking to robots in real time","author":"Lynch","year":"2022"},{"key":"B50","volume-title":"Isaac gym: High performance gpu-based physics simulation for robot learning","author":"Makoviychuk","year":"2021"},{"key":"B51","first-page":"25","volume-title":"Virtual character performance from speech","author":"Marsella","year":"2013"},{"key":"B52","doi-asserted-by":"publisher","first-page":"9051","DOI":"10.1609\/aaai.v28i1.9051","article-title":"Learning from unscripted deictic gesture and language for human-robot interactions","volume":"28","author":"Matuszek","year":"2014","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B53","volume-title":"Hand and mind 1","author":"McNeill","year":"1992"},{"key":"B54","volume-title":"Learning human behaviors from motion capture by adversarial imitation","author":"Merel","year":"2017"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3418302","article-title":"Design of hesitation gestures for nonverbal human-robot negotiation of conflicts","volume":"10","author":"Moon","year":"2021","journal-title":"ACM Trans. Human-Robot Interact. (THRI)"},{"key":"B56","first-page":"69","article-title":"Nonverbal leakage in robots: Communication of intentions through seemingly unintentional behavior","author":"Mutlu","year":"2009"},{"key":"B57","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2010.5654322","article-title":"Synchronized gesture and speech production for humanoid robots","author":"Ng-Thow-Hing","year":"2010"},{"key":"B58","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1109\/38.851755","article-title":"Design of a virtual human presenter","volume":"20","author":"Noma","year":"2000","journal-title":"IEEE Comput. Graph. Appl."},{"key":"B59","doi-asserted-by":"publisher","first-page":"2017","DOI":"10.1609\/aaai.v36i2.20097","article-title":"Teach: Task-driven embodied agents that chat","volume":"36","author":"Padmakumar","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B60","volume-title":"A social and neurobiological approach to pointing in speech and gesture","author":"Peeters","year":"2015"},{"key":"B61","doi-asserted-by":"publisher","first-page":"409","DOI":"10.3758\/s13423-020-01822-8","article-title":"A conceptual framework for the study of demonstrative reference","volume":"28","author":"Peeters","year":"2021","journal-title":"Psychonomic Bull. Rev."},{"key":"B62","doi-asserted-by":"publisher","first-page":"222","DOI":"10.3389\/fpsyg.2016.00222","article-title":"This and that revisited: A social and multimodal approach to spatial demonstratives","volume":"7","author":"Peeters","year":"2016","journal-title":"Front. Psychol."},{"key":"B63","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"Deepmimic: Example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng","year":"2018","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B64","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3476576.3476723","article-title":"Amp: Adversarial motion priors for stylized physics-based character control","volume":"40","author":"Peng","year":"2021","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B65","first-page":"9982","article-title":"Reverie: Remote embodied visual referring expression in real indoor environments","author":"Qi","year":"2020"},{"key":"B66","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1080\/088395199117315","article-title":"Animated agents for procedural training in virtual reality: Perception, cognition, and motor control","volume":"13","author":"Rickel","year":"1999","journal-title":"Appl. Artif. Intell."},{"key":"B67","first-page":"342","article-title":"Robot deictics: How gesture and context shape referential communication","author":"Saupp\u00e9","year":"2014"},{"key":"B68","first-page":"1","article-title":"Focusing computational visual attention in multi-modal human-robot interaction","author":"Schauerte","year":"2010"},{"key":"B69","volume-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017"},{"key":"B70","doi-asserted-by":"crossref","DOI":"10.1109\/DICTA.2015.7371296","article-title":"Probabilistic detection of pointing directions for human-robot interaction","author":"Shukla","year":"2015"},{"key":"B71","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1109\/mis.2001.956077","article-title":"Language games for autonomous robots","volume":"16","author":"Steels","year":"2001","journal-title":"IEEE Intell. Syst."},{"key":"B72","volume-title":"Imitation learning of robot policies by combining language, vision and demonstration","author":"Stepputtis","year":"2019"},{"key":"B73","first-page":"1441","article-title":"Natural deictic communication with humanoid robots","author":"Sugiyama","year":"2007"},{"key":"B74","doi-asserted-by":"publisher","first-page":"2016","DOI":"10.3389\/fpsyg.2020.02016","article-title":"Targeting in language: Unifying deixis and anaphora","volume":"11","author":"Talmy","year":"2020","journal-title":"Front. Psychol."},{"key":"B75","volume-title":"Creating multimodal interactive agents with imitation and self-supervised learning","author":"Team","year":"2021"},{"key":"B76","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3478513.3480570","article-title":"Transflower: Probabilistic autoregressive dance generation with multimodal attention","volume":"40","author":"Valle-P\u00e9rez","year":"2021","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B77","volume-title":"Advanced skills through multiple adversarial motion priors in reinforcement learning","author":"Vollenweider","year":"2022"},{"key":"B78","first-page":"4095","article-title":"Communicative learning with natural gestures for embodied navigation agents with human-in-the-scene","author":"Wu","year":"2021"},{"key":"B79","article-title":"Learning vision-guided quadrupedal locomotion end-to-end with cross-modal transformers","author":"Yang","year":"2021"},{"key":"B80","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3414685.3417838","article-title":"Speech gesture generation from the trimodal context of text, audio, and speaker identity","volume":"39","author":"Yoon","year":"2020","journal-title":"ACM Trans. Graph."},{"key":"B81","first-page":"1209","article-title":"Towards an automatic generation of natural gestures for a storyteller robot","author":"Zabala","year":"2022"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1110534\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T01:17:33Z","timestamp":1680139053000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1110534\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,30]]},"references-count":81,"alternative-id":["10.3389\/frobt.2023.1110534"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2023.1110534","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,30]]},"article-number":"1110534"}}