{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T04:12:29Z","timestamp":1788495149431,"version":"build-2803163510"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T00:00:00Z","timestamp":1781481600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"Research Grants Council (RGC) of Hong Kong","award":["C4042-23GF"],"award-info":[{"award-number":["C4042-23GF"]}]},{"name":"PolyU-EIT Collaborative PhD Training Programme","award":["220766263"],"award-info":[{"award-number":["220766263"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>We introduce a novel framework with strong generalization capabilities for enabling humanoid robots to perform semantically grounded, context-aware, and physically realizable expressive behaviors, with the goal of enhancing end-user engagement in open-world human\u2013robot interaction (HRI). To achieve this goal, we develop a new large language model\u2013based human cognition module that interprets user dialogs to infer latent intent and generates high-level multi-modal behavior descriptions. These semantic representations are mapped to speech and motion outputs through a dual-stage embodied behavior generation pipeline. The pipeline consists of a shape adaptation module that maps human body motions into the robot\u2019s kinematic space, followed by a motion retargeting module that generates executable joint trajectories under physical constraints. Additionally, the modular architecture enables seamless integration of state-of-the-art generative models and serves as a practical testbed for evaluating expressive behavior generation in real-world settings. We validate this system on a 58-DoF humanoid platform through both controlled video-based studies and live HRI experiments. The results show significant improvements over rule-based and handcrafted baselines in terms of expressiveness, behavioral appeal, and user engagement. This work helps bridging the gap between high-level language understanding and low-level robot control, thereby enabling scalable and human-aligned expressive behavior generation for embodied agents.<\/jats:p>","DOI":"10.1145\/3813107","type":"journal-article","created":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T14:16:21Z","timestamp":1778249781000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Enhancing End-user Engagement in Human\u2013Robot Interaction by Performing LLM-driven Expressive Behaviors"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1649-5370","authenticated-orcid":false,"given":"Shipeng","family":"Lyu","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University (PolyU), Kowloon, Hong Kong, and Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7492-632X","authenticated-orcid":false,"given":"Fangyuan","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University (PolyU), Kowloon, Hong Kong, and Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9581-7186","authenticated-orcid":false,"given":"Weiwei","family":"Lin","sequence":"additional","affiliation":[{"name":"Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9583-0055","authenticated-orcid":false,"given":"Guodong","family":"Guo","sequence":"additional","affiliation":[{"name":"Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3426-6638","authenticated-orcid":false,"given":"David","family":"Navarro-Alarcon","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University (PolyU), Kowloon, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,15]]},"reference":[{"issue":"5","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1080\/10400435.2021.1880493","article-title":"Humanoid robots in the care of older persons: A scoping review","volume":"34","author":"Andtfolk Malin","year":"2022","unstructured":"Malin Andtfolk, Linda Nyholm, Hilde Eide, and Lisbeth Fagerstr\u00f6m. 2022. Humanoid robots in the care of older persons: A scoping review. Assistive Technology 34, 5 (2022), 518\u2013526.","journal-title":"Assistive Technology"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3550454.3555435"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01726"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Xuxin Cheng Yandong Ji Junming Chen Ruihan Yang Ge Yang and Xiaolong Wang. 2024. Expressive whole-body control for humanoid robots. arXiv:2402.16796. Retrieved from https:\/\/arxiv.org\/abs\/2402.16796","DOI":"10.15607\/RSS.2024.XX.107"},{"key":"e_1_3_2_6_2","first-page":"390","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Dai Wenxun","year":"2024","unstructured":"Wenxun Dai, Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. 2024. Motionlcm: Real-time controllable motion generation via latent consistency model. In Proceedings of the European Conference on Computer Vision. Springer, 390\u2013408."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-022-09333-y"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-023-10133-5"},{"issue":"5","key":"e_1_3_2_9_2","first-page":"4083","article-title":"Can an embodied agent find your \u201ccat-shaped mug\u201d? LLM-based zero-shot object navigation","volume":"9","author":"Sashank Dorbala Vishnu","year":"2023","unstructured":"Vishnu Sashank Dorbala, James F. Mullen, Jr., and Dinesh Manocha. 2023. Can an embodied agent find your \u201ccat-shaped mug\u201d? LLM-based zero-shot object navigation. IEEE Robotics and Automation Letters 9, 5 (2024), 4083\u20134090.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.169"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00186"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19833-5_34"},{"key":"e_1_3_2_13_2","first-page":"7315","volume-title":"Proceedings of the 2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Habekost Jan-Gerrit","year":"2024","unstructured":"Jan-Gerrit Habekost, Connor G\u00e4de, Philipp Allgeuer, and Stefan Wermter. 2024. Inverse kinematics for neuro-robotic grasping with humanoid embodied agents. In Proceedings of the 2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 7315\u20137322."},{"key":"e_1_3_2_14_2","first-page":"457","volume-title":"Proceedings of the International Conference on Artificial Neural Networks","author":"Habekost Jan-Gerrit","year":"2023","unstructured":"Jan-Gerrit Habekost, Erik Strahl, Philipp Allgeuer, Matthias Kerzel, and Stefan Wermter. 2023. CycleIK: Neuro-inspired inverse kinematics. In Proceedings of the International Conference on Artificial Neural Networks. Springer, 457\u2013470."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2025.3575983"},{"key":"e_1_3_2_16_2","first-page":"1769","volume-title":"Proceedings of the Conference on Robot Learning","author":"Huang Wenlong","year":"2023","unstructured":"Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. 2023. Inner monologue: Embodied reasoning through planning with language models. In Proceedings of the Conference on Robot Learning. PMLR, 1769\u20131782."},{"key":"e_1_3_2_17_2","first-page":"809","volume-title":"Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition","author":"Inderbitzin Martin","year":"2011","unstructured":"Martin Inderbitzin, Aleksander V\u00e4ljam\u00e4e, Jose Maria Blanco Calvo, Paul F. M. J. Verschure, and Ulysses Bernardet. 2011. Expression of emotional states during locomotion based on canonical parameters. In Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, 809\u2013814."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3329370"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418815"},{"key":"e_1_3_2_20_2","first-page":"5463","volume-title":"Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA)","author":"Kwon Minae","year":"2024","unstructured":"Minae Kwon, Hengyuan Hu, Vivek Myers, Siddharth Karamcheti, Anca Dragan, and Dorsa Sadigh. 2024. Toward grounded commonsense reasoning. In Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5463\u20135470."},{"key":"e_1_3_2_21_2","first-page":"612","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Liu Haiyang","year":"2022","unstructured":"Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. 2022. Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In Proceedings of the European Conference on Computer Vision. Springer, 612\u2013630."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_3_2_23_2","unstructured":"Shipeng Lyu Fangyuan Wang Weiwei Lin Luhao Zhu David Navarro-Alarcon and Guodong Guo. 2025. HuBE: Cross-embodiment human-like behavior execution for humanoid robots. arXiv:2508.19002. Retrieved from https:\/\/arxiv.org\/abs\/2508.19002"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610977.3634999"},{"key":"e_1_3_2_25_2","first-page":"148","volume-title":"Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence","author":"Onorati Teresa","year":"2023","unstructured":"Teresa Onorati, \u00c1lvaro Castro-Gonz\u00e1lez, Javier Cruz del Valle, Paloma D\u00edaz, and Jos\u00e9 Carlos Castillo. 2023. Creating personalized verbal human-robot interactions using LLM with the robot mini. In Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence. Springer, 148\u2013159."},{"issue":"15","key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"8198","DOI":"10.3390\/ijerph18158198","article-title":"The humanoid robot Sil-Bot in a cognitive training program for community-dwelling elderly people with mild cognitive impairment during the COVID-19 pandemic: A randomized controlled trial","volume":"18","author":"Park Eun-A","year":"2021","unstructured":"Eun-A Park, Ae-Ri Jung, and Kyoung-A Lee. 2021. The humanoid robot Sil-Bot in a cognitive training program for community-dwelling elderly people with mild cognitive impairment during the COVID-19 pandemic: A randomized controlled trial. International Journal of Environmental Research and Public Health 18, 15 (2021), 8198.","journal-title":"International Journal of Environmental Research and Public Health"},{"key":"e_1_3_2_27_2","volume-title":"Proceedings of the International Conference on Computer Vision (ICCV)","author":"Petrovich Mathis","year":"2023","unstructured":"Mathis Petrovich, Michael J. Black, and G\u00fcl Varol. 2023. TMR: Text-to-motion retrieval using contrastive 3D human motion synthesis. In Proceedings of the International Conference on Computer Vision (ICCV)."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.3390\/app12147186"},{"key":"e_1_3_2_29_2","first-page":"28492","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Radford Alec","year":"2023","unstructured":"Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the International Conference on Machine Learning. PMLR, 28492\u201328518."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-022-00914-w"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559672"},{"key":"e_1_3_2_32_2","volume-title":"Disney Animation: The Illusion of Life","author":"Thomas Frank","year":"1981","unstructured":"Frank Thomas and Ollie Johnston. 1981. Disney Animation: The Illusion of Life. Abbeville Press, New York."},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the Snowbird Learning Workshop","author":"Torresani Lorenzo","year":"2006","unstructured":"Lorenzo Torresani, Peggy Hackney, and Christoph Bregler. 2006. Learning to synthesize motion styles. In Proceedings of the Snowbird Learning Workshop. Citeseer."},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1007\/s12369-014-0231-5","article-title":"Do robot performance and behavioral style affect human trust? A multi-method approach","volume":"6","author":"van den Brule Rik","year":"2014","unstructured":"Rik van den Brule, Ron Dotsch, Gijsbert Bijlstra, Daniel H. J. Wigboldus, and Pim Haselager. 2014. Do robot performance and behavioral style affect human trust? A multi-method approach. International Journal of Social Robotics 6 (2014), 519\u2013531.","journal-title":"International Journal of Social Robotics"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3344286"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3687272.3688302"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610978.3640562"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-023-10139-z"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00622"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3612503"},{"key":"e_1_3_2_41_2","first-page":"5860","volume-title":"Proceedings of the 32nd International Joint Conference on Artificial Intelligence","author":"Yang Sicheng","year":"2023","unstructured":"Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, and Long Xiao. 2023. DiffuseStyleGesture: Stylized audio-driven co-speech gesture generation with diffusion models. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence, 5860\u20135868."},{"key":"e_1_3_2_42_2","first-page":"2321","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Yang Sicheng","year":"2023","unstructured":"Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, and Haolin Zhuang. 2023. QPGesture: Quantization-based and phase-guided motion matching for natural speech-driven gesture generation. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2321\u20132330."},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","first-page":"1581110","DOI":"10.3389\/frobt.2025.1581110","article-title":"From text to motion: Grounding GPT-4 in a humanoid robot \u201calter3.\u201d","volume":"12","author":"Yoshida Takahide","year":"2025","unstructured":"Takahide Yoshida, Atsushi Masumori, and Takashi Ikegami. 2025. From text to motion: Grounding GPT-4 in a humanoid robot \u201calter3.\u201d Frontiers in Robotics and AI 12 (2025), 1581110.","journal-title":"Frontiers in Robotics and AI"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBIO.2016.7866528"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2024.3355414"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3658134"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3813107","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T14:15:46Z","timestamp":1781532946000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3813107"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,15]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3813107"],"URL":"https:\/\/doi.org\/10.1145\/3813107","relation":{},"ISSN":["2573-9522"],"issn-type":[{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,15]]},"assertion":[{"value":"2024-12-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}