{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T08:19:50Z","timestamp":1768378790519,"version":"3.49.0"},"reference-count":57,"publisher":"World Scientific Pub Co Pte Ltd","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Artif. Intell. Robot. Res."],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:p>Previous animatronic faces struggle to express emotions effectively due to hardware and software limitations. On the hardware side, earlier approaches either used rigid-driven mechanisms, which provided precise control but were difficult to design within constrained spaces, or tendon-driven mechanisms, which are more space-efficient but challenging to control. In contrast, we propose a hybrid actuation approach that combines the best of both worlds. The eyes and mouth\u2013 key areas for emotional expression \u2014 are controlled using rigid mechanisms for precise movement, while the nose and cheek, which convey subtle facial microexpressions, are driven by strings. This design allows us to build a compact yet versatile hardware platform capable of expressing a wide range of emotions. On the algorithmic side, our method introduces a self-modeling network that maps motor actions to facial landmarks, allowing us to automatically establish the relationship between blendshape coefficients for different facial expressions and the corresponding motor control signals through gradient backpropagation. We then train a neural network to map speech input to corresponding blendshape controls. With our method, we can generate distinct emotional expressions such as happiness, fear, disgust, and anger, from any given sentence, each with nuanced, emotion-specific control signals \u2014 a feature that has not been demonstrated in earlier systems. We release the hardware design and code at https:\/\/github.com\/ZZongzheng0918\/Morpheus-Hardware and https:\/\/github.com\/ZZongzheng0918\/Morpheus-Software .<\/jats:p>","DOI":"10.1142\/s2972335325500048","type":"journal-article","created":{"date-parts":[[2025,7,26]],"date-time":"2025-07-26T05:37:36Z","timestamp":1753508256000},"source":"Crossref","is-referenced-by-count":0,"title":["Morpheus: A Neural-Driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control"],"prefix":"10.1142","volume":"02","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-6909-1587","authenticated-orcid":false,"given":"Zongzheng","family":"Zhang","sequence":"first","affiliation":[{"name":"Institute for AI Industry Research (AIR), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2206-2778","authenticated-orcid":false,"given":"Jiawen","family":"Yang","sequence":"additional","affiliation":[{"name":"Institute for AI Industry Research (AIR), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1341-9618","authenticated-orcid":false,"given":"Ziqiao","family":"Peng","sequence":"additional","affiliation":[{"name":"Institute for AI Industry Research (AIR), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5630-6357","authenticated-orcid":false,"given":"Meng","family":"Yang","sequence":"additional","affiliation":[{"name":"MGI Tech, Shenzhen 518053, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8236-6609","authenticated-orcid":false,"given":"Jianzhu","family":"Ma","sequence":"additional","affiliation":[{"name":"Institute for AI Industry Research (AIR), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8090-2562","authenticated-orcid":false,"given":"Lin","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Astronautics, Beihang University, Beijing 100191, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8642-3519","authenticated-orcid":false,"given":"Huazhe","family":"Xu","sequence":"additional","affiliation":[{"name":"Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1928-7841","authenticated-orcid":false,"given":"Hang","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, Beijing 100084, P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7903-581X","authenticated-orcid":false,"given":"Hao","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute for AI Industry Research (AIR), Tsinghua University, Beijing 100084, P. R. China"}]}],"member":"219","published-online":{"date-parts":[[2025,9,6]]},"reference":[{"key":"S2972335325500048BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2005.1573603"},{"key":"S2972335325500048BIB002","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240804"},{"key":"S2972335325500048BIB003","first-page":"52","volume-title":"Int. Workshop on Computation for Metaphors, Analogy, and Agents","author":"Brooks R. A.","year":"1998"},{"key":"S2972335325500048BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/IECON.2000.973197"},{"key":"S2972335325500048BIB005","first-page":"2566","volume-title":"Seventeenth World Cong. Int. Federation of Automatic Control","author":"Lee D.-W.","year":"2008"},{"key":"S2972335325500048BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2008.4756017"},{"key":"S2972335325500048BIB007","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509173"},{"key":"S2972335325500048BIB008","doi-asserted-by":"publisher","DOI":"10.1109\/DEVLRN.2011.6037346"},{"key":"S2972335325500048BIB009","doi-asserted-by":"publisher","DOI":"10.3390\/biomimetics9030122"},{"key":"S2972335325500048BIB010","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adi4724"},{"key":"S2972335325500048BIB011","doi-asserted-by":"crossref","unstructured":"B. Li, H. Li and H. Liu, Driving animatronic robot facial expression from speech, arXiv preprint arXiv:2403.12670, 2024.","DOI":"10.1109\/IROS58592.2024.10801970"},{"key":"S2972335325500048BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.281935"},{"key":"S2972335325500048BIB013","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.282331"},{"key":"S2972335325500048BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/BioRob.2012.6290687"},{"key":"S2972335325500048BIB015","doi-asserted-by":"publisher","DOI":"10.5772\/62181"},{"key":"S2972335325500048BIB016","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9560797"},{"key":"S2972335325500048BIB017","first-page":"1","volume":"37","author":"Zhou Y.","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"S2972335325500048BIB018","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01034"},{"key":"S2972335325500048BIB019","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01821"},{"key":"S2972335325500048BIB020","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3611734"},{"key":"S2972335325500048BIB021","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00121"},{"key":"S2972335325500048BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01386"},{"key":"S2972335325500048BIB023","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01891"},{"key":"S2972335325500048BIB024","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.1993.367708"},{"key":"S2972335325500048BIB025","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2003.1248795"},{"key":"S2972335325500048BIB026","doi-asserted-by":"publisher","DOI":"10.1109\/ICMA.2006.257429"},{"key":"S2972335325500048BIB027","doi-asserted-by":"publisher","DOI":"10.1007\/3-211-38927-X_33"},{"key":"S2972335325500048BIB028","doi-asserted-by":"publisher","DOI":"10.1109\/ROBIO.2004.1521877"},{"key":"S2972335325500048BIB029","first-page":"72","volume-title":"AAAI Mobile Robot Competition","author":"Hanson D.","year":"2002"},{"key":"S2972335325500048BIB030","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2009.5379537"},{"key":"S2972335325500048BIB031","first-page":"561","volume-title":"2009 9th IEEE-RAS Int. Conf. Humanoid Robots","author":"Nakaoka S.","year":"2009"},{"key":"S2972335325500048BIB032","first-page":"1","volume-title":"2009 Int. Conf. Advanced Robotics","author":"Lin C.-Y.","year":"2009"},{"key":"S2972335325500048BIB033","doi-asserted-by":"publisher","DOI":"10.1016\/j.ohx.2020.e00117"},{"key":"S2972335325500048BIB034","doi-asserted-by":"publisher","DOI":"10.1115\/1.2959097"},{"key":"S2972335325500048BIB035","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00018-1"},{"key":"S2972335325500048BIB036","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-013-0199-6"},{"key":"S2972335325500048BIB037","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2017.7510658"},{"key":"S2972335325500048BIB038","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2018.00119"},{"key":"S2972335325500048BIB039","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2017.7510622"},{"key":"S2972335325500048BIB040","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2009.2029076"},{"key":"S2972335325500048BIB041","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2016.2599495"},{"key":"S2972335325500048BIB042","doi-asserted-by":"publisher","DOI":"10.1109\/CoASE.2014.6899473"},{"key":"S2972335325500048BIB043","doi-asserted-by":"publisher","DOI":"10.3390\/app9163379"},{"key":"S2972335325500048BIB044","doi-asserted-by":"crossref","first-page":"e0215216","DOI":"10.1371\/journal.pone.0215216","volume":"14","author":"Liu N.","year":"2019","journal-title":"PLoS One"},{"key":"S2972335325500048BIB045","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.800657"},{"key":"S2972335325500048BIB046","doi-asserted-by":"publisher","DOI":"10.1109\/Humanoids53995.2022.10000154"},{"key":"S2972335325500048BIB047","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48891.2023.10160409"},{"key":"S2972335325500048BIB048","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abn1944"},{"key":"S2972335325500048BIB049","doi-asserted-by":"publisher","DOI":"10.1145\/3592433"},{"key":"S2972335325500048BIB050","first-page":"20310","volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Wu G.","year":"2024"},{"key":"S2972335325500048BIB051","unstructured":"R. Liu, A. Canberk, S. Song and C. Vondrick, Differentiable robot rendering, arXiv:2410.13851, 2024."},{"key":"S2972335325500048BIB052","first-page":"359","volume-title":"Proc. 3rd Int. Conf. Knowledge Discovery and Data Mining","author":"Berndt D. J.","year":"1994"},{"key":"S2972335325500048BIB053","first-page":"12449","volume":"33","author":"Baevski A.","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S2972335325500048BIB054","unstructured":"C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.L. Chang, M. G. Yong, J. Lee\n                      et al\n                      ., Mediapipe: A framework for building perception pipelines, arXiv preprint arXiv:1906.08172 (2019)."},{"key":"S2972335325500048BIB055","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0196391"},{"key":"S2972335325500048BIB056","first-page":"3661","volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Zhang Z.","year":"2021"},{"key":"S2972335325500048BIB057","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2023.109368"}],"container-title":["International Journal of Artificial Intelligence and Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S2972335325500048","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T04:11:31Z","timestamp":1768363891000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S2972335325500048"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":57,"alternative-id":["10.1142\/S2972335325500048"],"URL":"https:\/\/doi.org\/10.1142\/s2972335325500048","relation":{},"ISSN":["2972-3353","2972-3361"],"issn-type":[{"value":"2972-3353","type":"print"},{"value":"2972-3361","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"article-number":"2550004"}}