{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T17:40:26Z","timestamp":1777657226101,"version":"3.51.4"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T00:00:00Z","timestamp":1632268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2021,9,22]]},"abstract":"<jats:p>We present a simple and intuitive approach for interactive control of physically simulated characters. Our work builds upon generative adversarial networks (GAN) and reinforcement learning, and introduces an imitation learning framework where an ensemble of classifiers and an imitation policy are trained in tandem given pre-processed reference clips. The classifiers are trained to discriminate the reference motion from the motion generated by the imitation policy, while the policy is rewarded for fooling the discriminators. Using our GAN-like approach, multiple motor control policies can be trained separately to imitate different behaviors. In runtime, our system can respond to external control signal provided by the user and interactively switch between different policies. Compared to existing method, our proposed approach has the following attractive properties: 1) achieves state-of-the-art imitation performance without manually designing and fine tuning a reward function; 2) directly controls the character without having to track any target reference pose explicitly or implicitly through a phase state; and 3) supports interactive policy switching without requiring any motion generation or motion matching mechanism. We highlight the applicability of our approach in a range of imitation and interactive control tasks, while also demonstrating its ability to withstand external perturbations as well as to recover balance. Overall, our approach has low runtime cost and can be easily integrated into interactive applications and games.<\/jats:p>","DOI":"10.1145\/3480148","type":"journal-article","created":{"date-parts":[[2021,9,28]],"date-time":"2021-09-28T04:43:36Z","timestamp":1632804216000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":28,"title":["A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control"],"prefix":"10.1145","volume":"4","author":[{"given":"Pei","family":"Xu","sequence":"first","affiliation":[{"name":"School of Computing, Clemson University, USA"}]},{"given":"Ioannis","family":"Karamouzas","sequence":"additional","affiliation":[{"name":"School of Computing, Clemson University, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,9,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356536"},{"key":"e_1_2_2_2_1","volume-title":"Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096","author":"Brock Andrew","year":"2018"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2485895.2485906"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274247.3274506"},{"key":"e_1_2_2_5_1","volume-title":"Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555","author":"Chung Junyoung","year":"2014"},{"key":"e_1_2_2_6_1","volume-title":"Proc. of GDC.","author":"Clavet Simon","year":"2016"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781156"},{"key":"e_1_2_2_8_1","unstructured":"Erwin Coumans and Yunfei Bai. 2016--2021. PyBullet a Python module for physics simulation for games robotics and machine learning. http:\/\/pybullet.org.  Erwin Coumans and Yunfei Bai. 2016--2021. PyBullet a Python module for physics simulation for games robotics and machine learning. http:\/\/pybullet.org."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13096"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01134.x"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/648054.743935"},{"key":"e_1_2_2_12_1","volume-title":"Generative multi-adversarial networks. arXiv preprint arXiv:1611.01673","author":"Durugkar Ishan","year":"2016"},{"key":"e_1_2_2_13_1","volume-title":"Proceedings of the 11th ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 211--219","author":"Geijtenbeek Thomas"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508399"},{"key":"e_1_2_2_15_1","volume-title":"Generative adversarial networks. arXiv preprint arXiv:1406.2661","author":"Goodfellow Ian J","year":"2014"},{"key":"e_1_2_2_16_1","volume-title":"Improved training of Wasserstein GANs. arXiv preprint arXiv:1704.00028","author":"Gulrajani Ishaan","year":"2017"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2767002"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392480"},{"key":"e_1_2_2_19_1","unstructured":"Nicolas Heess Srinivasan Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang SM Eslami Martin Riedmiller etal 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017).  Nicolas Heess Srinivasan Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang SM Eslami Martin Riedmiller et al. 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017)."},{"key":"e_1_2_2_20_1","volume-title":"Generative adversarial imitation learning. arXiv preprint arXiv:1606.03476","author":"Ho Jonathan","year":"2016"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392440"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073663"},{"key":"e_1_2_2_23_1","volume-title":"Learning agile and dynamic motor skills for legged robots. Science Robotics 4, 26","author":"Hwangbo Jemin","year":"2019"},{"key":"e_1_2_2_24_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/1921427.1921447"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983616"},{"key":"e_1_2_2_27_1","first-page":"1","article-title":"Scalable muscle-actuated human simulation and control","volume":"38","author":"Lee Seunghwan","year":"2019","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781155"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661233"},{"key":"e_1_2_2_30_1","volume-title":"Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971","author":"Lillicrap Timothy P","year":"2015"},{"key":"e_1_2_2_31_1","volume-title":"Geometric GAN. arXiv preprint arXiv:1705.02894","author":"Lim Jae Hyun","year":"2017"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392422"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982424"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12571"},{"key":"e_1_2_2_35_1","volume-title":"ACM SIGGRAPH 2010 papers. 1--10","author":"Liu Libin"},{"key":"e_1_2_2_36_1","volume-title":"Learning human behaviors from motion capture by adversarial imitation. arXiv preprint arXiv:1707.02201","author":"Merel Josh","year":"2017"},{"key":"e_1_2_2_37_1","volume-title":"Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957","author":"Miyato Takeru","year":"2018"},{"key":"e_1_2_2_38_1","volume-title":"Robotics: Science and Systems","volume":"4","author":"Mordatch Igor","year":"2014"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201305"},{"key":"e_1_2_2_41_1","volume-title":"International Conference on Machine Learning","volume":"1","author":"Ng Andrew Y","year":"2000"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356501"},{"key":"e_1_2_2_43_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"Deepmimic: Example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng Xue Bin","year":"2018","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_44_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073602","article-title":"Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning","volume":"36","author":"Peng Xue Bin","year":"2017","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_45_1","volume-title":"Jie Tan, and Sergey Levine.","author":"Peng Xue Bin","year":"2020"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459670"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/279943.279964"},{"key":"e_1_2_2_48_1","volume-title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120","author":"Saxe Andrew M","year":"2013"},{"key":"e_1_2_2_49_1","volume-title":"International Conference on Machine Learning. 1889--1897","author":"Schulman John","year":"2015"},{"key":"e_1_2_2_50_1","volume-title":"High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438","author":"Schulman John","year":"2015"},{"key":"e_1_2_2_51_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2011.30"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386025"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907001"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392381"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275023"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/1921427.1921445"},{"key":"e_1_2_2_58_1","article-title":"Terrain-adaptive bipedal locomotion control","volume":"29","author":"Zoran Popovi\u0107 Wu","year":"2010","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14115"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276509"},{"key":"e_1_2_2_61_1","first-page":"1","article-title":"Learning symmetric and low-energy locomotion","volume":"37","author":"Yu Wenhao","year":"2018","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_62_1","volume-title":"International Conference on Machine Learning. 7354--7363","author":"Zhang Han","year":"2019"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3480148","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3480148","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:16Z","timestamp":1750188676000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3480148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,22]]},"references-count":62,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,9,22]]}},"alternative-id":["10.1145\/3480148"],"URL":"https:\/\/doi.org\/10.1145\/3480148","relation":{},"ISSN":["2577-6193"],"issn-type":[{"value":"2577-6193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,22]]},"assertion":[{"value":"2021-09-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}