{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T16:49:43Z","timestamp":1792601383819,"version":"4.5.0"},"reference-count":111,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>The incredible feats of athleticism demonstrated by humans are made possible in part by a vast repertoire of general-purpose motor skills, acquired through years of practice and experience. These skills not only enable humans to perform complex tasks, but also provide powerful priors for guiding their behaviors when learning new tasks. This is in stark contrast to what is common practice in physics-based character animation, where control policies are most typically trained from scratch for each task. In this work, we present a large-scale data-driven framework for learning versatile and reusable skill embeddings for physically simulated characters. Our approach combines techniques from adversarial imitation learning and unsupervised reinforcement learning to develop skill embeddings that produce life-like behaviors, while also providing an easy to control representation for use on new downstream tasks. Our models can be trained using large datasets of unstructured motion clips, without requiring any task-specific annotation or segmentation of the motion data. By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills. We show that a single pre-trained model can be effectively applied to perform a diverse set of new tasks. Our system also allows users to specify tasks through simple reward functions, and the skill embedding then enables the character to automatically synthesize complex and naturalistic strategies in order to achieve the task objectives.<\/jats:p>","DOI":"10.1145\/3528223.3530110","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":245,"title":["ASE"],"prefix":"10.1145","volume":"41","author":[{"given":"Xue Bin","family":"Peng","sequence":"first","affiliation":[{"name":"University of California and NVIDIA, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunrong","family":"Guo","sequence":"additional","affiliation":[{"name":"NVIDIA, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lina","family":"Halper","sequence":"additional","affiliation":[{"name":"NVIDIA, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergey","family":"Levine","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanja","family":"Fidler","sequence":"additional","affiliation":[{"name":"University of Toronto, Canada and NVIDIA, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning. CoRR abs\/1703.01732","author":"Achiam Joshua","year":"2017","unstructured":"Joshua Achiam and Shankar Sastry. 2017. Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning. CoRR abs\/1703.01732 (2017). arXiv:1703.01732 http:\/\/arxiv.org\/abs\/1703.01732"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.325"},{"key":"e_1_2_2_3_1","volume-title":"Relative Variational Intrinsic Control. CoRR abs\/2012.07827","author":"Baumli Kate","year":"2020","unstructured":"Kate Baumli, David Warde-Farley, Steven Hansen, and Volodymyr Mnih. 2020. Relative Variational Intrinsic Control. CoRR abs\/2012.07827 (2020). arXiv:2012.07827 https:\/\/arxiv.org\/abs\/2012.07827"},{"key":"e_1_2_2_4_1","volume-title":"Garnett (Eds.)","volume":"29","author":"Bellemare Marc","year":"2016","unstructured":"Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. 2016. Unifying Count-Based Exploration and Intrinsic Motivation. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2016\/file\/afda332245e2af431fb7b672a68b659d-Paper.pdf"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356536"},{"key":"e_1_2_2_6_1","volume-title":"Large Scale GAN Training for High Fidelity Natural Image Synthesis. In International Conference on Learning Representations.","author":"Brock Andrew","year":"2019","unstructured":"Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large Scale GAN Training for High Fidelity Natural Image Synthesis. In International Conference on Learning Representations."},{"key":"e_1_2_2_7_1","volume-title":"Language Models are Few-Shot Learners. CoRR abs\/2005.14165","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. CoRR abs\/2005.14165 (2020). arXiv:2005.14165 https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_2_2_8_1","volume-title":"International Conference on Learning Representations.","author":"Burda Yuri","year":"2019","unstructured":"Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. 2019. Exploration by random network distillation. In International Conference on Learning Representations."},{"key":"e_1_2_2_9_1","volume-title":"Advances in Neural Information Processing Systems","author":"Chen Xi","year":"2016","unstructured":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2016\/file\/7c9d0b1f96aebd7b5eca8c3edaa19ebb-Paper.pdf"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274247.3274506"},{"key":"e_1_2_2_11_1","article-title":"Robust Task-based Control Policies for Physics-based Characters","volume":"28","author":"Coros Stelian","year":"2009","unstructured":"Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust Task-based Control Policies for Physics-based Characters. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 28, 5 (2009), Article 170.","journal-title":"ACM Trans. Graph. (Proc. SIGGRAPH Asia)"},{"key":"e_1_2_2_12_1","volume-title":"Generalized Biped Walking Control. ACM Transctions on Graphics 29, 4","author":"Coros Stelian","year":"2010","unstructured":"Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized Biped Walking Control. ACM Transctions on Graphics 29, 4 (2010), Article 130."},{"key":"e_1_2_2_13_1","doi-asserted-by":"crossref","unstructured":"M. Da Silva Y. Abe and J. Popovic. 2008. Simulation of Human Motion Data using Short-Horizon Model-Predictive Control. Computer Graphics Forum (2008).","DOI":"10.1111\/j.1467-8659.2008.01134.x"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781157"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_2_16_1","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=HkpbnH9lx","author":"Dinh Laurent","year":"2017","unstructured":"Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density estimation using Real NVP. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=HkpbnH9lx"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.167"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"655","author":"Donahue Jeff","year":"2014","unstructured":"Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 32), Eric P. Xing and Tony Jebara (Eds.). PMLR, Bejing, China, 647--655. https:\/\/proceedings.mlr.press\/v32\/donahue14.html"},{"key":"e_1_2_2_19_1","volume-title":"Proceedings of the 20th National Conference on Artificial Intelligence -","volume":"3","author":"Endo Gen","year":"2005","unstructured":"Gen Endo, Jun Morimoto, Takamitsu Matsubara, Jun Nakanishi, and Gordon Cheng. 2005. Learning CPG Sensory Feedback with Policy Gradient for Biped Locomotion for a Full-Body Humanoid. In Proceedings of the 20th National Conference on Artificial Intelligence - Volume 3 (Pittsburgh, Pennsylvania) (AAAI'05). AAAI Press, 1267--1273."},{"key":"e_1_2_2_20_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=SJx63jRqFm","author":"Eysenbach Benjamin","year":"2019","unstructured":"Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. 2019. Diversity is All You Need: Learning Skills without a Reward Function. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=SJx63jRqFm"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383287"},{"key":"e_1_2_2_22_1","volume-title":"5th International Conference on Learning Representations, ICLR","author":"Florensa Carlos","year":"2017","unstructured":"Carlos Florensa, Yan Duan, and Pieter Abbeel. 2017. Stochastic Neural Networks for Hierarchical Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=B1oK8aoxe"},{"key":"e_1_2_2_23_1","volume-title":"Advances in Neural Information Processing Systems","author":"Fu Justin","year":"2017","unstructured":"Justin Fu, John Co-Reyes, and Sergey Levine. 2017. EX2: Exploration with Exemplar Models for Deep Reinforcement Learning. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/1baff70e2669e8376347efd3a874a341-Paper.pdf"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2015.2505910"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508399"},{"key":"e_1_2_2_26_1","volume-title":"Positive force feedback in bouncing gaits? Proc","author":"Geyer Hartmut","year":"2003","unstructured":"Hartmut Geyer, Andre Seyfarth, and Reinhard Blickhan. 2003. Positive force feedback in bouncing gaits? Proc. Royal Society of London B: Biological Sciences 270, 1529 (2003), 2173--2183."},{"key":"e_1_2_2_27_1","volume-title":"Advances in Neural Information Processing Systems 27","author":"Goodfellow Ian","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2672--2680. http:\/\/papers.nips.cc\/paper\/5423-generative-adversarial-nets.pdf"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/10867651.1998.10487493"},{"key":"e_1_2_2_29_1","volume-title":"Variational Intrinsic Control. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Workshop Track Proceedings. OpenReview.net.","author":"Gregor Karol","year":"2017","unstructured":"Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. 2017. Variational Intrinsic Control. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Workshop Track Proceedings. OpenReview.net."},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research","author":"Hasenclever Leonard","year":"2020","unstructured":"Leonard Hasenclever, Fabio Pardo, Raia Hadsell, Nicolas Heess, and Josh Merel. 2020. CoMic: Complementary Task Learning & Mimicry for Reusable Skills. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR, 4105--4115. https:\/\/proceedings.mlr.press\/v119\/hasenclever20a.html"},{"key":"e_1_2_2_31_1","volume-title":"Advances in Neural Information Processing Systems","author":"Hausman Karol","year":"2017","unstructured":"Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, and Joseph J Lim. 2017. Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/632cee946db83e7a52ce5e8d6f0fed35-Paper.pdf"},{"key":"e_1_2_2_32_1","volume-title":"International Conference on Learning Representations.","author":"Hausman Karol","year":"2018","unstructured":"Karol Hausman, Jost Tobias Springenberg, Ziyu Wang, Nicolas Heess, and Martin Riedmiller. 2018. Learning an Embedding Space for Transferable Robot Skills. In International Conference on Learning Representations."},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"2691","author":"Hazan Elad","year":"2019","unstructured":"Elad Hazan, Sham Kakade, Karan Singh, and Abby Van Soest. 2019. Provably Efficient Maximum Entropy Exploration. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 2681--2691. https:\/\/proceedings.mlr.press\/v97\/hazan19a.html"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_35_1","volume-title":"Learning and Transfer of Modulated Locomotor Controllers. CoRR abs\/1610.05182","author":"Heess Nicolas","year":"2016","unstructured":"Nicolas Heess, Gregory Wayne, Yuval Tassa, Timothy P. Lillicrap, Martin A. Riedmiller, and David Silver. 2016. Learning and Transfer of Modulated Locomotor Controllers. CoRR abs\/1610.05182 (2016). arXiv:1610.05182"},{"key":"e_1_2_2_36_1","volume-title":"Reducing the Dimensionality of Data with Neural Networks. Science 313, 5786","author":"Hinton Geoffrey","year":"2006","unstructured":"Geoffrey Hinton and Ruslan Salakhutdinov. 2006. Reducing the Dimensionality of Data with Neural Networks. Science 313, 5786 (2006), 504--507."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218414"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322966"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2992393"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-66723-8_19"},{"key":"e_1_2_2_42_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_2_2_43_1","volume-title":"Kingma and Max Welling","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14--16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1312.6114"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289694"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.2983616"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781155"},{"key":"e_1_2_2_47_1","volume-title":"The Power of Scale for Parameter-Efficient Prompt Tuning. CoRR abs\/2104.08691","author":"Lester Brian","year":"2021","unstructured":"Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. CoRR abs\/2104.08691 (2021). arXiv:2104.08691 https:\/\/arxiv.org\/abs\/2104.08691"},{"key":"e_1_2_2_48_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"9","author":"Levine Sergey","year":"2013","unstructured":"Sergey Levine and Vladlen Koltun. 2013. Guided Policy Search. In Proceedings of the 30th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 28), Sanjoy Dasgupta and David McAllester (Eds.). PMLR, Atlanta, Georgia, USA, 1--9. https:\/\/proceedings.mlr.press\/v28\/levine13.html"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70139-4"},{"key":"e_1_2_2_50_1","first-page":"4","article-title":". Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning","volume":"37","author":"Libin Liu Jessica Hodgins","year":"2018","unstructured":"Jessica Hodgins Libin Liu. August 2018. Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning. ACM Transactions on Graphics 37, 4 (August 2018).","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392422"},{"key":"e_1_2_2_52_1","volume-title":"Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"6747","author":"Liu Hao","year":"2021","unstructured":"Hao Liu and Pieter Abbeel. 2021a. APS: Active Pretraining with Successor Features. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 6736--6747. https:\/\/proceedings.mlr.press\/v139\/liu21b.html"},{"key":"e_1_2_2_53_1","volume-title":"Behavior From the Void: Unsupervised Active Pre-Training. CoRR abs\/2103.04551","author":"Liu Hao","year":"2021","unstructured":"Hao Liu and Pieter Abbeel. 2021b. Behavior From the Void: Unsupervised Active Pre-Training. CoRR abs\/2103.04551 (2021). arXiv:2103.04551 https:\/\/arxiv.org\/abs\/2103.04551"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3083723"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2893476"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366173"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778865"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392433"},{"key":"e_1_2_2_59_1","volume-title":"Proceedings of the Conference on Robot Learning (Proceedings of Machine Learning Research","volume":"1132","author":"Lynch Corey","year":"2020","unstructured":"Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. 2020. Learning Latent Plans from Play. In Proceedings of the Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 100), Leslie Pack Kaelbling, Danica Kragic, and Komei Sugiura (Eds.). PMLR, 1113--1132. https:\/\/proceedings.mlr.press\/v100\/lynch20a.html"},{"key":"e_1_2_2_60_1","volume-title":"Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning. CoRR abs\/2108.10470","author":"Makoviychuk Viktor","year":"2021","unstructured":"Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. 2021. Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning. CoRR abs\/2108.10470 (2021). arXiv:2108.10470 https:\/\/arxiv.org\/abs\/2108.10470"},{"key":"e_1_2_2_61_1","volume-title":"Neural Probabilistic Motor Primitives for Humanoid Control. In International Conference on Learning Representations.","author":"Merel Josh","year":"2019","unstructured":"Josh Merel, Leonard Hasenclever, Alexandre Galashov, Arun Ahuja, Vu Pham, Greg Wayne, Yee Whye Teh, and Nicolas Heess. 2019. Neural Probabilistic Motor Primitives for Humanoid Control. In International Conference on Learning Representations."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392474"},{"key":"e_1_2_2_63_1","volume-title":"Weinberger (Eds.)","volume":"26","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2013\/file\/9aa42b31882ec039965f3c4923ce901b-Paper.pdf"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778808"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_2_66_1","volume-title":"Proceedings of the 27th International Conference on International Conference on Machine Learning","author":"Nair Vinod","unstructured":"Vinod Nair and Geoffrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning (Haifa, Israel) (ICML'10). Omnipress, Madison, WI, USA, 807--814."},{"key":"e_1_2_2_67_1","volume-title":"Garnett (Eds.)","volume":"29","author":"Nowozin Sebastian","year":"2016","unstructured":"Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2016\/file\/cedebb6e872f539bef8c3f919874e9d7-Paper.pdf"},{"key":"e_1_2_2_68_1","volume-title":"Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748","author":"van den Oord Aaron","year":"2018","unstructured":"Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018a. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)."},{"key":"e_1_2_2_69_1","volume-title":"Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748","author":"van den Oord Aaron","year":"2018","unstructured":"Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018b. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)."},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356501"},{"key":"e_1_2_2_71_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.70"},{"key":"e_1_2_2_73_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"5071","author":"Pathak Deepak","year":"2019","unstructured":"Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. 2019. Self-Supervised Exploration via Disagreement. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 5062--5071. https:\/\/proceedings.mlr.press\/v97\/pathak19a.html"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201311"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_76_1","unstructured":"Xue Bin Peng Michael Chang Grace Zhang Pieter Abbeel and Sergey Levine. 2019. MCP: Learning Composable Hierarchical Control with Multiplicative CompositionalPolicies. In Advances in Neural Information Processing Systems 32 H. Wallach H. Larochelle A. Beygelzimer F. d'Alch\u00e9-Buc E. Fox and R. Garnett (Eds.). Curran Associates Inc. 3681--3692. http:\/\/papers.nips.cc\/paper\/8626-mcp-learning-composable-hierarchical-control-with-multiplicative-compositional-policies.pdf"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459670"},{"key":"e_1_2_2_78_1","first-page":"1","article-title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research 21, 140 (2020), 1--67. http:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/122718.122755"},{"key":"e_1_2_2_80_1","unstructured":"Reallusion. 2022. 3D Animation and 2D Cartoons Made Simple. http:\/\/www.reallusion.com."},{"key":"e_1_2_2_81_1","volume-title":"High-Dimensional Continuous Control Using Generalized Advantage Estimation. CoRR abs\/1506.02438","author":"Schulman John","year":"2015","unstructured":"John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. 2015. High-Dimensional Continuous Control Using Generalized Advantage Estimation. CoRR abs\/1506.02438 (2015). arXiv:1506.02438"},{"key":"e_1_2_2_82_1","volume-title":"Proximal Policy Optimization Algorithms. CoRR abs\/1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. CoRR abs\/1707.06347 (2017). arXiv:1707.06347 http:\/\/arxiv.org\/abs\/1707.06347"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8462891"},{"key":"e_1_2_2_84_1","volume-title":"Dynamics-Aware Unsupervised Discovery of Skills. In International Conference on Learning Representations.","author":"Sharma Archit","year":"2020","unstructured":"Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. 2020. Dynamics-Aware Unsupervised Discovery of Skills. In International Conference on Learning Representations."},{"key":"e_1_2_2_85_1","volume-title":"Proc. of IEEE International Conference on Robotics and Animation.","author":"Sharon Dana","unstructured":"Dana Sharon and Michiel van de Panne. 2005. Synthesis of Controllers for Stylized Planar Bipedal Walking. In Proc. of IEEE International Conference on Robotics and Animation."},{"key":"e_1_2_2_86_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"2265","author":"Sohl-Dickstein Jascha","year":"2015","unstructured":"Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 2256--2265. https:\/\/proceedings.mlr.press\/v37\/sohl-dickstein15.html"},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276511"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1101\/2020.08.11.246801"},{"key":"e_1_2_2_89_1","volume-title":"Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models. CoRR abs\/1507.00814","author":"Stadie Bradly C.","year":"2015","unstructured":"Bradly C. Stadie, Sergey Levine, and Pieter Abbeel. 2015. Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models. CoRR abs\/1507.00814 (2015). arXiv:1507.00814 http:\/\/arxiv.org\/abs\/1507.00814"},{"key":"e_1_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2007.08.009"},{"key":"e_1_2_2_91_1","volume-title":"Barto","author":"Sutton Richard S.","year":"1998","unstructured":"Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). MIT Press, Cambridge, MA, USA."},{"key":"e_1_2_2_92_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601121"},{"key":"e_1_2_2_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964953"},{"key":"e_1_2_2_94_1","volume-title":"Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel.","author":"Tang Haoran","year":"2017","unstructured":"Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, OpenAI Xi Chen, Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel. 2017. #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/3a20f62a0af1aa152670bab3c602feed-Paper.pdf"},{"key":"e_1_2_2_95_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386025"},{"key":"e_1_2_2_96_1","volume-title":"Proceedings of Graphics Interface '94","author":"van de Panne Michiel","year":"1994","unstructured":"Michiel van de Panne, Ryan Kim, and Eugene Flume. 1994. Virtual Wind-up Toys for Animation. In Proceedings of Graphics Interface '94. 208--215."},{"key":"e_1_2_2_97_1","unstructured":"Oriol Vinyals Alexander Toshev Samy Bengio and Dumitru Erhan. 2014. Show and Tell: A Neural Image Caption Generator. http:\/\/arxiv.org\/abs\/1411.4555 cite arxiv:1411.4555."},{"key":"e_1_2_2_98_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601192"},{"key":"e_1_2_2_99_1","doi-asserted-by":"publisher","DOI":"10.1145\/1661412.1618514"},{"key":"e_1_2_2_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185521"},{"key":"e_1_2_2_101_1","unstructured":"Tingwu Wang Yunrong Guo Maria Shugrina and Sanja Fidler. 2020. UniCon: Universal Neural Controller For Physics-based Character Motion. arXiv:2011.15119 [cs.GR]"},{"key":"e_1_2_2_102_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295284"},{"key":"e_1_2_2_103_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392381"},{"key":"e_1_2_2_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459761"},{"key":"e_1_2_2_105_1","doi-asserted-by":"publisher","DOI":"10.1145\/3480148"},{"key":"e_1_2_2_106_1","volume-title":"Diversity-Sensitive Conditional Generative Adversarial Networks. In International Conference on Learning Representations.","author":"Yang Dingdong","year":"2019","unstructured":"Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tiangchen Zhao, and Honglak Lee. 2019. Diversity-Sensitive Conditional Generative Adversarial Networks. In International Conference on Learning Representations."},{"key":"e_1_2_2_107_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778811"},{"key":"e_1_2_2_108_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360680"},{"key":"e_1_2_2_109_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276509"},{"key":"e_1_2_2_110_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201397"},{"key":"e_1_2_2_111_1","doi-asserted-by":"publisher","DOI":"10.1145\/545261.545276"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530110","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530110","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:26Z","timestamp":1750186946000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530110"}},"subtitle":["large-scale reusable adversarial skill embeddings for physically simulated characters"],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":111,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530110"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530110","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}