{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T03:27:13Z","timestamp":1769743633280,"version":"3.49.0"},"reference-count":89,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T00:00:00Z","timestamp":1619395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2021,4,26]]},"abstract":"<jats:p>Physics-based character animation has seen significant advances in recent years with the adoption of Deep Reinforcement Learning (DRL). However, DRL-based learning methods are usually computationally expensive and their performance crucially depends on the choice of hyperparameters. Tuning hyperparameters for these methods often requires repetitive training of control policies, which is even more computationally prohibitive. In this work, we propose a novel Curriculum-based Multi-Fidelity Bayesian Optimization framework (CMFBO) for efficient hyperparameter optimization of DRL-based character control systems. Using curriculum-based task difficulty as fidelity criterion, our method improves searching efficiency by gradually pruning search space through evaluation on easier motor skill tasks. We evaluate our method on two physics-based character control tasks: character morphology optimization and hyperparameter tuning of DeepMimic. Our algorithm significantly outperforms state-of-the-art hyperparameter optimization methods applicable for physics-based character animation. In particular, we show that hyperparameters optimized through our algorithm result in at least 5x efficiency gain comparing to author-released settings in DeepMimic.<\/jats:p>","DOI":"10.1145\/3451254","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T04:17:37Z","timestamp":1619669857000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Efficient Hyperparameter Optimization for Physics-based Character Animation"],"prefix":"10.1145","volume":"4","author":[{"given":"Zeshi","family":"Yang","sequence":"first","affiliation":[{"name":"Simon Fraser University, Canada"}]},{"given":"Zhiqi","family":"Yin","sequence":"additional","affiliation":[{"name":"Simon Fraser University, Canada"}]}],"member":"320","published-online":{"date-parts":[[2021,4,28]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2314658"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553380"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356536"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1015390108"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1921427.1921443"},{"key":"e_1_2_2_6_1","first-page":"10","article-title":"Preference galleries for material design","volume":"105","author":"Brochu Eric","year":"2007","journal-title":"SIGGRAPH Posters"},{"key":"e_1_2_2_7_1","volume-title":"Openai gym. arXiv preprint arXiv:1606.01540","author":"Brockman Greg","year":"2016"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781156"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964954"},{"key":"e_1_2_2_10_1","unstructured":"Erwin Coumans and Yunfei Bai. 2016--2019. PyBullet a Python module for physics simulation for games robotics and machine learning. http:\/\/pybullet.org.  Erwin Coumans and Yunfei Bai. 2016--2019. PyBullet a Python module for physics simulation for games robotics and machine learning. http:\/\/pybullet.org."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781157"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487294"},{"key":"e_1_2_2_13_1","volume-title":"Herke Van Hoof, and David Meger","author":"Fujimoto Scott","year":"2018"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508399"},{"key":"e_1_2_2_15_1","volume-title":"since","year":"2012"},{"key":"e_1_2_2_16_1","volume-title":"Reinforcement learning for improving agent design. Artificial life 25, 4","author":"David Ha.","year":"2019"},{"key":"e_1_2_2_17_1","unstructured":"Sehoon Ha Stelian Coros Alexander Alspach Joohyung Kim and Katsu Yamane. 2017. Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem.. In Robotics: Science and systems.  Sehoon Ha Stelian Coros Alexander Alspach Joohyung Kim and Katsu Yamane. 2017. Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem.. In Robotics: Science and systems."},{"key":"e_1_2_2_18_1","volume-title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint arXiv:1801.01290","author":"Haarnoja Tuomas","year":"2018"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2767002"},{"key":"e_1_2_2_20_1","volume-title":"Towards a new evolutionary computation","author":"Hansen Nikolaus"},{"key":"e_1_2_2_21_1","unstructured":"Nicolas Heess Dhruva TB Srinivasan  Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang SM Eslami etal 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017).  Nicolas Heess Dhruva TB Srinivasan Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang SM Eslami et al. 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218414"},{"key":"e_1_2_2_23_1","volume-title":"Neural fidelity warping for efficient robot morphology design. arXiv preprint arXiv:2012.04195","author":"Hu Sha","year":"2020"},{"key":"e_1_2_2_24_1","volume-title":"International Conference on Machine Learning. PMLR, 4455--4464","author":"Huang Wenlong","year":"2020"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1477926.1477936"},{"key":"e_1_2_2_26_1","volume-title":"Conference on Robot Learning. PMLR, 233--246","author":"Jaquier No\u00e9mie","year":"2020"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008306431147"},{"key":"e_1_2_2_28_1","unstructured":"Kirthevasan Kandasamy Gautam Dasarathy Junier B Oliva Jeff Schneider and Barnab\u00e1s P\u00f3czos. 2016. Gaussian process bandit optimisation with multi-fidelity evaluations. In Advances in Neural Information Processing Systems. 992--1000.  Kirthevasan Kandasamy Gautam Dasarathy Junier B Oliva Jeff Schneider and Barnab\u00e1s P\u00f3czos. 2016. Gaussian process bandit optimisation with multi-fidelity evaluations. In Advances in Neural Information Processing Systems. 992--1000."},{"key":"e_1_2_2_29_1","volume-title":"Multi-fidelity bayesian optimisation with continuous approximations. Advances in Neural Information Processing Systems","author":"Kandasamy Kirthevasan","year":"2017"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-30353-1_31"},{"key":"e_1_2_2_31_1","unstructured":"Aaron Klein Stefan Falkner Simon Bartels Philipp Hennig and Frank Hutter. 2017. Fast bayesian optimization of machine learning hyperparameters on large datasets. In Artificial Intelligence and Statistics. PMLR 528--536.  Aaron Klein Stefan Falkner Simon Bartels Philipp Hennig and Frank Hutter. 2017. Fast bayesian optimization of machine learning hyperparameters on large datasets. In Artificial Intelligence and Statistics. PMLR 528--536."},{"key":"e_1_2_2_32_1","unstructured":"Ilya Kostrikov. 2018. PyTorch Implementations of Reinforcement Learning Algorithms. https:\/\/github.com\/ikostrikov\/pytorch-a2c-ppo-acktr-gail.  Ilya Kostrikov. 2018. PyTorch Implementations of Reinforcement Learning Algorithms. https:\/\/github.com\/ikostrikov\/pytorch-a2c-ppo-acktr-gail."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392444"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073598"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793802"},{"key":"e_1_2_2_36_1","volume-title":"International Conference on Learning Representations","author":"Lillicrap Timothy P","year":"2016"},{"key":"e_1_2_2_37_1","volume-title":"Automatic design and manufacture of robotic lifeforms. Nature 406, 6799","author":"Lipson Hod","year":"2000"},{"key":"e_1_2_2_38_1","volume-title":"On the limited memory BFGS method for large scale optimization. Mathematical programming 45, 1--3","author":"Liu Dong C","year":"1989"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_2_40_1","volume-title":"Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning. In Conference on Robot Learning. PMLR, 854--869","author":"Luck Kevin Sebastian","year":"2020"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392433"},{"key":"e_1_2_2_42_1","doi-asserted-by":"crossref","unstructured":"Li-Ke Ma Zeshi Yang Tong Xin Baining Guo and KangKang Yin. 2021. Learning and Exploring Motor Skills with Spacetime Bounds. Computer Graphics Forum 40 2(2021).  Li-Ke Ma Zeshi Yang Tong Xin Baining Guo and KangKang Yin. 2021. Learning and Exploring Motor Skills with Spacetime Bounds. Computer Graphics Forum 40 2(2021).","DOI":"10.1111\/cgf.142630"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392474"},{"key":"e_1_2_2_44_1","volume-title":"International conference on machine learning. 1928--1937","author":"Mnih Volodymyr","year":"2016"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_2_46_1","volume-title":"Knowing the what but not the where in Bayesian optimization. arXiv preprint arXiv:1905.02685","author":"Nguyen Vu","year":"2019"},{"key":"e_1_2_2_47_1","volume-title":"Bayesian optimization for iterative learning. Advances in Neural Information Processing Systems 33","author":"Nguyen Vu","year":"2020"},{"key":"e_1_2_2_48_1","unstructured":"Jahng-Hyon Park and Haruhiko Asada. 1994. Concurrent design optimization of mechanical structure and control for high speed robots. (1994).  Jahng-Hyon Park and Haruhiko Asada. 1994. Concurrent design optimization of mechanical structure and control for high speed robots. (1994)."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356501"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2001.973363"},{"key":"e_1_2_2_51_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"Deepmimic: Example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng Xue Bin","year":"2018","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766910"},{"key":"e_1_2_2_53_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2897824.2925881","article-title":"Terrain-adaptive locomotion skills using deep reinforcement learning","volume":"35","author":"Peng Xue Bin","year":"2016","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073602","article-title":"Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning","volume":"36","author":"Peng Xue Bin","year":"2017","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275014"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/3516.537042"},{"key":"e_1_2_2_57_1","volume-title":"Summer School on Machine Learning","author":"Rasmussen Carl Edward"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793537"},{"key":"e_1_2_2_59_1","volume-title":"International conference on machine learning. 1889--1897","author":"Schulman John","year":"2015"},{"key":"e_1_2_2_60_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/192161.192167"},{"key":"e_1_2_2_62_1","unstructured":"Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951--2959.  Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951--2959."},{"key":"e_1_2_2_63_1","volume-title":"International conference on machine learning. 2171--2180","author":"Snoek Jasper","year":"2015"},{"key":"e_1_2_2_64_1","volume-title":"The 22nd International Conference on Artificial Intelligence and Statistics. 3158--3167","author":"Song Jialin","year":"2019"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989587"},{"key":"e_1_2_2_66_1","volume-title":"Proceedings of the 27th annual international conference on machine learning.","author":"Srinivas Niranjan","year":"2010"},{"key":"e_1_2_2_67_1","unstructured":"Kevin Swersky Jasper Snoek and Ryan P Adams. 2013. Multi-task bayesian optimization. In Advances in neural information processing systems. 2004--2012.  Kevin Swersky Jasper Snoek and Ryan P Adams. 2013. Multi-task bayesian optimization. In Advances in neural information processing systems. 2004--2012."},{"key":"e_1_2_2_68_1","volume-title":"Freeze-thaw Bayesian optimization. arXiv preprint arXiv:1406.3896","author":"Swersky Kevin","year":"2014"},{"key":"e_1_2_2_69_1","volume-title":"Multi-fidelity Bayesian optimization with max-value entropy search. arXiv preprint arXiv:1901.08275","author":"Takeno Shion","year":"2019"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2011.30"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2018.XIV.010"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386025"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322996"},{"key":"e_1_2_2_74_1","volume-title":"Computer Animation and Simulation'95","author":"de Panne Michiel Van"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2012.2208196"},{"key":"e_1_2_2_76_1","doi-asserted-by":"crossref","unstructured":"Pauli Virtanen Ralf Gommers Travis E Oliphant Matt Haberland Tyler Reddy David Cournapeau Evgeni Burovski Pearu Peterson Warren Weckesser Jonathan Bright etal 2020. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods 17 3 (2020) 261--272.  Pauli Virtanen Ralf Gommers Travis E Oliphant Matt Haberland Tyler Reddy David Cournapeau Evgeni Burovski Pearu Peterson Warren Weckesser Jonathan Bright et al. 2020. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods 17 3 (2020) 261--272.","DOI":"10.1038\/s41592-020-0772-5"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601192"},{"key":"e_1_2_2_78_1","doi-asserted-by":"crossref","unstructured":"Jack M Wang David J Fleet and Aaron Hertzmann. 2009. Optimizing walking controllers. In ACM SIGGRAPH Asia 2009 papers. 1--8.  Jack M Wang David J Fleet and Aaron Hertzmann. 2009. Optimizing walking controllers. In ACM SIGGRAPH Asia 2009 papers. 1--8.","DOI":"10.1145\/1661412.1618514"},{"key":"e_1_2_2_79_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185521","article-title":"Optimizing locomotion controllers using biologically-based actuators and objectives","volume":"31","author":"Wang Jack M","year":"2012","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_80_1","volume-title":"International Conference on Learning Representations.","author":"Wang Tingwu","year":"2018"},{"key":"e_1_2_2_81_1","volume-title":"Nando De Freitas, et al","author":"Wang Ziyu","year":"2013"},{"key":"e_1_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392381"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356499"},{"key":"e_1_2_2_84_1","first-page":"1","article-title":"Terrain-adaptive bipedal locomotion control","volume":"29","author":"Zoran Popovi\u0107 Wu","year":"2010","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_85_1","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation.","author":"Xie Zhaoming"},{"key":"e_1_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360680"},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276509"},{"key":"e_1_2_2_88_1","volume-title":"Greg Turk, and C Karen Liu.","author":"Yu Wenhao","year":"2019"},{"key":"e_1_2_2_89_1","first-page":"1","article-title":"Learning symmetric and low-energy locomotion","volume":"37","author":"Yu Wenhao","year":"2018","journal-title":"ACM Transactions on Graphics (TOG)"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3451254","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3451254","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:25Z","timestamp":1750268965000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3451254"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,26]]},"references-count":89,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,4,26]]}},"alternative-id":["10.1145\/3451254"],"URL":"https:\/\/doi.org\/10.1145\/3451254","relation":{},"ISSN":["2577-6193"],"issn-type":[{"value":"2577-6193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,26]]},"assertion":[{"value":"2021-04-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}