{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T02:23:27Z","timestamp":1769912607299,"version":"3.49.0"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2017,11,20]],"date-time":"2017-11-20T00:00:00Z","timestamp":1511136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"MSIT(Ministry of Science and ICT) of Korea","award":["0536-20170040"],"award-info":[{"award-number":["0536-20170040"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2017,12,31]]},"abstract":"<jats:p>\n            Imaginary winged creatures in computer animation applications are expected to perform a variety of motor skills in a physically realistic and controllable manner. Designing physics-based controllers for a flying creature is still very challenging particularly when the dynamic model of the creatures is high-dimensional, having many degrees of freedom. In this paper, we present a control method for flying creatures, which are aerodynamically simulated, interactively controllable, and equipped with a variety of motor skills such as soaring, gliding, hovering, and diving. Each motor skill is represented as Deep Neural Networks (DNN) and learned using Deep Q-Learning (DQL). Our control method is example-guided in the sense that it provides the user with direct control over the learning process by allowing the user to specify keyframes of motor skills. Our novel learning algorithm was inspired by evolutionary strategies of Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to improve the convergence rate and the final quality of the control policy. The effectiveness of our\n            <jats:italic>Evolutionary DQL<\/jats:italic>\n            method is demonstrated with imaginary winged creatures flying in a physically simulated environment and their motor skills learned automatically from user-provided keyframes.\n          <\/jats:p>","DOI":"10.1145\/3130800.3130833","type":"journal-article","created":{"date-parts":[[2017,11,22]],"date-time":"2017-11-22T16:25:08Z","timestamp":1511367908000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":46,"title":["How to train your dragon"],"prefix":"10.1145","volume":"36","author":[{"given":"Jungdam","family":"Won","sequence":"first","affiliation":[{"name":"Seoul National University"}]},{"given":"Jongho","family":"Park","sequence":"additional","affiliation":[{"name":"Seoul National University"}]},{"given":"Kwanyu","family":"Kim","sequence":"additional","affiliation":[{"name":"Seoul National University"}]},{"given":"Jehee","family":"Lee","sequence":"additional","affiliation":[{"name":"Seoul National University"}]}],"member":"320","published-online":{"date-parts":[[2017,11,20]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.325"},{"key":"e_1_2_2_2_1","unstructured":"Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. (2016).  Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. (2016)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618516"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964954"},{"key":"e_1_2_2_5_1","volume-title":"Dart: Dynamic Animation and Robotics Toolkit. https:\/\/dartsim.github.io\/.","year":"2012"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781157"},{"key":"e_1_2_2_7_1","unstructured":"Yan Duan Xi Chen Rein Houthooft John Schulman and Pieter Abbeel. 2016. Benchmarking Deep Reinforcement Learning for Continuous Control. CoRR abs\/1604.06778 (2016).  Yan Duan Xi Chen Rein Houthooft John Schulman and Pieter Abbeel. 2016. Benchmarking Deep Reinforcement Learning for Continuous Control. CoRR abs\/1604.06778 (2016)."},{"key":"e_1_2_2_8_1","volume-title":"In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATSfi10)","author":"Glorot Xavier","year":"2010"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280816"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682626"},{"key":"e_1_2_2_11_1","volume-title":"Annual Conference on Neural Information Processing Systems (NIPS","author":"Heess Nicolas","year":"2015"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2516971.2516976"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1028523.1028535"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1781155"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661233"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882261.1866160"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/2946645.2946684"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning (ICML","author":"Levine Sergey","year":"2014"},{"key":"e_1_2_2_19_1","unstructured":"Timothy P. Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. CoRR abs\/1509.02971 (2015).  Timothy P. Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. CoRR abs\/1509.02971 (2015)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2893476"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_2_22_1","volume-title":"Robotics: Science and Systems (RSS","author":"Mordatch Igor","year":"2014"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925881"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_26_1","unstructured":"Xue Bin Peng and Michiel van de Panne. 2016. Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter? CoRR abs\/1611.01055 (2016).  Xue Bin Peng and Michiel van de Panne. 2016. Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter? CoRR abs\/1611.01055 (2016)."},{"key":"e_1_2_2_27_1","unstructured":"Tom Schaul John Quan Ioannis Antonoglou and David Silver. 2015. Prioritized Experience Replay. CoRR abs\/1511.05952 (2015).  Tom Schaul John Quan Ioannis Antonoglou and David Silver. 2015. Prioritized Experience Replay. CoRR abs\/1511.05952 (2015)."},{"key":"e_1_2_2_28_1","unstructured":"John Schulman Sergey Levine Philipp Moritz Michael I. Jordan and Pieter Abbeel. 2015a. Trust Region Policy Optimization. CoRR abs\/1502.05477 (2015).   John Schulman Sergey Levine Philipp Moritz Michael I. Jordan and Pieter Abbeel. 2015a. Trust Region Policy Optimization. CoRR abs\/1502.05477 (2015)."},{"key":"e_1_2_2_29_1","unstructured":"John Schulman Philipp Moritz Sergey Levine Michael I. Jordan and Pieter Abbeel. 2015b. High-Dimensional Continuous Control Using Generalized Advantage Estimation. CoRR abs\/1506.02438 (2015).  John Schulman Philipp Moritz Sergey Levine Michael I. Jordan and Pieter Abbeel. 2015b. High-Dimensional Continuous Control Using Generalized Advantage Estimation. CoRR abs\/1506.02438 (2015)."},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning (ICML","author":"Silver David","year":"2014"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276511"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601121"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964953"},{"key":"e_1_2_2_34_1","unstructured":"TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). http:\/\/tensorflow.org\/ Software availablefromtensorflow.org.  TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). http:\/\/tensorflow.org\/ Software availablefromtensorflow.org."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276386"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ADPRL.2007.368199"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778810"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185521"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882360"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276509"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3130800.3130833","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3130800.3130833","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:26:18Z","timestamp":1750213578000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3130800.3130833"}},"subtitle":["example-guided control of flapping flight"],"short-title":[],"issued":{"date-parts":[[2017,11,20]]},"references-count":40,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2017,12,31]]}},"alternative-id":["10.1145\/3130800.3130833"],"URL":"https:\/\/doi.org\/10.1145\/3130800.3130833","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,20]]},"assertion":[{"value":"2017-11-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}