{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T08:05:56Z","timestamp":1790841956735,"version":"4.1.0"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,7,19]],"date-time":"2021-07-19T00:00:00Z","timestamp":1626652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>\n                    In two-player competitive sports, such as\n                    <jats:italic>boxing<\/jats:italic>\n                    and\n                    <jats:italic>fencing<\/jats:italic>\n                    , athletes often demonstrate efficient and tactical movements during a competition. In this paper, we develop a learning framework that generates control policies for physically simulated athletes who have many degrees-of-freedom. Our framework uses a two step-approach, learning basic skills and learning bout-level strategies, with deep reinforcement learning, which is inspired by the way that people how to learn competitive sports. We develop a policy model based on an encoder-decoder structure that incorporates an autoregressive latent variable, and a mixture-of-experts decoder. To show the effectiveness of our framework, we implemented two competitive sports,\n                    <jats:italic>boxing<\/jats:italic>\n                    and\n                    <jats:italic>fencing<\/jats:italic>\n                    , and demonstrate control policies learned by our framework that can generate both tactical and natural-looking behaviors. We also evaluate the control policies with comparisons to other learning configurations and with ablation studies.\n                  <\/jats:p>","DOI":"10.1145\/3450626.3459761","type":"journal-article","created":{"date-parts":[[2021,7,19]],"date-time":"2021-07-19T20:04:26Z","timestamp":1626725066000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["Control strategies for physically simulated characters performing two-player competitive sports"],"prefix":"10.1145","volume":"40","author":[{"given":"Jungdam","family":"Won","sequence":"first","affiliation":[{"name":"Facebook AI Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deepak","family":"Gopinath","sequence":"additional","affiliation":[{"name":"Facebook AI Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Hodgins","sequence":"additional","affiliation":[{"name":"Facebook AI Research"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,7,19]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Bai Yu","year":"2020","unstructured":"Yu Bai and Chi Jin . 2020 . Provable Self-Play Algorithms for Competitive Reinforcement Learning . In Proceedings of the 37th International Conference on Machine Learning , Vol. 119 . PMLR, 551--560. http:\/\/proceedings.mlr.press\/v119\/bai20a.html Yu Bai and Chi Jin. 2020. Provable Self-Play Algorithms for Competitive Reinforcement Learning. In Proceedings of the 37th International Conference on Machine Learning, Vol. 119. PMLR, 551--560. http:\/\/proceedings.mlr.press\/v119\/bai20a.html"},{"key":"e_1_2_2_2_1","volume-title":"Emergent Tool Use From Multi-Agent Autocurricula. CoRR","author":"Baker Bowen","year":"2019","unstructured":"Bowen Baker , Ingmar Kanitscheider , Todor M. Markov , Yi Wu , Glenn Powell , Bob McGrew , and Igor Mordatch . 2019. Emergent Tool Use From Multi-Agent Autocurricula. CoRR ( 2019 ). arXiv:1909.07528 Bowen Baker, Ingmar Kanitscheider, Todor M. Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. 2019. Emergent Tool Use From Multi-Agent Autocurricula. CoRR (2019). arXiv:1909.07528"},{"key":"e_1_2_2_3_1","unstructured":"Trapit Bansal Jakub Pachocki Szymon Sidor Ilya Sutskever and Igor Mordatch. 2018. Emergent Complexity via Multi-Agent Competition. arXiv:1710.03748  Trapit Bansal Jakub Pachocki Szymon Sidor Ilya Sutskever and Igor Mordatch. 2018. Emergent Complexity via Multi-Agent Competition. arXiv:1710.03748"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356536"},{"key":"e_1_2_2_5_1","volume-title":"Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control. CoRR abs\/1802.04765","author":"Berseth Glen","year":"2018","unstructured":"Glen Berseth , Cheng Xie , Paul Cernek , and Michiel van de Panne . 2018. Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control. CoRR abs\/1802.04765 ( 2018 ). Glen Berseth, Cheng Xie, Paul Cernek, and Michiel van de Panne. 2018. Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Control. CoRR abs\/1802.04765 (2018)."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2007.913919"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274247.3274506"},{"key":"e_1_2_2_8_1","volume-title":"Proceedings of the Fifteenth National\/Tenth Conference on Artificial Intelligence\/Innovative Applications of Artificial Intelligence (AAAI '98\/IAAI '98)","author":"Claus Caroline","year":"1998","unstructured":"Caroline Claus and Craig Boutilier . 1998 . The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems . In Proceedings of the Fifteenth National\/Tenth Conference on Artificial Intelligence\/Innovative Applications of Artificial Intelligence (AAAI '98\/IAAI '98) . 746--752. Caroline Claus and Craig Boutilier. 1998. The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. In Proceedings of the Fifteenth National\/Tenth Conference on Artificial Intelligence\/Innovative Applications of Artificial Intelligence (AAAI '98\/IAAI '98). 746--752."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275048"},{"key":"e_1_2_2_10_1","unstructured":"CMU. 2002. CMU Graphics Lab Motion Capture Database. http:\/\/mocap.cs.cmu.edu\/.  CMU. 2002. CMU Graphics Lab Motion Capture Database. http:\/\/mocap.cs.cmu.edu\/."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3424636.3426894"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.116"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778770"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073663"},{"key":"e_1_2_2_15_1","volume-title":"Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98)","author":"Hu Junling","unstructured":"Junling Hu and Michael P. Wellman . 1998. Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm . In Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98) . 242--250. Junling Hu and Michael P. Wellman. 1998. Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98). 242--250."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.80"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601170"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531385"},{"key":"e_1_2_2_19_1","volume-title":"How to Train Your DRAGAN. CoRR abs\/1705.07215","author":"Kodali Naveen","year":"2017","unstructured":"Naveen Kodali , Jacob D. Abernethy , James Hays , and Zsolt Kira . 2017. How to Train Your DRAGAN. CoRR abs\/1705.07215 ( 2017 ). Naveen Kodali, Jacob D. Abernethy, James Hays, and Zsolt Kira. 2017. How to Train Your DRAGAN. CoRR abs\/1705.07215 (2017)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2008.22"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360679"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1028523.1028535"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141972"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322972"},{"key":"e_1_2_2_25_1","unstructured":"Eric Liang Richard Liaw Philipp Moritz Robert Nishihara Roy Fox Ken Goldberg Joseph E. Gonzalez Michael I. Jordan and Ion Stoica. 2018. RLlib: Abstractions for Distributed Reinforcement Learning. arXiv:1712.09381  Eric Liang Richard Liaw Philipp Moritz Robert Nishihara Roy Fox Ken Goldberg Joseph E. Gonzalez Michael I. Jordan and Ion Stoica. 2018. RLlib: Abstractions for Distributed Reinforcement Learning. arXiv:1712.09381"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-55860-335-6.50027-1"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1218064.1218093"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083723"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201315"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295385"},{"key":"e_1_2_2_31_1","volume-title":"Learning human behaviors from motion capture by adversarial imitation. CoRR abs\/1707.02201","author":"Merel Josh","year":"2017","unstructured":"Josh Merel , Yuval Tassa , Dhruva TB , Sriram Sr inivasan, Jay Lemmon , Ziyu Wang , Greg Wayne , and Nicolas Heess . 2017. Learning human behaviors from motion capture by adversarial imitation. CoRR abs\/1707.02201 ( 2017 ). Josh Merel, Yuval Tassa, Dhruva TB, Sriram Srinivasan, Jay Lemmon, Ziyu Wang, Greg Wayne, and Nicolas Heess. 2017. Learning human behaviors from motion capture by adversarial imitation. CoRR abs\/1707.02201 (2017)."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2020.2977374"},{"key":"e_1_2_2_33_1","unstructured":"Afshin OroojlooyJadid and Davood Hajinezhad. 2020. A Review of Cooperative Multi-Agent Deep Reinforcement Learning. arXiv:1908.03963  Afshin OroojlooyJadid and Davood Hajinezhad. 2020. A Review of Cooperative Multi-Agent Deep Reinforcement Learning. arXiv:1908.03963"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356501"},{"key":"e_1_2_2_35_1","first-page":"8024","article-title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas Kopf , Edward Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019 . PyTorch: An Imperative Style, High-Performance Deep Learning Library . In Advances in Neural Information Processing Systems 32. 8024 -- 8035 . Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. 8024--8035.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201311"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_38_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"1295","author":"Seldin Yevgeny","year":"2014","unstructured":"Yevgeny Seldin and Aleksandrs Slivkins . 2014 . One Practical Algorithm for Both Stochastic and Adversarial Bandits . In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research , Vol. 32). 1287-- 1295 . http:\/\/proceedings.mlr.press\/v32\/seldinb14.html Yevgeny Seldin and Aleksandrs Slivkins. 2014. One Practical Algorithm for Both Stochastic and Adversarial Bandits. In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 32). 1287--1295. http:\/\/proceedings.mlr.press\/v32\/seldinb14.html"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409067"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1315184.1315194"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1342250.1342271"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.257"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356505"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2011.30"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1805964.1805970"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295284"},{"key":"e_1_2_2_47_1","volume-title":"8th International Conference on Learning Representations, ICLR 2020","author":"Wijmans Erik","year":"2020","unstructured":"Erik Wijmans , Abhishek Kadian , Ari Morcos , Stefan Lee , Irfan Essa , Devi Parikh , Manolis Savva , and Dhruv Batra . 2020 . DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames . In 8th International Conference on Learning Representations, ICLR 2020 , Addis Ababa, Ethiopia , April 26-30, 2020. Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. 2020. DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392381"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356499"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661271"},{"key":"e_1_2_2_51_1","volume-title":"Proc. ACM SIGGRAPH \/ Eurographics Symposium on Computer Animation.","author":"Xie Zhaoming","unstructured":"Zhaoming Xie , Hung Yu Ling , Nam Hee Kim , and Michiel van de Panne. 2020. ALL-STEPS: Curriculum-driven Learning of Stepping Stone Skills . In Proc. ACM SIGGRAPH \/ Eurographics Symposium on Computer Animation. Zhaoming Xie, Hung Yu Ling, Nam Hee Kim, and Michiel van de Panne. 2020. ALL-STEPS: Curriculum-driven Learning of Stepping Stone Skills. In Proc. ACM SIGGRAPH \/ Eurographics Symposium on Computer Animation."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1507149.1507184"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201397"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/545261.545276"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073249"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459761","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3450626.3459761","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:17:16Z","timestamp":1750177036000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459761"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,19]]},"references-count":55,"aliases":["10.1145\/3476576.3476725"],"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3450626.3459761"],"URL":"https:\/\/doi.org\/10.1145\/3450626.3459761","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,19]]},"assertion":[{"value":"2021-07-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}