{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:19:20Z","timestamp":1750220360457,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,3]],"date-time":"2021-08-03T00:00:00Z","timestamp":1627948800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8,3]]},"DOI":"10.1145\/3472538.3472583","type":"proceedings-article","created":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T22:48:54Z","timestamp":1634856534000},"page":"1-5","source":"Crossref","is-referenced-by-count":0,"title":["Modular Reinforcement Learning Framework for Learners and Educators"],"prefix":"10.1145","author":[{"given":"Rachael","family":"Versaw","sequence":"first","affiliation":[{"name":"The Pennsylvania State University, United States"}]},{"given":"Samantha","family":"Schultz","sequence":"additional","affiliation":[{"name":"The Pennsylvania State University, United States"}]},{"given":"Kevin","family":"Lu","sequence":"additional","affiliation":[{"name":"The Pennsylvania State University, United States"}]},{"given":"Richard","family":"Zhao","sequence":"additional","affiliation":[{"name":"University of Calgary, Canada"}]}],"member":"320","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2014.2367105"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015","author":"Bontrager Philip","year":"2019","unstructured":"Philip Bontrager , Ahmed Khalifa , Damien Anderson , Matthew Stephenson , Christoph Salge , and Julian Togelius . 2019 . \u201d Superstition\u201d in the Network: Deep Reinforcement Learning Plays Deceptive Games . In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015 . 10\u201316. Philip Bontrager, Ahmed Khalifa, Damien Anderson, Matthew Stephenson, Christoph Salge, and Julian Togelius. 2019. \u201d Superstition\u201d in the Network: Deep Reinforcement Learning Plays Deceptive Games. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015. 10\u201316."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2006.92"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2014.2369345"},{"key":"e_1_3_2_1_5_1","volume-title":"International conference on machine learning. PMLR, 1515\u20131528","author":"Florensa Carlos","year":"2018","unstructured":"Carlos Florensa , David Held , Xinyang Geng , and Pieter Abbeel . 2018 . Automatic goal generation for reinforcement learning agents . In International conference on machine learning. PMLR, 1515\u20131528 . Carlos Florensa, David Held, Xinyang Geng, and Pieter Abbeel. 2018. Automatic goal generation for reinforcement learning agents. In International conference on machine learning. PMLR, 1515\u20131528."},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015","author":"Frazier Spencer","year":"2019","unstructured":"Spencer Frazier and Mark Riedl . 2019 . Improving deep reinforcement learning in minecraft with action advice . In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015 . 146\u2013152. Spencer Frazier and Mark Riedl. 2019. Improving deep reinforcement learning in minecraft with action advice. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015. 146\u2013152."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2014.2363042"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1822348.1822359"},{"key":"e_1_3_2_1_9_1","volume-title":"Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627(2018).","author":"Juliani Arthur","year":"2018","unstructured":"Arthur Juliani , Vincent-Pierre Berges , Ervin Teng , Andrew Cohen , Jonathan Harper , Chris Elion , Chris Goy , Yuan Gao , Hunter Henry , Marwan Mattar , 2018 . Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627(2018). Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, 2018. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627(2018)."},{"key":"e_1_3_2_1_10_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a014","author":"Lee Dennis","year":"2018","unstructured":"Dennis Lee , Haoran Tang , Jeffrey Zhang , Huazhe Xu , Trevor Darrell , and Pieter Abbeel . 2018 . Modular architecture for starcraft ii with deep reinforcement learning . In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a014 . Dennis Lee, Haoran Tang, Jeffrey Zhang, Huazhe Xu, Trevor Darrell, and Pieter Abbeel. 2018. Modular architecture for starcraft ii with deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a014."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3337722.3337740"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/10494820.2018.1525411"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2004.1342727"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1836135.1836141"},{"key":"e_1_3_2_1_15_1","volume-title":"International conference on machine learning. PMLR","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih , Adria\u00a0Puigdomenech Badia , Mehdi Mirza , Alex Graves , Timothy Lillicrap , Tim Harley , David Silver , and Koray Kavukcuoglu . 2016 . Asynchronous methods for deep reinforcement learning . In International conference on machine learning. PMLR , 1928\u20131937. Volodymyr Mnih, Adria\u00a0Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International conference on machine learning. PMLR, 1928\u20131937."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0L Naps Guido R\u00f6\u00dfling Vicki Almstrum Wanda Dann Rudolf Fleischer Chris Hundhausen Ari Korhonen Lauri Malmi Myles McNally Susan Rodger 2002. Exploring the role of visualization and engagement in computer science education. In Working group reports from ITiCSE on Innovation and technology in computer science education. 131\u2013152.  Thomas\u00a0L Naps Guido R\u00f6\u00dfling Vicki Almstrum Wanda Dann Rudolf Fleischer Chris Hundhausen Ari Korhonen Lauri Malmi Myles McNally Susan Rodger 2002. Exploring the role of visualization and engagement in computer science education. In Working group reports from ITiCSE on Innovation and technology in computer science education. 131\u2013152.","DOI":"10.1145\/782941.782998"},{"key":"e_1_3_2_1_17_1","volume-title":"Hierarchical reinforcement learning with monte carlo tree search in computer fighting game","author":"Pinto Ivan\u00a0Pereira","year":"2018","unstructured":"Ivan\u00a0Pereira Pinto and Luciano\u00a0Reis Coutinho . 2018. Hierarchical reinforcement learning with monte carlo tree search in computer fighting game . IEEE transactions on games 11, 3 ( 2018 ), 290\u2013295. Ivan\u00a0Pereira Pinto and Luciano\u00a0Reis Coutinho. 2018. Hierarchical reinforcement learning with monte carlo tree search in computer fighting game. IEEE transactions on games 11, 3 (2018), 290\u2013295."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2007.09.020"},{"volume-title":"On-line Q-learning using connectionist systems. Vol.\u00a037","author":"Rummery A","key":"e_1_3_2_1_19_1","unstructured":"Gavin\u00a0 A Rummery and Mahesan Niranjan . 1994. On-line Q-learning using connectionist systems. Vol.\u00a037 . University of Cambridge , Department of Engineering Cambridge, UK. Gavin\u00a0A Rummery and Mahesan Niranjan. 1994. On-line Q-learning using connectionist systems. Vol.\u00a037. University of Cambridge, Department of Engineering Cambridge, UK."},{"key":"e_1_3_2_1_20_1","volume-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 6419","author":"Silver David","year":"2018","unstructured":"David Silver , Thomas Hubert , Julian Schrittwieser , Ioannis Antonoglou , Matthew Lai , Arthur Guez , Marc Lanctot , Laurent Sifre , Dharshan Kumaran , Thore Graepel , 2018. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 6419 ( 2018 ), 1140\u20131144. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, 2018. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 6419 (2018), 1140\u20131144."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3402942.3409789"},{"volume-title":"Reinforcement learning: An introduction","author":"Sutton S","key":"e_1_3_2_1_22_1","unstructured":"Richard\u00a0 S Sutton and Andrew\u00a0 G Barto . 2018. Reinforcement learning: An introduction . MIT press . Richard\u00a0S Sutton and Andrew\u00a0G Barto. 2018. Reinforcement learning: An introduction. MIT press."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CIG.2018.8490422"},{"key":"e_1_3_2_1_24_1","volume-title":"Proceedings of the International Conference on the Foundations of Digital Games (FDG).","author":"Treanor Mike","year":"2015","unstructured":"Mike Treanor , Alexander Zook , Mirjam\u00a0 P Eladhari , Julian Togelius , Gillian Smith , Michael Cook , Tommy Thompson , Brian Magerko , John Levine , and Adam Smith . 2015 . AI-based game design patterns . In Proceedings of the International Conference on the Foundations of Digital Games (FDG). Mike Treanor, Alexander Zook, Mirjam\u00a0P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian Magerko, John Levine, and Adam Smith. 2015. AI-based game design patterns. In Proceedings of the International Conference on the Foundations of Digital Games (FDG)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69736-7_23"},{"key":"e_1_3_2_1_26_1","volume-title":"Proceedings of the International Conference on the Foundations of Digital Games (FDG).","author":"van Rozen Riemer","year":"2015","unstructured":"Riemer van Rozen . 2015 . A Pattern-Based Game Mechanics Design Assistant . In Proceedings of the International Conference on the Foundations of Digital Games (FDG). Riemer van Rozen. 2015. A Pattern-Based Game Mechanics Design Assistant. In Proceedings of the International Conference on the Foundations of Digital Games (FDG)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"e_1_3_2_1_28_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence, Vol.\u00a030","author":"Wollowski Michael","year":"2016","unstructured":"Michael Wollowski , Robert Selkowitz , Laura Brown , Ashok Goel , George Luger , Jim Marshall , Andrew Neel , Todd Neller , and Peter Norvig . 2016 . A survey of current practice and teaching of AI . In Proceedings of the AAAI Conference on Artificial Intelligence, Vol.\u00a030 . Michael Wollowski, Robert Selkowitz, Laura Brown, Ashok Goel, George Luger, Jim Marshall, Andrew Neel, Todd Neller, and Peter Norvig. 2016. A survey of current practice and teaching of AI. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol.\u00a030."},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015","author":"Xu Sijia","year":"2019","unstructured":"Sijia Xu , Hongyu Kuang , Zhuang Zhi , Renjie Hu , Yang Liu , and Huyang Sun . 2019 . Macro action selection with deep reinforcement learning in starcraft . In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015 . 94\u201399. Sijia Xu, Hongyu Kuang, Zhuang Zhi, Renjie Hu, Yang Liu, and Huyang Sun. 2019. Macro action selection with deep reinforcement learning in starcraft. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol.\u00a015. 94\u201399."},{"volume-title":"Artificial intelligence and games. Vol.\u00a02","author":"Yannakakis N","key":"e_1_3_2_1_30_1","unstructured":"Georgios\u00a0 N Yannakakis and Julian Togelius . 2018. Artificial intelligence and games. Vol.\u00a02 . Springer . Georgios\u00a0N Yannakakis and Julian Togelius. 2018. Artificial intelligence and games. Vol.\u00a02. Springer."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2016.2593710"}],"event":{"name":"FDG'21: The 16th International Conference on the Foundations of Digital Games 2021","acronym":"FDG'21","location":"Montreal QC Canada"},"container-title":["The 16th International Conference on the Foundations of Digital Games (FDG) 2021"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3472538.3472583","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3472538.3472583","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:24Z","timestamp":1750191444000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3472538.3472583"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,3]]},"references-count":31,"alternative-id":["10.1145\/3472538.3472583","10.1145\/3472538"],"URL":"https:\/\/doi.org\/10.1145\/3472538.3472583","relation":{},"subject":[],"published":{"date-parts":[[2021,8,3]]}}}