{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T14:11:05Z","timestamp":1771855865315,"version":"3.50.1"},"reference-count":303,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2020,6,30]],"date-time":"2020-06-30T00:00:00Z","timestamp":1593475200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["SFRH\/BD\/145723\/2019"],"award-info":[{"award-number":["SFRH\/BD\/145723\/2019"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>In general, games pose interesting and complex problems for the implementation of intelligent agents and are a popular domain in the study of artificial intelligence. In fact, games have been at the center of some of the most well-known achievements in artificial intelligence. From classical board games such as chess, checkers, backgammon and Go, to video games such as Dota 2 and StarCraft II, artificial intelligence research has devised computer programs that can play at the level of a human master and even at a human world champion level. Planning and learning, two well-known and successful paradigms of artificial intelligence, have greatly contributed to these achievements. Although representing distinct approaches, planning and learning try to solve similar problems and share some similarities. They can even complement each other. This has led to research on methodologies to combine the strengths of both approaches to derive better solutions. This paper presents a survey of the multiple methodologies that have been proposed to integrate planning and learning in the context of games. In order to provide a richer contextualization, the paper also presents learning and planning techniques commonly used in games, both in terms of their theoretical foundations and applications.<\/jats:p>","DOI":"10.3390\/app10134529","type":"journal-article","created":{"date-parts":[[2020,6,30]],"date-time":"2020-06-30T09:04:59Z","timestamp":1593507899000},"page":"4529","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["A Survey of Planning and Learning in Games"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9503-9084","authenticated-orcid":false,"given":"Fernando Fradique","family":"Duarte","sequence":"first","affiliation":[{"name":"Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Nuno","family":"Lau","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7099-1247","authenticated-orcid":false,"given":"Artur","family":"Pereira","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Luis Paulo","family":"Reis","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, Department of Informatics Engineering, University of Porto, 4099-002 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1109\/TG.2017.2737145","article-title":"Pac-Man Conquers Academia: Two Decades of Research Using a Classic Arcade Game","volume":"10","author":"Rohlfshagen","year":"2018","journal-title":"IEEE Trans. Games"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1109\/TG.2019.2901021","article-title":"General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms","volume":"11","author":"Liu","year":"2019","journal-title":"IEEE Trans. Games"},{"key":"ref_3","first-page":"107","article-title":"The International General Game Playing Competition","volume":"34","author":"Genesereth","year":"2013","journal-title":"AI Mag."},{"key":"ref_4","unstructured":"Prada, R., Lopes, P., Catarino, J., Quiterio, J., and Melo, F.S. (September, January 31). The Geometry Friends Game AI Competition. Proceedings of the Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1177\/105971239700500305","article-title":"Incremental Evolution of Complex General Behavior","volume":"5","author":"Gomez","year":"1997","journal-title":"Adapt. Behav."},{"key":"ref_6","unstructured":"Asensio, J.M.L., Donate, J.P., and Cortez, P. (2014, January 26\u201329). Evolving Artificial Neural Networks Applied to Generate Virtual Characters. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mahmoud, I.M., Li, L., Wloka, D., and Ali, M.Z. (2014, January 26\u201329). Believable NPCs In Serious Games: HTN Planning Approach Based on Visual Perception. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932891"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Torrey, L. (2010, January 11\u201313). Crowd Simulation Via Multi-Agent Reinforcement Learning. Proceedings of the 6th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v6i1.12390"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0004-3702(01)00129-1","article-title":"Deep Blue","volume":"134","author":"Campbell","year":"2002","journal-title":"Artif. Intell."},{"key":"ref_10","first-page":"21","article-title":"CHINOOK The World Man-Machine Checkers Champion","volume":"17","author":"Schaeffer","year":"1996","journal-title":"AI Mag."},{"key":"ref_11","unstructured":"Buro, M. (1998, January 11\u201312). From Simple Features to Sophisticated Evaluation Functions. Proceedings of the International Conference on Computers and Games, Tsukuba, Japan."},{"key":"ref_12","unstructured":"Russell, S.J., and Norvig, P. (2010). Artificial Intelligence A Modern Approach, Prentice Hall."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1023\/A:1007634325138","article-title":"Learning to Play Chess Using Temporal Differences","volume":"40","author":"Baxter","year":"2000","journal-title":"Mach. Learn."},{"key":"ref_14","unstructured":"Veness, J., Silver, D., Uther, W., and Blair, A. (2009, January 7\u201310). Bootstrapping from Game Tree Search. Proceedings of the 22nd International Conference on Neural Information Processing Systems, Vancouver, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1162\/neco.1994.6.2.215","article-title":"TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play","volume":"6","author":"Tesauro","year":"1994","journal-title":"Neural Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_18","unstructured":"Berner, C., Brockman, G., Chan, B., Cheung, V., D\u0119biak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., and Hesse, C. (2019). Dota 2 with Large Scale Deep Reinforcement Learning. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Arulkumaran, K., Cully, A., and Togelius, J. (2019, January 13\u201317). Alphastar: An evolutionary computation perspective. Proceedings of the Genetic and Evolutionary Computation Conference Companion, Prague, Czech Republic.","DOI":"10.1145\/3319619.3321894"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1017\/S026988891200001X","article-title":"A Review of Machine Learning for Automated Planning","volume":"27","author":"Borrajo","year":"2012","journal-title":"Knowl. Eng. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Efthymiadis, K., and Kudenko, D. (2013, January 11\u201313). Using Plan-Based Reward Shaping To Learn Strategies in StarCraft: Broodwar. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633622"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Vlahavas, I., and Vrakas, D. (2008). Reinforcement Learning and Automated Planning: A Survey. Artificial Intelligence for Advanced Problem Solving Techniques, IGI Global.","DOI":"10.4018\/978-1-59904-705-8"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.entcom.2016.01.002","article-title":"Towards a system of customized video game mechanics based on player personality: Relating the Big Five personality traits with difficulty adaptation in a first-person shooter game","volume":"13","author":"Nagle","year":"2016","journal-title":"Entertain. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gama, J., Santos Costa, V., Jorge, A., and Brazdil, P. (2009, January 3\u20135). Player Modeling for Intelligent Difficulty Adjustment. Proceedings of the International Conference on Discovery Science, Porto, Portugal.","DOI":"10.1007\/978-3-642-04747-3"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kaidan, M., Harada, T., Chu, C.Y., and Thawonmas, R. (2016, January 24\u201329). Procedural Generation of Angry Birds Levels with Adjustable Difficulty. Proceedings of the IEEE Congress on Evolutionary Computation, Vancouver, Canada.","DOI":"10.1109\/CEC.2016.7743939"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hao, Y., He, S., Wang, J., Liu, X., Yang, J., and Huang, W. (2010, January 10\u201312). Dynamic Difficulty Adjustment of Game AI by MCTS for the Game Pac-Man. Proceedings of the Sixth International Conference on Natural Computation, Yantai, China.","DOI":"10.1109\/ICNC.2010.5584761"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kim, H.T., and Kim, K.J. (2014, January 26\u201329). Learning to Recommend Game Contents for Real-Time Strategy Garners. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932883"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Justesen, N., and Risi, S. (2017, January 22\u201325). Learning Macromanagement in StarCraft from Replays using Deep Learning. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080430"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Weber, B.G., and Mateas, M. (2009, January 7\u201310). A Data Mining Approach to Strategy Prediction. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Milano, Italy.","DOI":"10.1109\/CIG.2009.5286483"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yamamoto, K., Mizuno, S., Chu, C.Y., and Thawonmas, R. (2014, January 26\u201329). Deduction of Fighting-Game Countermeasures Using the k-Nearest Neighbor Algorithm and a Game Simulator. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932915"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kristo, T., and Maulidevi, N.U. (2016, January 26\u201327). Deduction of Fighting Game Countermeasures using Neuroevolution of Augmenting Topologies. Proceedings of the International Conference on Data and Software Engineering, Denpasar, Indonesia.","DOI":"10.1109\/ICODSE.2016.7936127"},{"key":"ref_32","unstructured":"Valls-Vargas, J., Onta\u00f1\u00f3n, S., and Zhu, J. (2015, January 15\u201318). Exploring Player Trace Segmentation for Dynamic Play Style Prediction. Proceedings of the 11th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Santa Cruz, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Mahlmann, T., Drachen, A., Togelius, J., Canossa, A., and Yannakakis, G.N. (2010, January 18\u201321). Predicting Player Behavior in Tomb Raider: Underworld. Proceedings of the IEEE Conference on Computational Intelligence and Games, Dublin, Ireland.","DOI":"10.1109\/ITW.2010.5593355"},{"key":"ref_34","unstructured":"Chen, Z., El-nasr, M.S., Canossa, A., Badler, J., Tignor, S., and Colvin, R. (2015, January 15\u201318). Modeling Individual Differences through Frequent Pattern Mining on Role-Playing Game Actions. Proceedings of the 11th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AAAI Technical Report WS-15-23, Santa Cruz, CA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1109\/TSMCA.2009.2028152","article-title":"Preference Learning for Cognitive Modeling: A Case Study on Entertainment Preferences","volume":"39","author":"Yannakakis","year":"2009","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_36","unstructured":"Packard, B., and Onta\u00f1\u00f3n, S. (2017, January 27\u201329). Policies for Active Learning from Demonstration. Proceedings of the AAAI Spring Symposium on Learning from Observation of Humans Technical Report SS-17-06, Palo Alto, CA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Packard, B., and Onta\u00f1\u00f3n, S. (2018, January 21\u201323). Learning Behavior from Limited Demonstrations in the Context of Games. Proceedings of the 31st International Florida Artificial Intelligence Research Society Conference, Melbourne, FL, USA.","DOI":"10.1609\/aimag.v39i4.2826"},{"key":"ref_38","unstructured":"Ross, S., Gordon, G.J., and Bagnell, J.A. (2011;, January 11\u201313). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_39","unstructured":"Sharma, M., Holmes, M., Santamaria, J., Irani, A., Isbell, C., and Ram, A. (2007, January 6\u201312). Transfer Learning in Real-Time Strategy Games Using Hybrid CBR\/RL. Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India."},{"key":"ref_40","first-page":"1633","article-title":"Transfer Learning for Reinforcement Learning Domains: A Survey","volume":"10","author":"Taylor","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ferreira, L., and Toledo, C. (2014, January 26\u201329). A Search-based Approach for Generating Angry Birds Levels. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932912"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1109\/TCIAIG.2011.2149523","article-title":"Generating Missions and Spaces for Adaptable Play Experiences","volume":"3","author":"Dormans","year":"2011","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.robot.2004.03.012","article-title":"Teaching Bayesian Behaviours to Video Game Characters","volume":"47","author":"Hy","year":"2004","journal-title":"Rob. Auton. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.knosys.2018.04.030","article-title":"Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone","volume":"153","author":"Tonda","year":"2018","journal-title":"Knowl. -Based Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Liapis, A., Holmg\u00e5rd, C., Yannakakis, G.N., and Togelius, J. (2015, January 8\u201310). Procedural Personas as Critics for Dungeon Generation. Proceedings of the 18th European Conference on the Applications of Evolutionary Computation, Copenhagen, Denmark.","DOI":"10.1007\/978-3-319-16549-3_27"},{"key":"ref_46","first-page":"75","article-title":"Automatically Generating Game Tactics through Evolutionary Learning","volume":"27","author":"Ponsen","year":"2006","journal-title":"AI Mag."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TCIAIG.2016.2642158","article-title":"Creating AI Characters for Fighting Games using Genetic Programming","volume":"9","author":"Cant","year":"2017","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_48","unstructured":"Maissiat, J., and Meneguzzi, F. (2013, January 16\u201318). Adaptive High-Level Strategy Learning in StarCraft. Proceedings of the 12th Brazilian Symposium on Games and Digital Entertainment, S\u00e3o Paulo, Brazil."},{"key":"ref_49","unstructured":"Denzinger, J., and Winder, C. (2005, January 4\u20136). Combining coaching and learning to create cooperative character behavior. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Colchester, UK."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Sharifi, A.A., Zhao, R., and Szafron, D. (2010, January 11\u201313). Learning Companion Behaviors Using Reinforcement Learning in Games. Proceedings of the 6th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Palo Alto, CA, USA.","DOI":"10.1609\/aiide.v6i1.12392"},{"key":"ref_51","unstructured":"Raschka, S. (2015). Python Machine Learning, Packt Publishing Ltd.. [1st ed.]."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/S0004-3702(01)00093-5","article-title":"Improving heuristic mini-max search by supervised learning","volume":"134","author":"Buro","year":"2002","journal-title":"Artif. Intell."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Mendes, A., Togelius, J., and Nealen, A. (2016, January 20\u201323). Hyper-heuristic General Video Game Playing. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Santorini, Greece.","DOI":"10.1109\/CIG.2016.7860398"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Laskey, M., Staszak, S., Hsieh, W.Y.S., Mahler, J., Pokorny, F.T., Dragan, A.D., and Goldberg, K. (2016, January 16\u201321). SHIV: Reducing Supervisor Burden in DAgger using Support Vectors for Efficient Learning from Demonstrations in High Dimensional State Spaces. Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487167"},{"key":"ref_55","unstructured":"Young, J., and Hawes, N. (2014, January 3\u20137). Learning Micro-Management Skills in RTS Games by Imitating Experts. Proceedings of the 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Raleigh, NC, USA."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Park, H., Cho, H.C., Lee, K.Y., and Kim, K.J. (2012, January 26\u201327). Prediction of Early Stage Opponents Strategy for StarCraft AI using Scouting and Machine Learning. Proceedings of the WASA 2012: Workshop at SIGGRAPH Asia, Singapore.","DOI":"10.1145\/2425296.2425298"},{"key":"ref_57","unstructured":"Asayama, K., Moriyama, K., Fukui, K.I., and Numao, M. (September, January 31). Prediction as Faster Perception in a Real-Time Fighting Video Game. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_58","unstructured":"Weber, B.G., John, M., Mateas, M., and Jhala, A. (, January 9\u201311August). Modeling player retention in Madden NFL 11. Proceedings of the 23rd Innovative Applications of Artificial Intelligence Conference, San Francisco, CA, USA."},{"key":"ref_59","unstructured":"Weber, B.G., Mateas, M., and Jhala, A. (July, January 28). Using Data Mining to Model Player Experience. Proceedings of the 2nd International Workshop on Evaluating Player Experience in Games, Bordeaux, France."},{"key":"ref_60","unstructured":"Zocca, V., Spacagna, G., Slater, D., and Roelants, P. (2017). Python Deep Learning, Packt Publishing Ltd."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Tesauro, G. (1990, January 17\u201321). Neurogammon: A Neural-Network Backgammon Program. Proceedings of the International Joint Conference on Neural Networks, San Diego, CA, USA.","DOI":"10.1109\/IJCNN.1990.137821"},{"key":"ref_62","unstructured":"Tesauro, G. (1989). Connectionist Learning of Expert Preferences by Comparison Training. Advances in Neural Information Processing Systems 1 (NIPS 1988), Morgan Kaufmann."},{"key":"ref_63","unstructured":"Griffith, N.J.L., and Lynch, M. (1997, January 10\u201313). NeuroDraughts: The role of representation, search, training regime and architecture in a TD draughts player. Proceedings of the 8th Ireland Conference on Artificial Intelligence, Ulster, UK."},{"key":"ref_64","unstructured":"Thrun, S. (1995). Learning To Play the Game of Chess. Advances in Neural Information Processing Systems 7 (NIPS 1994), The MIT Press."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Gemine, Q., Safadi, F., Fonteneau, R., and Ernst, D. (2012, January 11\u201314). Imitative Learning for Real-Time Strategy Games. Proceedings of the IEEE Conference on Computational Intelligence and Games, Granada, Spain.","DOI":"10.1109\/CIG.2012.6374186"},{"key":"ref_66","first-page":"2863","article-title":"Action-Conditional Video Prediction using Deep Networks in Atari Games","volume":"Volume 2","author":"Cortes","year":"2015","journal-title":"Advances in Neural Information Processing Systems 28 (NIPS 2015)"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Luo, Z., Guzdial, M., Liao, N., and Riedl, M. (2018, January 13\u201317). Player Experience Extraction from Gameplay Video. Proceedings of the 14th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Edmonton, Canada.","DOI":"10.1609\/aiide.v14i1.13024"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Gaspareto, O.B., Barone, D.A.C., and Schneider, A.M. (2008, January 18\u201320). Neural Networks Applied to Speed Cheating Detection in Online Computer Games. Proceedings of the 4th International Conference on Natural Computation, Jinan, China.","DOI":"10.1109\/ICNC.2008.720"},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Mori, K., Ito, S., Harada, T., Thawonmas, R., and Kim, K.J. (2017, January 11\u201312). Feature Extraction of Gameplays for Similarity Calculation in Gameplay Recommendation. Proceedings of the IEEE 10th International Workshop on Computational Intelligence and Applications, Hiroshima, Japan.","DOI":"10.1109\/IWCIA.2017.8203580"},{"key":"ref_70","first-page":"223","article-title":"Reinforcement Learning with Attention that Works: A Self-Supervised Approach","volume":"Volume 1143","author":"Gedeon","year":"2019","journal-title":"Neural Information Processing. ICONIP 2019. Communications in Computer and Information Science"},{"key":"ref_71","unstructured":"Racani\u00e8re, S., Weber, T., Reichert, D.P., Buesing, L., Guez, A., Rezende, D., Badia, A.P., Vinyals, O., Heess, N., and Li, Y. (2017). Imagination-Augmented Agents for Deep Reinforcement Learning. Advances in Neural Information Processing Systems 30 (NIPS 2017), Curran Associates Inc."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Pathak, D., Agrawal, P., Efros, A.A., and Darrell, T. (2017, January 6\u201311). Curiosity-driven Exploration by Self-supervised Prediction. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.","DOI":"10.1109\/CVPRW.2017.70"},{"key":"ref_73","unstructured":"Ha, D., and Schmidhuber, J. (2018). World Models. Zenodo."},{"key":"ref_74","unstructured":"Peng, P., Wen, Y., Yang, Y., Yuan, Q., Tang, Z., Long, H., and Wang, J. (2017). Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games. arXiv."},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Eiben, A.E., and Smith, J.E. (2015). Introduction to Evolutionary Computing, Springer. [2nd ed.].","DOI":"10.1007\/978-3-662-44874-8"},{"key":"ref_76","unstructured":"Yen, G.Y., and Fogel, D.B. (2006). Computational Intelligence in Games. Computational Intelligence: Principles and Practice, IEEE."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/TCIAIG.2015.2494596","article-title":"Neuroevolution in Games: State of the Art and Open Challenges","volume":"9","author":"Risi","year":"2017","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1023\/A:1008224732364","article-title":"Evolving Neural Networks to Play Go","volume":"8","author":"Richards","year":"1998","journal-title":"Appl. Intell."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"1947","DOI":"10.1109\/JPROC.2004.837633","article-title":"A Self-Learning Evolutionary Chess Program","volume":"92","author":"Fogel","year":"2004","journal-title":"Proc. IEEE"},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/S0925-2312(01)00594-X","article-title":"Verifying Anaconda\u2019s expert rating by competing against Chinook: Experiments in co-evolving a neural checkers player","volume":"42","author":"Fogel","year":"2002","journal-title":"Neurocomputing"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1007\/s10489-014-0536-y","article-title":"LS-VisionDraughts: Improving the performance of an agent for checkers by integrating computational intelligence, reinforcement learning and a powerful search method","volume":"41","author":"Neto","year":"2014","journal-title":"Appl. Intell."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.entcom.2012.10.001","article-title":"Imitating human playing styles in Super Mario Bros","volume":"4","author":"Ortega","year":"2013","journal-title":"Entertain. Comput."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1007\/s10489-013-0470-4","article-title":"An evolutionary-based hyper-heuristic approach for the Jawbreaker puzzle","volume":"40","author":"Cuadra","year":"2014","journal-title":"Appl. Intell."},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Pedersen, C., Togelius, J., and Yannakakis, G.N. (2009, January 7\u201310). Modeling Player Experience in Super Mario Bros. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Milano, Italy.","DOI":"10.1109\/CIG.2009.5286482"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Togelius, J., and Schmidhuber, J. (2008, January 15\u201318). An Experiment in Automatic Game Design. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Perth, Australia.","DOI":"10.1109\/CIG.2008.5035629"},{"key":"ref_86","unstructured":"Hom, V., and Marks, J. (2007, January 6\u20138). Automatic Design of Balanced Board Games. Proceedings of the 3rd Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, California."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Mourato, F., Dos Santos, M.P., and Birra, F. (2011, January 8\u201311). Automatic level generation for platform videogames using Genetic Algorithms. Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology, Lisbon, Portugal.","DOI":"10.1145\/2071423.2071433"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Hastings, E., Guha, R., and Stanley, K.O. (2007, January 1\u20135). NEAT Particles: Design, Representation, and Animation of Particle System Effects. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Honolulu, HI, USA.","DOI":"10.1109\/CIG.2007.368092"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1109\/TCIAIG.2009.2038365","article-title":"Automatic Content Generation in the Galactic Arms Race Video Game","volume":"1","author":"Hastings","year":"2009","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Togelius, J., De Nardi, R., and Lucas, S.M. (2007, January 1\u20135). Towards automatic personalised content creation for racing games. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Honolulu, HI, USA.","DOI":"10.1109\/CIG.2007.368106"},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1162\/106454600568861","article-title":"Talking helps: Evolving communicating agents for the predator-prey pursuit problem","volume":"6","author":"Jim","year":"2000","journal-title":"Artif. Life"},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Bishop, J., and Miikkulainen, R. (2013, January 11\u201313). Evolutionary Feature Evaluation for Online Reinforcement Learning. Proceedings of the Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633648"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Davison, T., and Denzinger, J. (2012, January 11\u201314). The Huddle: Combining AI Techniques to Coordinate a Player\u2019s Game Characters. Proceedings of the IEEE Conference on Computational Intelligence and Games, Granada, Spain.","DOI":"10.1109\/CIG.2012.6374157"},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1109\/TEVC.2005.856210","article-title":"Real-Time Neuroevolution in the NERO Video Game","volume":"9","author":"Stanley","year":"2005","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_95","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.]."},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-Learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn"},{"key":"ref_97","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards, King\u2019s College."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/BF00992697","article-title":"Practical Issues in Temporal Difference Learning","volume":"8","author":"Tesauro","year":"1992","journal-title":"Mach. Learn"},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1147\/rd.33.0210","article-title":"Some Studies in Machine Learning Using the Game of Checkers","volume":"3","author":"Samuel","year":"1959","journal-title":"IBM J. Res. Dev."},{"key":"ref_100","unstructured":"Huang, H.H., and Wang, T. (September, January 31). Learning Overtaking and Blocking Skills in Simulated Car Racing. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_101","unstructured":"Abbeel, P., and Ng, A.Y. Apprenticeship Learning via Inverse Reinforcement Learning. Proceedings of the 21st International Conference on Machine Learning, Banff, AB, Canada. Available online: https:\/\/ai.stanford.edu\/~ang\/papers\/icml04-apprentice.pdf."},{"key":"ref_102","doi-asserted-by":"crossref","unstructured":"Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., and Osband, I. (2018, January 2\u20137). Deep Q-learning from Demonstrations. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11757"},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Tastan, B., and Sukthankar, G. (2011, January 10\u201314). Learning Policies for First Person Shooter Games Using Inverse Reinforcement Learning. Proceedings of the 7th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v7i1.12430"},{"key":"ref_104","unstructured":"Knox, W.B., and Stone, P. (2012, January 4\u20138). Reinforcement Learning from Simultaneous Human and MDP Reward. Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain. International Foundation for Autonomous Agents and Multiagent Systems."},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Knox, W.B., and Stone, P. (2009, January 1\u20134). Interactively Shaping Agents via Human Reinforcement The TAMER Framework. Proceedings of the 5th International Conference on Knowledge Capture, Redondo Beach, CA, USA.","DOI":"10.1145\/1597735.1597738"},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"Shaker, N. (2016, January 20\u201323). Intrinsically Motivated Reinforcement Learning: A Promising Framework for Procedural Content Generation. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Santorini, Greece.","DOI":"10.1109\/CIG.2016.7860450"},{"key":"ref_107","unstructured":"Usunier, N., Synnaeve, G., Lin, Z., and Chintala, S. (2017, January 24\u201326). Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."},{"key":"ref_108","doi-asserted-by":"crossref","unstructured":"Wender, S., and Watson, I. (2012, January 11\u201314). Applying Reinforcement Learning to Small Scale Combat in the Real-Time Strategy Game StarCraft:Broodwar. Proceedings of the Conference on Computational Intelligence and Games, Granada, Spain.","DOI":"10.1109\/CIG.2012.6374183"},{"key":"ref_109","unstructured":"Bellemare, M.G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R. (2016, January 5\u201310). Unifying Count-Based Exploration and Intrinsic Motivation. Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), Barcelona, Spain."},{"key":"ref_110","unstructured":"Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. (May, January 30). Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, Canada."},{"key":"ref_111","unstructured":"Kulkarni, T.D., Narasimhan, K.R., Saeedi, A., and Tenenbaum, J.B. (2016, January 5\u201310). Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation. Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), Barcelona, Spain."},{"key":"ref_112","unstructured":"Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., and Efros, A.A. (2019, January 6\u20139). Large-Scale Study of Curiosity-Driven Learning. Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, LA, USA."},{"key":"ref_113","unstructured":"Wu, Y., and Tian, Y. (2017, January 24\u201326). Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."},{"key":"ref_114","unstructured":"Hamrick, J.B., Ballard, A.J., Pascanu, R., Vinyals, O., Heess, N., and Battaglia, P.W. (2017, January 24\u201326). Metacontrol for Adaptive imagination-Based Optimization. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."},{"key":"ref_115","unstructured":"Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (, January 4\u20139). Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_116","unstructured":"Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., and Mordatch, I. (May, January 26). Emergent Tool Use From Multi-Agent Autocurricula. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_117","unstructured":"Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., and Mordatch, I. (May, January 30). Emergent Complexity via Multi-Agent Competition. Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_118","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1126\/science.aau6249","article-title":"Human-level performance in 3D multiplayer games with population-based reinforcement learning","volume":"364","author":"Jaderberg","year":"2019","journal-title":"Science"},{"key":"ref_119","unstructured":"Koller, D., and Friedman, N. (2009). Probabilistic Graphical Models Principles and Techniques, The MIT Press."},{"key":"ref_120","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1008238218679","article-title":"Bayesian Models for Keyhole Plan Recognition in an Adventure Game","volume":"8","author":"Albrecht","year":"1998","journal-title":"User Model. User-adapt. Interact."},{"key":"ref_121","unstructured":"Hostetler, J., Dereszynski, E., Dietterich, T., and Fern, A. (2012, January 14\u201318). Inferring Strategies from Limited Reconnaissance in Real-time Strategy Games. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, Catalina Island, CA, USA."},{"key":"ref_122","doi-asserted-by":"crossref","unstructured":"Synnaeve, G., and Bessi\u00e8re, P. (2011, January 10\u201314). A Bayesian Model for Plan Recognition in RTS Games Applied to StarCraft. Proceedings of the 7th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v7i1.12429"},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Leece, M., and Jhala, A. (2014, January 26\u201329). Opponent State Modeling In RTS games with limited information using Markov Random Fields. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932877"},{"key":"ref_124","unstructured":"Parra, R., and Garrido, L. (November, January 27). Bayesian Networks for Micromanagement Decision Imitation in the RTS Game Starcraft. Proceedings of the 11th Mexican international conference on Advances in Computational Intelligence, San Luis Potos\u00ed, Mexico."},{"key":"ref_125","unstructured":"Synnaeve, G., and Bessi\u00e8re, P. (September, January 31). A Bayesian Model for RTS Units Control applied to StarCraft. Proceedings of the IEEE Conference on Computational Intelligence and Games, Seoul, Korea."},{"key":"ref_126","first-page":"55","article-title":"Applications of Machine Learning and Rule Induction. Commun","volume":"38","author":"Langley","year":"1995","journal-title":"ACM"},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"39","DOI":"10.3233\/AIC-1994-7104","article-title":"Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches","volume":"7","author":"Agnar","year":"1994","journal-title":"AI Commun."},{"key":"ref_128","doi-asserted-by":"crossref","unstructured":"Aha, D.W., Molineaux, M., and Ponsen, M. (2005, January 23\u201326). Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game. Proceedings of the International Conference on Case-Based Reasoning, Chicago, IL, USA.","DOI":"10.1007\/11536406_4"},{"key":"ref_129","doi-asserted-by":"crossref","unstructured":"Weber, B.G., and Mateas, M. (2009, January 14\u201316). Case-Based Reasoning for Build Order in Real-Time Strategy Games. Proceedings of the 5th Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, CA, USA.","DOI":"10.1609\/aiide.v5i1.12360"},{"key":"ref_130","unstructured":"Szczepanski, T., and Aamodt, A. (2009, January 20\u201323). Case-based Reasoning for Improved Micromanagement in Real-Time Strategy Games. Proceedings of the Workshop on Case-Based Reasoning for Computer Games, 8th International Conference on Case-Based Reasoning, Seattle, WA, USA."},{"key":"ref_131","doi-asserted-by":"crossref","unstructured":"Hsieh, J.L., and Sun, C.T. (2008, January 1\u20138). Building a Player Strategy Model by Analyzing Replays of Real-Time Strategy Games. Proceedings of the IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China.","DOI":"10.1109\/IJCNN.2008.4634237"},{"key":"ref_132","first-page":"45","article-title":"Online Adaptation of Game Opponent AI with Dynamic Scripting","volume":"3","author":"Spronck","year":"2004","journal-title":"Int. J. Intell. Games Simul."},{"key":"ref_133","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10994-006-6205-6","article-title":"Adaptive game AI with dynamic scripting","volume":"63","author":"Spronck","year":"2006","journal-title":"Mach. Learn"},{"key":"ref_134","unstructured":"Ponsen, M., and Spronck, P. (2004, January 8\u201310). Improving Adaptive Game AI with Evolutionary Learning. Proceedings of the 5th Game-On International Conference on Computer Games: Artificial Intelligence, Design and Education, Wolverhampton, UK."},{"key":"ref_135","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1613\/jair.3912","article-title":"The Arcade Learning Environment: An Evaluation Platform for General Agents","volume":"47","author":"Bellemare","year":"2013","journal-title":"J. Artif. Intell. Res."},{"key":"ref_136","doi-asserted-by":"crossref","unstructured":"Kempka, M., Wydmuch, M., Runc, G., Toczek, J., and Jaskowski, W. (2016, January 20\u201323). ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Santorini, Greece.","DOI":"10.1109\/CIG.2016.7860433"},{"key":"ref_137","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., and Tassa, Y. (2012, January 7\u201312). MuJoCo: A physics engine for model-based control. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref_138","unstructured":"Juliani, A., Berges, V.-P., Vckay, E., Gao, Y., Henry, H., Mattar, M., and Lange, D. (2018). Unity: A General Platform for Intelligent Agents. arXiv."},{"key":"ref_139","unstructured":"Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv."},{"key":"ref_140","unstructured":"Yin, H., Luo, L., Cai, W., Ong, Y.S., and Zhong, J. (September, January 31). A Data-driven Approach for Online Adaptation of Game Difficult. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_141","doi-asserted-by":"crossref","unstructured":"Rabin, S. (2015). Game AI Pro 2: Collected Wisdom of Game AI Professionals, CRC Press.","DOI":"10.1201\/b18373"},{"key":"ref_142","doi-asserted-by":"crossref","unstructured":"Rabin, S. (2014). Game AI Pro: Collected Wisdom of Game AI Professionals, CRC Press.","DOI":"10.1201\/b16725"},{"key":"ref_143","unstructured":"Rabin, S. (2002). AI Game Programming Wisdom, Charles River Media."},{"key":"ref_144","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1177\/105971239300100403","article-title":"E?cient Learning and Planning Within the Dyna Framework","volume":"1","author":"Peng","year":"1993","journal-title":"Adapt. Behav."},{"key":"ref_145","doi-asserted-by":"crossref","unstructured":"Wang, Y., and Gelly, S. (2007, January 1\u20135). Modifications of UCT and sequence-like simulations for Monte-Carlo Go. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Honolulu, HI, USA.","DOI":"10.1109\/CIG.2007.368095"},{"key":"ref_146","unstructured":"Coulom, R. (2006, January 29\u201331). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. Proceedings of the 5th International Conference on Computer and Games, Turin, Italy."},{"key":"ref_147","doi-asserted-by":"crossref","unstructured":"Churchill, D., Saffidine, A., and Buro, M. (2012, January 8\u201312). Fast Heuristic Search for RTS Game Combat Scenarios. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v8i1.12527"},{"key":"ref_148","doi-asserted-by":"crossref","unstructured":"Silver, D. (2005, January 1\u20133). Cooperative Pathfinding. Proceedings of the 1st AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Marina del Rey, CA, USA.","DOI":"10.1609\/aiide.v1i1.18726"},{"key":"ref_149","doi-asserted-by":"crossref","unstructured":"Harabor, D., and Botea, A. (2008, January 15\u201318). Hierarchical Path Planning for Multi-Size Agents in Heterogeneous Environments. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Perth, Australia.","DOI":"10.1109\/CIG.2008.5035648"},{"key":"ref_150","doi-asserted-by":"crossref","unstructured":"Maggiore, G., Santos, C., Dini, D., Peters, F., Bouwknegt, H., and Spronck, P. (2013, January 11\u201313). LGOAP: Adaptive layered planning for real-time videogames. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633624"},{"key":"ref_151","unstructured":"Drake, J., Safonova, A., and Likhachev, M. (2017, January 5\u20139). Towards Adaptability of Demonstration-Based Training of NPC Behavior. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Snowbird, UT, USA."},{"key":"ref_152","doi-asserted-by":"crossref","unstructured":"Tremblay, J., Borodovski, A., and Verbrugge, C. (2014, January 3\u20137). I Can Jump! Exploring Search Algorithms for Simulating Platformer Players. Proceedings of the 10th Artificial Intelligence and Interactive Digital Entertainment Conference, Raleigh, NC, USA.","DOI":"10.1609\/aiide.v10i3.12744"},{"key":"ref_153","doi-asserted-by":"crossref","unstructured":"Togelius, J., Karakovskiy, S., and Baumgarten, R. (2010, January 18\u201323). The 2009 Mario AI Competition. Proceedings of the IEEE Congress on Evolutionary Computation, Barcelona, Spain.","DOI":"10.1109\/CEC.2010.5586133"},{"key":"ref_154","doi-asserted-by":"crossref","unstructured":"Ghallab, M., Nau, D., and Traverso, P. (2004). Automated Planning Theory and Practice, Morgan Kaufmann Publishers.","DOI":"10.1016\/B978-155860856-6\/50021-1"},{"key":"ref_155","first-page":"119","article-title":"SHPE: HTN Planning for Video Games","volume":"Volume 504","author":"Cazenave","year":"2014","journal-title":"Computer Games. CGW 2014. Communications in Computer and Information Science"},{"key":"ref_156","doi-asserted-by":"crossref","unstructured":"Nguyen, C., Reifsnyder, N., Gopalakrishnan, S., and Munoz-Avila, H. (2017, January 22\u201325). Automated Learning of Hierarchical Task Networks for Controlling Minecraft Agents. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080440"},{"key":"ref_157","doi-asserted-by":"crossref","unstructured":"Hoang, H., Lee-Urban, S., and Mu\u00f1oz-Avila, H. (2005, January 1\u20133). Hierarchical Plan Representations for Encoding Strategic Game AI. Proceedings of the 1st Artificial Intelligence and Interactive Digital Entertainment Conference, Marina del Rey, CA, USA.","DOI":"10.1609\/aiide.v1i1.18717"},{"key":"ref_158","unstructured":"Gorniak, P., and Davis, I. (2007, January 6\u20138). SquadSmart Hierarchical Planning and Coordinated Plan Execution for Squads of Characters. Proceedings of the 3rd Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, CA, USA."},{"key":"ref_159","doi-asserted-by":"crossref","unstructured":"Neufeld, X., Mostaghim, S., and Perez-Liebana, D. (2017, January 27\u201329). HTN Fighter: Planning in a Highly-Dynamic Game. Proceedings of the 9th Conference on Computer Science and Electronic Engineering, Colchester, UK.","DOI":"10.1109\/CEEC.2017.8101623"},{"key":"ref_160","unstructured":"Kelly, J.P., Botea, A., and Koenig, S. (2008, January 22\u201324). Offline Planning with Hierarchical Task Networks in Video Games. Proceedings of the 4th Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, CA, USA."},{"key":"ref_161","unstructured":"Guilherme Da Silva, F.A., and Feij\u00f3, B. (September, January 31). Personality Traits in Plots with Nondeterministic Planning for Interactive Storytelling. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_162","unstructured":"Neufeld, X., Mostaghim, S., and Perez-Liebana, D. (2019, January 12). A Hybrid Planning and Execution Approach Through HTN and MCTS. Proceedings of the 3rd Workshop on Integrated Planning, Acting, and Execution, Berkeley, CA, USA."},{"key":"ref_163","first-page":"282","article-title":"Bandit based Monte-Carlo Planning","volume":"Volume 4212","author":"Kocsis","year":"2006","journal-title":"Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science"},{"key":"ref_164","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A Survey of Monte Carlo Tree Search Methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_165","unstructured":"Dann, M., Zambettau, F., and Thangarajah, J. (September, January 31). An Improved Approach to Reinforcement Learning in Computer Go. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_166","unstructured":"Barrett, S., Stone, P., and Kraus, S. (2011, January 2\u20136). Empirical Evaluation of Ad Hoc Teamwork in the Pursuit Domain. Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems, Taipei, Taiwan."},{"key":"ref_167","doi-asserted-by":"crossref","unstructured":"Williams, P.R., Walton-Rivers, J., Perez-Liebana, D., and Lucas, S.M. (2015, January 24\u201325). Monte Carlo Tree Search Applied to Co-operative Problems. Proceedings of the 7th Computer Science and Electronic Engineering Conference, Colchester, UK.","DOI":"10.1109\/CEEC.2015.7332728"},{"key":"ref_168","doi-asserted-by":"crossref","unstructured":"Demediuk, S., Tamassia, M., Raffe, W.L., Zambetta, F., Li, X., and Mueller, F. (2017, January 22\u201325). Monte Carlo Tree Search Based Algorithms for Dynamic Difficulty Adjustment. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080415"},{"key":"ref_169","doi-asserted-by":"crossref","unstructured":"Devlin, S., Anspoka, A., Sephton, N., Cowling, P.I., and Rollason, J. (2016, January 8\u201312). Combining Gameplay Data With Monte Carlo Tree Search To Emulate Human Play. Proceedings of the 12th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Burlingame, CA, USA.","DOI":"10.1609\/aiide.v12i1.12858"},{"key":"ref_170","doi-asserted-by":"crossref","unstructured":"Whitehouse, D., Cowling, P.I., Powley, E.J., and Rollason, J. (2013, January 14\u201315). Integrating Monte Carlo Tree Search with Knowledge-Based Methods to Create Engaging Play in a Commercial Mobile Game. Proceedings of the 9th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Boston, MA, USA.","DOI":"10.1609\/aiide.v9i1.12679"},{"key":"ref_171","doi-asserted-by":"crossref","unstructured":"Bowen, N., Todd, J., and Sukthankar, G. (2013, January 11\u201313). Adjutant Bot: An Evaluation of Unit Micromanagement Tactics. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633664"},{"key":"ref_172","doi-asserted-by":"crossref","unstructured":"Justesen, N., Tillman, B., Togelius, J., and Risi, S. (2014, January 26\u201329). Script- and Cluster-based UCT for StarCraft. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932900"},{"key":"ref_173","unstructured":"Balla, R., and Fern, A. (2009, January 11\u201317). UCT for Tactical Assault Planning in Real-Time Strategy Games. Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, CA, USA."},{"key":"ref_174","doi-asserted-by":"crossref","unstructured":"Ishii, R., Ito, S., Ishihara, M., Harada, T., and Thawonmas, R. (2018, January 14\u201317). Monte-Carlo Tree Search Implementation of Fighting Game AIs Having Personas. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Maastricht, Netherlands.","DOI":"10.1109\/CIG.2018.8490367"},{"key":"ref_175","doi-asserted-by":"crossref","unstructured":"Sarratt, T., Pynadath, D.V., and Jhala, A. (2014, January 26\u201329). Converging to a Player Model In Monte-Carlo Tree Search. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932881"},{"key":"ref_176","doi-asserted-by":"crossref","unstructured":"Sanselone, M., Sanchez, S., Sanza, C., Panzoli, D., and Duthen, Y. (2014, January 26\u201329). Constrained control of non-playing characters using Monte Carlo Tree Search. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932886"},{"key":"ref_177","first-page":"358","article-title":"Self-Adaptive MCTS for General Video Game Playing","volume":"Volume 10784","author":"Sim","year":"2018","journal-title":"Applications of Evolutionary Computation. EvoApplications 2018. Lecture Notes in Computer Science"},{"key":"ref_178","doi-asserted-by":"crossref","unstructured":"Sironi, C.F., and Winands, M.H.M. (2018, January 14\u201317). Analysis of Self-Adaptive Monte Carlo Tree Search in General Video Game Playing. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Maastricht, The Netherlands.","DOI":"10.1109\/CIG.2018.8490402"},{"key":"ref_179","doi-asserted-by":"crossref","unstructured":"Sironi, C.F., Liu, J., and Winands, M.H.M. (2018). Self-Adaptive Monte-Carlo Tree Search in General Game Playing. IEEE Trans. Games (Early Access).","DOI":"10.1109\/CIG.2018.8490402"},{"key":"ref_180","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0004-3702(71)90010-5","article-title":"STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving","volume":"2","author":"Fikes","year":"1971","journal-title":"Artif. Intell."},{"key":"ref_181","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1093\/logcom\/4.5.467","article-title":"ADL and the State-Transition Model of Action","volume":"4","author":"Pednault","year":"1994","journal-title":"J. Log. Comput."},{"key":"ref_182","first-page":"14","article-title":"PDDL\u2014The Planning Domain Definition Language","volume":"3","author":"Ghallab","year":"1998","journal-title":"AIPS Plan. Comm."},{"key":"ref_183","unstructured":"Orkin, J. (2006, January 20\u201324). Three States and a Plan: The A.I. of F.E.A.R. Proceedings of the Game Developers Conference\u2014GDC 2006, San Jose, CA, USA."},{"key":"ref_184","unstructured":"Kovarsky, A., and Buro, M. (December, January 29). A First Look at Build-Order Optimization in Real-Time Strategy Games. Proceedings of the 7th International Conference on Intelligent Games and Simulation, Braunschweig, Germany."},{"key":"ref_185","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MIS.2002.1024751","article-title":"A Behavior Language for Story-Based Believable Agents","volume":"17","author":"Mateas","year":"2002","journal-title":"IEEE Intell. Syst."},{"key":"ref_186","doi-asserted-by":"crossref","unstructured":"Weber, B.G., Mawhorter, P., Mateas, M., and Jhala, A. (2010, January 18\u201321). Reactive Planning Idioms for Multi-Scale Game AI. Proceedings of the IEEE Conference on Computational Intelligence and Games, Dublin, Ireland.","DOI":"10.1109\/ITW.2010.5593363"},{"key":"ref_187","doi-asserted-by":"crossref","unstructured":"Branquinho, A.A.B., and Lopes, C.R. (2010, January 10\u201313). Planning for Resource Production in Real-Time Strategy Games Based on Partial Order Planning, Search and Learning. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey.","DOI":"10.1109\/ICSMC.2010.5642498"},{"key":"ref_188","doi-asserted-by":"crossref","unstructured":"Shirvani, A., Farrell, R., and Ware, S.G. (2018, January 13\u201317). Combining Intentionality and Belief: Revisiting Believable Character Plans. Proceedings of the 14th Artificial Intelligence and Interactive Digital Entertainment Conference, Edmonton, Canada.","DOI":"10.1609\/aiide.v14i1.13037"},{"key":"ref_189","doi-asserted-by":"crossref","unstructured":"Van Lent, M., Riedl, M.O., Carpenter, P., Mcalinden, R., and Brobst, P. (2005, January 1\u20133). Increasing Replayability with Deliberative and Reactive Planning. Proceedings of the 1st Artificial Intelligence and Interactive Digital Entertainment Conference, Marina del Rey, CA, USA.","DOI":"10.1609\/aiide.v1i1.18729"},{"key":"ref_190","doi-asserted-by":"crossref","unstructured":"Weber, B.G., Mateas, M., and Jhala, A. (2010, January 11\u201313). Applying Goal-Driven Autonomy to StarCraft. Proceedings of the 6th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Palo Alto, CA, USA.","DOI":"10.1609\/aiide.v6i1.12401"},{"key":"ref_191","doi-asserted-by":"crossref","unstructured":"Labranche, S., Sola, N., Callies, S., and Beaudry, E. (2014, January 26\u201329). Using Partial Satisfaction Planning to Automatically Select NPCs\u2019 Goals and Generate Plans In a Simulation Game. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932890"},{"key":"ref_192","doi-asserted-by":"crossref","unstructured":"Abelha, P., Gottin, V., Ciarlini, A., Araujo, E., Furtado, A., Feijo, B., Silva, F., and Pozzer, C. (2013, January 14\u201318). A Nondeterministic Temporal Planning Model for Generating Narratives with Continuous Change in Interactive Storytelling. Proceedings of the 9th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Boston, MA, USA.","DOI":"10.1609\/aiide.v9i1.12676"},{"key":"ref_193","unstructured":"Shirvani, A., Ware, S.G., and Farrell, R. (2017, January 5\u20139). A Possible Worlds Model of Belief for State-Space Narrative Planning. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Little Cottonwood Canyon, UT, USA."},{"key":"ref_194","unstructured":"Ware, S.G., and Young, R.M. (2014, January 3\u20137). Glaive: A State-Space Narrative Planner Supporting Intentionality and Conflict. Proceedings of the 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Raleigh, NC, USA."},{"key":"ref_195","doi-asserted-by":"crossref","unstructured":"Li, B., and Riedl, M.O. (2010, January 11\u201313). An Offline Planning Approach to Game Plotline Adaptation. Proceedings of the 6th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Palo Alto, CA, USA.","DOI":"10.1609\/aiide.v6i1.12394"},{"key":"ref_196","unstructured":"LaValle, S.M. (2020, May 29). Available online: https:\/\/citeseerx.ist.psu.edu\/viewdoc\/download;jsessionid=BDCAC5E777FC7B257783418EE8F274C4?doi=10.1.1.35.1853&rep=rep1&type=pdf."},{"key":"ref_197","doi-asserted-by":"crossref","unstructured":"Shepherd, J.J., Tang, J., and O\u2019Kane, J. (2009, January 11\u201312). Path-Finding with Motion Constraints in Real Time Strategies. Proceedings of the Annual International Conferences on Computer Games, Multimedia and Allied Technology, Amara, Singapore.","DOI":"10.5176\/978-981-08-3190-5_415"},{"key":"ref_198","unstructured":"Xu, S. (2015). Improving Companion Ai in Small-Scale Attrition Games. [Master\u2019s Thesis, McGill University]."},{"key":"ref_199","unstructured":"Soares, R., Leal, F., Prada, R., and Melo, F. (2016, January 1\u20136). Rapidly-Exploring Random Tree approach for Geometry Friends. Proceedings of the 1st International Joint Conference of DiGRA and FDG, Dundee, UK."},{"key":"ref_200","doi-asserted-by":"crossref","unstructured":"Bauer, A., and Popovi\u0107, Z. (2012, January 8\u201312). RRT-Based Game Level Analysis, Visualization, and Visual Refinement. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v8i1.12512"},{"key":"ref_201","unstructured":"Bauer, A., Cooper, S., and Popovi\u0107, Z. (2013, January 14\u201317). Automated Redesign of Local Playspace Properties. Proceedings of the 8th International Conference on the Foundations of Digital Games, Chania, Greece."},{"key":"ref_202","unstructured":"Tremblay, J., Torres, P.A., Rikovitch, N., and Verbrugge, C. (2013, January 14\u201318). An Exploration Tool for Predicting Stealthy Behaviour. Proceedings of the 9th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Boston, MA, USA. AAAI Technical Report WS-13-20."},{"key":"ref_203","unstructured":"Osborn, J.C., Lambrigger, B., and Mateas, M. (2017, January 5\u20139). HyPED: Modeling and Analyzing Action Games as Hybrid Systems. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Snowbird, UT, USA."},{"key":"ref_204","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1023\/A:1011081305027","article-title":"A Survey on Case-Based Planning","volume":"16","author":"Spalazzi","year":"2001","journal-title":"Artif. Intell. Rev."},{"key":"ref_205","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1111\/j.1467-8640.2009.00344.x","article-title":"On-Line Case-Based Planning","volume":"26","author":"Mishra","year":"2010","journal-title":"Comput. Intell."},{"key":"ref_206","unstructured":"Coman, A., and Mu\u00f1oz-Avila, H. (2012, January 8\u201312). Plan-Based Character Diversity. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, MA, USA."},{"key":"ref_207","first-page":"164","article-title":"On the Role of Explanation for Hierarchical Case- Based Planning in Real-Time Strategy Games","volume":"Volume 4626","author":"Weber","year":"2007","journal-title":"Case-Based Reasoning Research and Development. ICCBR 2007. Lecture Notes in Computer Science"},{"key":"ref_208","first-page":"164","article-title":"Case-Based Planning and Execution for Real-Time Strategy Games","volume":"Volume 4626","author":"Weber","year":"2007","journal-title":"Case-Based Reasoning Research and Development. ICCBR 2007. Lecture Notes in Computer Science"},{"key":"ref_209","unstructured":"Onta\u00f1\u00f3n, S., Bonnette, K., Mahindrakar, P., G\u00f3mez-Mart\u00edn, M.A., Long, K., Radhakrishnan, J., Shah, R., and Ram, A. (2019, January 12). Learning from Human Demonstrations for Real-Time Case-Based Planning. Proceedings of the IJCAI-09 Workshop on Learning Structural Knowledge From Observations, Pasadena, CA, USA."},{"key":"ref_210","unstructured":"Cheng, D.C., and Thawonmas, R. (2004, January 8\u201310). Case-based plan recognition for real-time strategy games. Proceedings of the 5th Game-On International Conference, Reading, UK."},{"key":"ref_211","doi-asserted-by":"crossref","unstructured":"Colledanchise, M., and \u00d6gren, P. (2018). Behavior Trees in Robotics and AI: An Introduction, CRC Press. [1st ed.].","DOI":"10.1201\/9780429489105"},{"key":"ref_212","doi-asserted-by":"crossref","unstructured":"Ocio, S. (2012, January 8\u201312). Adapting AI Behaviors To Players in Driver San Francisco Hinted-Execution Behavior Trees. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, MA, USA.","DOI":"10.1609\/aiide.v8i1.12499"},{"key":"ref_213","doi-asserted-by":"crossref","unstructured":"Johansson, A., and Dell\u2019Acqua, P. (2012, January 11\u201314). Emotional Behavior Trees. Proceedings of the IEEE Conference on Computational Intelligence and Games, Granada, Spain.","DOI":"10.1109\/CIG.2012.6374177"},{"key":"ref_214","unstructured":"Tremblay, J., and Verbrugge, C. (2013, January 14\u201317). Adaptive Companions in FPS Games. Proceedings of the 8th International Conference on the Foundations of Digital Games, Chania, Greece."},{"key":"ref_215","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Yao, J., Yin, Q., and Zha, Y. (2018). Learning Behavior Trees for Autonomous Agents with Hybrid Constraints Evolution. Appl. Sci., 8.","DOI":"10.3390\/app8071077"},{"key":"ref_216","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1109\/TG.2018.2816806","article-title":"Learning of Behavior Trees for Autonomous Agents","volume":"11","author":"Colledanchise","year":"2019","journal-title":"IEEE Trans. Games"},{"key":"ref_217","doi-asserted-by":"crossref","unstructured":"Dey, R., and Child, C. (2013, January 11\u201313). QL-BT: Enhancing Behaviour Tree Design and Implementation with Q-Learning. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633623"},{"key":"ref_218","unstructured":"Kadlec, R., T\u00f3th, C., \u010cern\u00fd, M., Bart\u00e1k, R., and Brom, C. (2012, January 8\u201312). Planning Is the Game: Action Planning as a Design Tool and Game Mechanism. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA."},{"key":"ref_219","doi-asserted-by":"crossref","unstructured":"Horswill, I.D. (2018, January 13\u201317). Postmortem: MKULTRA, An Experimental AI-Based Game. Proceedings of the 14th Artificial Intelligence and Interactive Digital Entertainment Conference, Edmonton, AB, Canada.","DOI":"10.1609\/aiide.v14i1.13027"},{"key":"ref_220","unstructured":"Robison, E. (2017, January 5\u20139). A Sandbox for Modeling Social AI. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. Workshop on Experimental AI in Games WS-17-19, Snowbird, UT, USA."},{"key":"ref_221","doi-asserted-by":"crossref","unstructured":"Cash, S.P., and Young, R.M. (2009, January 14\u201316). Bowyer: A Planning Tool for Bridging the Gap between Declarative and Procedural Domains. Proceedings of the 5th Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, CA, USA.","DOI":"10.1609\/aiide.v5i1.12357"},{"key":"ref_222","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artint.2013.11.002","article-title":"The actor\u2019s view of automated planning and acting: A position paper","volume":"208","author":"Ghallab","year":"2014","journal-title":"Artif. Intell."},{"key":"ref_223","unstructured":"Alc\u00e1zar, V., Guzm\u00e1n, C., Prior, D., Borrajo, D., Castillo, L., and Onaindia, E. (2010, January 1\u20132). PELEA: Planning, Learning and Execution Architecture. Proceedings of the 28th Workshop of the UK Planning and Scheduling Special Interest Group, Brescia, Italy."},{"key":"ref_224","unstructured":"Guzm\u00e1n, C., Alc\u00e1zar, V., Prior, D., Onaindia, E., Borrajo, D., Fdez-Olivares, J., and Quintero, E. (2012, January 26). PELEA: A Domain-Independent Architecture for Planning, Execution and Learning. Proceedings of the 6th Scheduling and Planning Applications woRKshop, S\u00e3o Paulo, Brazil."},{"key":"ref_225","doi-asserted-by":"crossref","unstructured":"Yang, F., Lyu, D., Liu, B., and Gustafson, S. (2018, January 13\u201319). PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/675"},{"key":"ref_226","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/TG.2017.2782846","article-title":"Building a Planner: A Survey of Planning Systems Used in Commercial Video Games","volume":"11","author":"Neufeld","year":"2019","journal-title":"IEEE Trans. Games"},{"key":"ref_227","doi-asserted-by":"crossref","unstructured":"Grzes, M., and Kudenko, D. (2008, January 6\u20138). Plan-based Reward Shaping for Reinforcement Learning. Proceedings of the 4th IEEE International Conference on Intelligent Systems, Varna, Bulgaria.","DOI":"10.1109\/IS.2008.4670492"},{"key":"ref_228","unstructured":"Guo, X., Singh, S., Lee, H., Lewis, R., and Wang, X. (2014, January 8\u201313). Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning. Proceedings of the Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_229","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013, January 9). Playing Atari with Deep Reinforcement Learning. Proceedings of the NIPS Deep Learning Workshop, Lake Tahoe, CA, USA."},{"key":"ref_230","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal Difference Learning and TD-Gammon. Commun","volume":"38","author":"Tesauro","year":"1995","journal-title":"ACM"},{"key":"ref_231","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/TCIAIG.2013.2286295","article-title":"A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft","volume":"5","author":"Ontanon","year":"2013","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_232","unstructured":"Pascanu, R., Li, Y., Vinyals, O., Heess, N., Buesing, L., Racani\u00e8re, S., Reichert, D., Weber, T., Wierstra, D., and Battaglia, P. (2017). Learning model-based planning from scratch. arXiv."},{"key":"ref_233","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H.D., Hsu, D., Lee, W.S., Leong, T.Y., Kaelbling, L.P., Lozano-Perez, T., and Grant, A.H. (2011, January 10\u201314). CAPIR: Collaborative action planning with intention recognition. Proceedings of the 7th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v7i1.12425"},{"key":"ref_234","first-page":"1391","article-title":"Particle filters","volume":"19","year":"2013","journal-title":"Bernoulli"},{"key":"ref_235","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1049\/ip-f-2.1993.0015","article-title":"Novel approach to nonlinear and linear Bayesian state estimation","volume":"140","author":"Gordon","year":"1993","journal-title":"IEEE Proc. F Radar Signal Process."},{"key":"ref_236","unstructured":"Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., and Culotta, A. (2010). Monte-Carlo Planning in Large POMDPs. Advances in Neural Information Processing Systems 23 (NIPS 2010), Curran Associates, Inc."},{"key":"ref_237","doi-asserted-by":"crossref","unstructured":"Macindoe, O., Kaelbling, L.P., and Lozano-P\u00e9rez, T. (2012, January 8\u201312). POMCoP: Belief Space Planning for Sidekicks in Cooperative Games. Proceedings of the 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v8i1.12510"},{"key":"ref_238","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1126\/science.aay2400","article-title":"Superhuman AI for multiplayer poker","volume":"365","author":"Brown","year":"2019","journal-title":"Science"},{"key":"ref_239","unstructured":"Liebana, D.P., Samothrakis, S., Lucas, S.M., and Rohlfshagen, P. (2013, January 6\u201310). Rolling Horizon Evolution versus Tree Search for Navigation in Single-Player Real-Time Games. Proceedings of the Genetic and Evolutionary Computation Conference, Amsterdam, The Netherlands."},{"key":"ref_240","doi-asserted-by":"crossref","unstructured":"Gaina, R.D., Lucas, S.M., and Perez-Liebana, D. (2017, January 22\u201325). Rolling Horizon Evolution Enhancements in General Video Game Playing. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080420"},{"key":"ref_241","doi-asserted-by":"crossref","unstructured":"Perez-Liebana, D., Stephenson, M., Gaina, R.D., Renz, J., and Lucas, S.M. (2017, January 22\u201325). Introducing Real World Physics and Macro-Actions to General Video Game AI. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080443"},{"key":"ref_242","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1109\/TCIAIG.2017.2738156","article-title":"Playing Multi-Action Adversarial Games: Online Evolutionary Planning versus Tree Search","volume":"10","author":"Justesen","year":"2018","journal-title":"IEEE Trans. Games"},{"key":"ref_243","doi-asserted-by":"crossref","unstructured":"Justesen, N., and Risi, S. (2017, January 15\u201319). Continual Online Evolutionary Planning for In-Game Build Order Adaptation in StarCraft. Proceedings of the Genetic and Evolutionary Computation Conference, Berlin Germany.","DOI":"10.1145\/3071178.3071210"},{"key":"ref_244","doi-asserted-by":"crossref","unstructured":"Quadflieg, J., Preuss, M., Kramer, O., and Rudolph, G. (2010, January 18\u201321). Learning the Track and Planning Ahead in a Car Racing Controller. Proceedings of the IEEE Conference on Computational Intelligence and Games, Dublin, Ireland.","DOI":"10.1109\/ITW.2010.5593327"},{"key":"ref_245","unstructured":"Clark, C., and Fleshner, A. (2017, January 5\u20139). Fast Random Genetic Search for Large-Scale RTS Combat Scenarios. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Snowbird, UT, USA."},{"key":"ref_246","unstructured":"Trusty, A., Onta\u00f1\u00f3n, S., and Ram, A. (2008, January 22\u201324). Stochastic Plan Optimization in Real-Time Strategy Game. Proceedings of the 4th Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, CA, USA."},{"key":"ref_247","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images. Advances in Neural Information Processing Systems 28 (NIPS 2015), Curran Associates, Inc."},{"key":"ref_248","doi-asserted-by":"crossref","unstructured":"Tamar, A., Wu, Y., Thomas, G., Levine, S., and Abbeel, P. (2017, January 19\u201325). Value Iteration Networks. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia.","DOI":"10.24963\/ijcai.2017\/700"},{"key":"ref_249","unstructured":"Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., and Silver, D. (2018, January 10\u201315). Learning to Search with MCTSnets. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_250","unstructured":"Schmidhuber, J. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models. arXiv."},{"key":"ref_251","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/j.tics.2012.08.006","article-title":"Planning as inference","volume":"16","author":"Botvinick","year":"2012","journal-title":"Trends Cogn. Sci."},{"key":"ref_252","doi-asserted-by":"crossref","unstructured":"Toussaint, M., and Storkey, A. (2006, January 25\u201329). Probabilistic Inference for Solving Discrete and Continuous State Markov Decision Processes. Proceedings of the 23rd international conference on Machine learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143963"},{"key":"ref_253","doi-asserted-by":"crossref","unstructured":"Baier, H., and Cowling, P.I. (2018, January 14\u201317). Evolutionary MCTS for Multi-Action Adversarial Games. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Maastricht, The Netherlands.","DOI":"10.1109\/CIG.2018.8490403"},{"key":"ref_254","unstructured":"Guo, X., Singh, S., Lewis, R., and Lee, H. (2016, January 9\u201315). Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games. Proceedings of the International Joint Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_255","doi-asserted-by":"crossref","unstructured":"Perez, D., Samothrakis, S., and Lucas, S. (2014, January 26\u201329). Knowledge-based Fast Evolutionary MCTS for General Video Game Playing. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932868"},{"key":"ref_256","unstructured":"Robles, D., Rohlfshagen, P., and Lucas, S.M. (September, January 31). Learning Non-Random Moves for Playing Othello: Improving Monte Carlo Tree Search. Proceedings of the IEEE Conference on Computational Intelligence and Games, Seoul, Korea."},{"key":"ref_257","doi-asserted-by":"crossref","unstructured":"Onta\u00f1\u00f3n, S. (2016, January 20\u201323). Informed Monte Carlo Tree Search for Real-Time Strategy games. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Santorini, Greece.","DOI":"10.1109\/CIG.2016.7860394"},{"key":"ref_258","doi-asserted-by":"crossref","unstructured":"Swiechowski, M., Tajmajer, T., and Janusz, A. (2018, January 14\u201317). Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Maastricht, The Netherlands.","DOI":"10.1109\/CIG.2018.8490368"},{"key":"ref_259","doi-asserted-by":"crossref","unstructured":"Alhejali, A.M., and Lucas, S.M. (2013, January 11\u201313). Using Genetic Programming to Evolve Heuristics for a Monte Carlo Tree Search Ms Pac-Man Agent. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Niagara Falls, ON, Canada.","DOI":"10.1109\/CIG.2013.6633639"},{"key":"ref_260","first-page":"349","article-title":"Fast Evolutionary Adaptation for Monte Carlo Tree Search","volume":"Volume 8602","author":"Mora","year":"2014","journal-title":"Applications of Evolutionary Computation. EvoApplications 2014. Lecture Notes in Computer Science"},{"key":"ref_261","doi-asserted-by":"crossref","unstructured":"Vodopivec, T., and \u0160ter, B. (2014, January 26\u201329). Enhancing Upper Confidence Bounds for Trees with Temporal Difference Values. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Dortmund, Germany.","DOI":"10.1109\/CIG.2014.6932895"},{"key":"ref_262","doi-asserted-by":"crossref","unstructured":"Gelly, S., and Silver, D. (2007, January 20\u201324). Combining Online and Offline Knowledge in UCT. Proceedings of the 24th international conference on Machine learning, Corvalis, OR, USA.","DOI":"10.1145\/1273496.1273531"},{"key":"ref_263","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1145\/122344.122377","article-title":"Dyna, an Integrated Architecture for Learning, Planning, and Reacting","volume":"2","author":"Sutton","year":"1991","journal-title":"ACM SIGART Bull."},{"key":"ref_264","doi-asserted-by":"crossref","unstructured":"Sutton, R.S. (1990, January 21\u201323). Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming. Proceedings of the 7th International Conference on Machine Learning, Austin, TX, USA.","DOI":"10.1016\/B978-1-55860-141-3.50030-4"},{"key":"ref_265","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1007\/BF00993104","article-title":"Prioritized Sweeping: Reinforcement Learning With Less Data and Less Time","volume":"13","author":"Moore","year":"1993","journal-title":"Mach. Learn"},{"key":"ref_266","first-page":"2266","article-title":"RTP-Q: A Reinforcement Learning System with Time Constraints Exploration Planning for Accelerating the Learning Rate","volume":"E82\u2013A","author":"Zhao","year":"1999","journal-title":"IEEE Trans. Fundam. Electron. Commun. Comput. Sci."},{"key":"ref_267","unstructured":"Silver, D. (2009). Reinforcement Learning and Simulation-Based Search in Computer Go. [Ph.D. Thesis, University of Alberta]."},{"key":"ref_268","unstructured":"Silver, D., van Hasselt, H., Hessel, M., Schaul, T., Guez, A., Harley, T., Dulac-Arnold, G., Reichert, D., Rabinowitz, N., and Barreto, A. (2017, January 6\u201311). The Predictron: End-To-End Learning and Planning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_269","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s10994-012-5280-0","article-title":"Temporal-difference search in computer Go","volume":"87","author":"Silver","year":"2012","journal-title":"Mach. Learn"},{"key":"ref_270","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1613\/jair.5507","article-title":"On Monte Carlo Tree Search and Reinforcement Learning","volume":"60","author":"Vodopivec","year":"2017","journal-title":"J. Artif. Intell. Res."},{"key":"ref_271","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_272","doi-asserted-by":"crossref","unstructured":"Bacon, P.-L., Harb, J., and Precup, D. (2017, January 4\u20139). The Option-Critic Architecture. Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10916"},{"key":"ref_273","first-page":"407","article-title":"Meta-Case-Based Reasoning: Using Functional Models to Adapt Case-Based Agents","volume":"Volume 2080","author":"Aha","year":"2001","journal-title":"Case-Based Reasoning Research and Development. ICCBR 2001. Lecture Notes in Computer Science"},{"key":"ref_274","unstructured":"Ulam, P., Goel, A., and Jones, J. (2014, January 25\u201326). Reflection in Action: Model-Based Self-Adaptation in Game Playing Agents. Proceedings of the AAAI Workshop. Technical Report WS-04-04, San Jose, CA, USA."},{"key":"ref_275","doi-asserted-by":"crossref","unstructured":"Arrabales, R., Ledezma, A., and Sanchis, A. (2009, January 7\u201310). Towards Conscious-like Behavior in Computer Game Characters. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Milano, Italy.","DOI":"10.1109\/CIG.2009.5286473"},{"key":"ref_276","unstructured":"Arrabales, R., Ledezma, A., and Sanchis, A. (2009, January 11\u201314). CERA-CRANIUM: A Test Bed for Machine Consciousness Research. Proceedings of the International Workshop on Machine Consciousness, Hong Kong, China."},{"key":"ref_277","doi-asserted-by":"crossref","unstructured":"Arrabales, R., Ledezma, A., and Sanchis, A. (2009, January 15\u201317). Establishing a Roadmap and Metrics for Conscious Machines Development. Proceedings of the 8th IEEE International Conference on Cognitive Informatics, Hong Kong, China.","DOI":"10.1109\/COGINF.2009.5250795"},{"key":"ref_278","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1016\/S0893-6080(97)00011-7","article-title":"Networks of Spiking Neurons: The Third Generation of Neural Network Models","volume":"10","author":"Maass","year":"1997","journal-title":"Neural Netw."},{"key":"ref_279","first-page":"125","article-title":"A Spiking Neural Network Based Autonomous Reinforcement Learning Model and Its Application in Decision Making","volume":"Volume 10023","author":"Liu","year":"2016","journal-title":"Advances in Brain Inspired Cognitive Systems. BICS 2016. Lecture Notes in Computer Science"},{"key":"ref_280","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1523\/JNEUROSCI.2854-15.2016","article-title":"Goal-Directed Decision Making with Spiking Neurons","volume":"36","author":"Friedrich","year":"2016","journal-title":"J. Neurosci."},{"key":"ref_281","doi-asserted-by":"crossref","unstructured":"Funge, J., Tu, X., and Terzopoulos, D. (1999, January 8\u201313). Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.","DOI":"10.1145\/311535.311538"},{"key":"ref_282","unstructured":"Tadepalli, P. (1989, January 20\u201325). Lazy Explanation-Based Learning: A Solution to the Intractable Theory Problem. Proceedings of the 11th International Joint Conference on Artificial Intelligence - Volume 1, Detroit, MI, USA."},{"key":"ref_283","doi-asserted-by":"crossref","unstructured":"Thompson, T., and Levine, J. (2009, January 7\u201310). Realtime Execution of Automated Plans using Evolutionary Robotics. Proceedings of the IEEE Symposium on Computational Intelligence and Games, Milano, Italy.","DOI":"10.1109\/CIG.2009.5286456"},{"key":"ref_284","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1080\/09528139508953801","article-title":"Integrating Planning and Learning: The PRODIGY Architecture","volume":"7","author":"Veloso","year":"1995","journal-title":"J. Exp. Theor. Artif. Intell."},{"key":"ref_285","unstructured":"Jim\u00e9nez, S., Fern\u00e1ndez, F., and Borrajo, D. (2008, January 13\u201317). The PELA Architecture: Integrating Planning and Learning to Improve Execution. Proceedings of the 23rd national conference on Artificial intelligence, Chicago, IL, USA."},{"key":"ref_286","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/S0304-3975(00)00078-5","article-title":"Temporal difference learning applied to game playing and the results of application to shogi","volume":"252","author":"Beal","year":"2001","journal-title":"Theor. Comput. Sci."},{"key":"ref_287","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_288","unstructured":"Guyon, I., Luxburg, U., von Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., and Garnett, R. (2017). Thinking Fast and Slow with Deep Learning and Tree Search. Advances in Neural Information Processing Systems 30 (NIPS 2017), Curran Associates, Inc."},{"key":"ref_289","doi-asserted-by":"crossref","unstructured":"Perez-Liebana, D., Lucas, S.M., Gaina, R.D., Togelius, J., Khalifa, A., and Liu, J. (2019). General Video Game Artificial Intelligence, Morgan & Claypool.","DOI":"10.1007\/978-3-031-02122-0"},{"key":"ref_290","unstructured":"Squillero, G., and Sim, K. (2017). Evolving Game-Specific UCB Alternatives for General Video Game Playing. Applications of Evolutionary Computation. EvoApplications 2017. Lecture Notes in Computer Science, Springer."},{"key":"ref_291","doi-asserted-by":"crossref","unstructured":"Ilhan, E., and Etaner-Uyar, A.S. (2017, January 22\u201325). Monte Carlo Tree Search with Temporal-Difference Learning for General Video Game Playing. Proceedings of the IEEE Conference on Computational Intelligence and Games, New York, NY, USA.","DOI":"10.1109\/CIG.2017.8080453"},{"key":"ref_292","first-page":"1","article-title":"True Online Temporal-Difference Learning","volume":"17","author":"Mahmood","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_293","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"ref_294","doi-asserted-by":"crossref","unstructured":"Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., and Graepel, T. (2019). Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv.","DOI":"10.1038\/s41586-020-03051-4"},{"key":"ref_295","doi-asserted-by":"crossref","unstructured":"Keehl, O., and Smith, A.M. (2019, January 20\u201323). Monster Carlo 2: Integrating Learning and Tree Search for Machine Playtesting. Proceedings of the IEEE Conference on Games, London, UK.","DOI":"10.1109\/CIG.2019.8847989"},{"key":"ref_296","doi-asserted-by":"crossref","unstructured":"Keehl, O., and Smith, A.M. (2018, January 14\u201317). Monster Carlo: An MCTS-based Framework for Machine Playtesting Unity Games. Proceedings of the IEEE Conference on Computatonal Intelligence and Games, Maastricht, The Netherlands.","DOI":"10.1109\/CIG.2018.8490363"},{"key":"ref_297","unstructured":"Sigurdson, D., and Bulitko, V. (2017, January 5\u20139). Deep Learning for Real-Time Heuristic Search Algorithm Selection. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Snowbird, UT, USA."},{"key":"ref_298","unstructured":"Quit\u00e9rio, J., Prada, R., and Melo, F.S. (September, January 31). A Reinforcement Learning Approach for the Circle Agent of Geometry Friends. Proceedings of the IEEE Conference on Computational Intelligence and Games, Tainan, Taiwan."},{"key":"ref_299","doi-asserted-by":"crossref","unstructured":"Chen, Y., and Li, Y. (2019, January 20\u201323). Macro and Micro Reinforcement Learning for Playing Nine-ball Pool. Proceedings of the IEEE Conference on Games, London, UK.","DOI":"10.1109\/CIG.2019.8848113"},{"key":"ref_300","doi-asserted-by":"crossref","unstructured":"Tastan, B., Chang, Y., and Sukthankar, G. (2012, January 11\u201314). Learning to Intercept Opponents in First Person Shooter Games. Proceedings of the IEEE Conference on Computational Intelligence and Games, Granada, Spain.","DOI":"10.1109\/CIG.2012.6374144"},{"key":"ref_301","doi-asserted-by":"crossref","unstructured":"Barriga, N.A., Stanescu, M., and Buro, M. (2017, January 5\u20139). Combining Strategic Learning and Tactical Search in Real-Time Strategy Games. Proceedings of the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Snowbird, UT, USA.","DOI":"10.1609\/aiide.v13i1.12922"},{"key":"ref_302","unstructured":"Archibald, C., Altman, A., and Shoham, Y. (2009, January 11\u201317). Analysis of a Winning Computational Billiards Player. Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, CA, USA."},{"key":"ref_303","unstructured":"Hefny, A.S., Hatem, A.A., Shalaby, M.M., and Atiya, A.F. (2008, January 22\u201324). Cerberus: Applying Supervised and Reinforcement Learning Techniques to Capture the Flag Games. Proceedings of the 4th Artificial Intelligence and Interactive Digital Entertainment Conference, Palo Alto, CA, USA."}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/10\/13\/4529\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:44:48Z","timestamp":1760175888000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/10\/13\/4529"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,30]]},"references-count":303,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["app10134529"],"URL":"https:\/\/doi.org\/10.3390\/app10134529","relation":{},"ISSN":["2076-3417"],"issn-type":[{"value":"2076-3417","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,30]]}}}