{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T16:57:47Z","timestamp":1774025867395,"version":"3.50.1"},"reference-count":124,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2008,4,1]],"date-time":"2008-04-01T00:00:00Z","timestamp":1207008000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"published-print":{"date-parts":[[2008,4]]},"DOI":"10.1007\/s10462-009-9112-y","type":"journal-article","created":{"date-parts":[[2009,8,4]],"date-time":"2009-08-04T11:33:30Z","timestamp":1249385610000},"page":"123-161","source":"Crossref","is-referenced-by-count":58,"title":["Machine learning in digital games: a survey"],"prefix":"10.1007","volume":"29","author":[{"given":"Leo","family":"Galway","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Darryl","family":"Charles","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michaela","family":"Black","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2009,8,5]]},"reference":[{"key":"9112_CR1","doi-asserted-by":"crossref","unstructured":"Agapitos A, Togelius J, Lucas SM (2007a) Evolving controllers for simulated car racing using object orientated genetic programming. In: Thierens D et\u00a0al (eds) Proceedings of the 2007 genetic and evolutionary computation conference. ACM Press, pp 1543\u20131550","DOI":"10.1145\/1276958.1277271"},{"key":"9112_CR2","doi-asserted-by":"crossref","unstructured":"Agapitos A, Togelius J, Lucas SM (2007b) Multiobjective techniques for the use of state in genetic programming applied to simulated car racing. In: Proceedings of the 2007 IEEE congress on evolutionary computation. IEEE Press, pp 1562\u20131569","DOI":"10.1109\/CEC.2007.4424659"},{"key":"9112_CR3","volume-title":"AI game programming wisdom 3","author":"C Baekkelund","year":"2006","unstructured":"Baekkelund C (2006) A brief comparison of machine learning methods. In: Rabin S (eds) AI game programming wisdom 3, 1st edn. Charles River Media, Hingham","edition":"1"},{"key":"9112_CR4","unstructured":"Baluja S (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. Technical report CMU-CS-94-163, Carnegie Mellon University"},{"key":"9112_CR5","unstructured":"Bauckhage C, Thurau C (2004) Towards a fair \u2018n square aimbot\u2014using mixtures of experts to learn context aware weapon handling. In: El-Rhalibi A, Van Welden D (eds) Proceedings of the 5th international conference on intelligent games and simulation. Eurosis, pp 20\u201324"},{"key":"9112_CR6","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-04378-3","volume-title":"The theory of evolutionary strategies","author":"H-G Beyer","year":"2001","unstructured":"Beyer H-G (2001) The theory of evolutionary strategies. Springer, Berlin"},{"issue":"1\u20132","key":"9112_CR7","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/S0004-3702(01)00130-8","volume":"134","author":"D Billings","year":"2002","unstructured":"Billings D, Davidson A, Schaeffer J, Szafron D (2002) The challenge of poker. Artif Intell 134(1\u20132): 201\u2013240","journal-title":"Artif Intell"},{"key":"9112_CR8","doi-asserted-by":"crossref","unstructured":"Blumberg B, Downie M, Ivanov Y, Berlin M, Johnson MP, Tomlinson B (2002) Integrated learning for interactive synthetic characters. In: Proceedings of the 29th annual conference on computer graphics and interactive techniques. ACM Press, pp 417\u2013426","DOI":"10.1145\/566570.566597"},{"key":"9112_CR9","volume-title":"Proceedings of the 2005 IEEE symposium on computational intelligence and games","author":"J Bradley","year":"2005","unstructured":"Bradley J, Hayes G (2005a) Adapting reinforcement learning for computer games: using group utility functions. In: Kendall G, Lucas SM (eds) Proceedings of the 2005 IEEE symposium on computational intelligence and games. IEEE Press, Piscataway"},{"key":"9112_CR10","doi-asserted-by":"crossref","unstructured":"Bradley J, Hayes G (2005b) Group utility functions: learning equilibria between groups of agents in computer games by modifying the reinforcement signal. In: Proceedings of the 2005 IEEE congress on evolutionary computation. IEEE Press, pp 1914\u20131921","DOI":"10.1109\/CEC.2005.1554921"},{"issue":"2","key":"9112_CR11","first-page":"123","volume":"24","author":"L Breiman","year":"1996","unstructured":"Breiman L (1996) Bagging predictors. Mach Learn 24(2): 123\u2013140","journal-title":"Mach Learn"},{"key":"9112_CR12","unstructured":"Bryant BD, Miikkulainen R (2003) Neuroevolution for adaptive teams. In: Proceedings of the 2003 IEEE congress on evolutionary computation. IEEE Press, pp 2194\u20132201"},{"key":"9112_CR13","doi-asserted-by":"crossref","unstructured":"Bryant BD, Miikkulainen R (2006a) Exploiting sensor symmetries in example-based training for intelligent agents. In: Louis SJ, Kendall G (eds) Proceedings of the 2006 IEEE symposium on computational intelligence and games. IEEE Press, pp 90\u201397","DOI":"10.1109\/CIG.2006.311686"},{"key":"9112_CR14","doi-asserted-by":"crossref","unstructured":"Bryant BD, Miikkulainen R (2006b) Evolving stochastic controller networks for intelligent game agents. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 1007\u20131014","DOI":"10.1109\/CEC.2006.1688419"},{"key":"9112_CR15","unstructured":"Bryant BD, Miikkulainen R (2007) Acquiring visibly intelligent behavior with example-guided neuroevolution. In: Proceedings of the 22nd National conference on artificial intelligence. AAAI Press, Menlo park, pp 801\u2013808"},{"key":"9112_CR16","volume-title":"Programming game AI by example","author":"M Buckland","year":"2005","unstructured":"Buckland M (2005) Programming game AI by example. Wordware Publishing, Plano"},{"issue":"3","key":"9112_CR17","first-page":"189","volume":"20","author":"M Buro","year":"1997","unstructured":"Buro M (1997) The othello match of the year: Takeshi Murakami vs. Logistello. ICCA J 20(3): 189\u2013193","journal-title":"ICCA J"},{"issue":"1\u20132","key":"9112_CR18","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0004-3702(01)00129-1","volume":"134","author":"M Campbell","year":"2002","unstructured":"Campbell M, Hoane AJ, Hsu F-H (2002) Deep blue. Artif Intell 134(1\u20132): 57\u201383","journal-title":"Artif Intell"},{"key":"9112_CR19","doi-asserted-by":"crossref","unstructured":"Chaperot B, Fyfe C (2006) Improving artificial intelligence in a motocross game. In: Louis SJ, Kendall G (eds) Proceedings of the 2006 IEEE symposium on computational intelligence and games. IEEE Press, pp 181\u2013186","DOI":"10.1109\/CIG.2006.311698"},{"key":"9112_CR20","unstructured":"Charles D (2003) Enhancing gameplay: challenges for artificial intelligence in digital games. In: Proceedings of digital games research conference 2003. University of Utrecht, 4\u20136 November 2003"},{"key":"9112_CR21","unstructured":"Charles D, McGlinchey S (2004) The past, present and future of artificial neural networks in digital games. In: Mehdi Q, Gough N, Natkin S, Al-Dabass D (eds) Proceedings of the 5th international conference on computer games: artificial intelligence, design and education. The University of Wolverhampton, pp 163\u2013169"},{"issue":"2","key":"9112_CR22","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2): 182\u2013197","journal-title":"IEEE Trans Evol Comput"},{"issue":"1","key":"9112_CR23","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s10479-005-5724-z","volume":"134","author":"P-T DeBoer","year":"2004","unstructured":"DeBoer P-T, Kroese DP, Mannor S, Rubinstein RY (2004) A tutorial on the cross-entropy method. Ann Oper Res 134(1): 19\u201367","journal-title":"Ann Oper Res"},{"key":"9112_CR24","first-page":"19","volume-title":"Foundations of genetic algorithms 2","author":"KA DeJong","year":"1992","unstructured":"DeJong KA, Sarma J (1992) Generation gaps revisited. In: Whitley D (eds) Foundations of genetic algorithms 2. Morgan Kaufmann, San Mateo, pp 19\u201328"},{"key":"9112_CR25","first-page":"80","volume":"2","author":"P Demasi","year":"2003","unstructured":"Demasi P, Cruz AJ (2003) Online coevolution for action games. Int J Intell Games Simul 2: 80\u201388","journal-title":"Int J Intell Games Simul"},{"key":"9112_CR26","first-page":"26","volume-title":"Lecture notes in computer science, vol. 1864","author":"TG Dietterich","year":"2000","unstructured":"Dietterich TG (2000) An overview of MAXQ hierarchical reinforcement learning. In: Choueiry BY, Walsh T (eds) Lecture notes in computer science, vol. 1864. Springer, Berlin, pp 26\u201344"},{"key":"9112_CR27","first-page":"39","volume-title":"Proceedings of the 1st artificial intelligence and interactive digital entertainment conference","author":"T D\u2019Silva","year":"2005","unstructured":"D\u2019Silva T, Janik R, Chrien M, Stanley KO, Miikkulainen R (2005) Retaining learned behavior during real-time neuroevolution. In: Young M, Laird J (eds) Proceedings of the 1st artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 39\u201344"},{"key":"9112_CR28","unstructured":"Duan J, Gough NE, Mehdi QH (2002) Multi-agent reinforcement learning for computer game agents. In: Mehdi QH, Gough NE (eds) Proceedings of the 3rd international conference on intelligent games and simulation. The University of Wolverhampton, pp 104\u2013109"},{"key":"9112_CR29","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","volume":"14","author":"JL Elman","year":"1990","unstructured":"Elman JL (1990) Finding structure in time. Cog Sci 14: 179\u2013211","journal-title":"Cog Sci"},{"key":"9112_CR30","first-page":"46","volume":"8","author":"R Evans","year":"2001","unstructured":"Evans R (2001) The future of AI in games: a personal view. Game Dev 8: 46\u201349","journal-title":"Game Dev"},{"key":"9112_CR31","unstructured":"Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Saitta L (ed) Proceedings of the 13th international conference on machine learning. Morgan Kaufmann, pp 148\u2013156"},{"key":"9112_CR32","first-page":"11","volume-title":"Machines that learn to play games","author":"J F\u00fcrnkranz","year":"2001","unstructured":"F\u00fcrnkranz J (2001) Machine learning in games: a survey. In: F\u00fcrnkranz J, Kubat M (eds) Machines that learn to play games. Nova Science Publishers, Huntington, pp 11\u201359"},{"key":"9112_CR33","doi-asserted-by":"crossref","unstructured":"Gallagher M, Ledwich M (2007) Evolving Pac-Man players: can we learn from raw input? In: Proceedings of the 2007 IEEE symposium on computational intelligence in games. IEEE Press, pp 282\u2013287","DOI":"10.1109\/CIG.2007.368110"},{"key":"9112_CR34","doi-asserted-by":"crossref","unstructured":"Gallagher M, Ryan A (2003) Learning to play Pac-Man: an evolutionary, rule-based approach. In: Proceedings of the 2003 IEEE congress on evolutionary computation. IEEE Press, pp 2462\u20132469","DOI":"10.1109\/CEC.2003.1299397"},{"key":"9112_CR35","unstructured":"Galway L, Charles D, Black M (2007) Temporal difference control within a dynamic environment. In: Roccetti M (ed) Proceedings of the 8th international conference on intelligent games and simulation. Eurosis, pp 42\u201347"},{"key":"9112_CR36","first-page":"54","volume-title":"Challenges in game artificial intelligence: papers from the 2004 AAAI workshop","author":"B Geisler","year":"2004","unstructured":"Geisler B (2004) Integrated machine learning for behavior modeling in video games. In: Fu D, Henke S, Orkin J (eds) Challenges in game artificial intelligence: papers from the 2004 AAAI workshop. AAAI Press, Menlo Park, pp 54\u201362"},{"key":"9112_CR37","volume-title":"Genetic algorithms in search, optimization and machine learning","author":"DE Goldberg","year":"1989","unstructured":"Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading"},{"key":"9112_CR38","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1177\/105971239700500305","volume":"5","author":"F Gomez","year":"1997","unstructured":"Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5: 317\u2013342","journal-title":"Adapt Behav"},{"key":"9112_CR39","first-page":"14","volume-title":"Proceedings of the 3rd artificial intelligence and interactive digital entertainment conference","author":"P Gorniak","year":"2007","unstructured":"Gorniak P, Davis I (2007) SquadSmart hierarchical planning and coordinated plan execution for squads of characters. In: Schaeffer J, Mateas M (eds) Proceedings of the 3rd artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 14\u201319"},{"key":"9112_CR40","unstructured":"Graepel T, Herbrich R, Gold J (2004) Learning to fight. In: Mehdi Q, Gough N, Natkin, S, Al-Dabass D (eds) Proceedings of 5th international conference on computer games: artificial intelligence, design and education. The University of Wolverhampton, pp 193\u2013200"},{"key":"9112_CR41","doi-asserted-by":"crossref","unstructured":"Grand S, Cliff D, Malhotra A (1997) Creatures: artificial life autonomous software agents for home entertainment. In: Proceedings of the 1st international conference on autonomous agents. ACM Press, pp 22\u201329","DOI":"10.1145\/267658.267663"},{"issue":"6","key":"9112_CR42","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1109\/72.329697","volume":"5","author":"MT Hagan","year":"1994","unstructured":"Hagan MT, Menhaj M (1994) Training feed-forward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5(6): 989\u2013993","journal-title":"IEEE Trans Neural Netw"},{"key":"9112_CR43","volume-title":"Neural networks: a comprehensive foundation","author":"S Haykin","year":"1994","unstructured":"Haykin S (1994) Neural networks: a comprehensive foundation. Prentice-Hall, Upper Saddle River"},{"key":"9112_CR44","first-page":"63","volume-title":"Proceedings of the 1st artificial intelligence and interactive digital entertainment conference","author":"H Hoang","year":"2005","unstructured":"Hoang H, Lee-Urban S, Mu\u00f1oz-Avila H (2005) Hierarchical plan representations for encoding strategic game AI. In: Young M, Laird J (eds) Proceedings of the 1st artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 63\u201368"},{"key":"9112_CR45","volume-title":"Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control and artificial intelligence","author":"JH Holland","year":"1975","unstructured":"Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control and artificial intelligence. MIT Press, Cambridge"},{"key":"9112_CR46","volume-title":"Behind deep blue: building the computer that defeated the World Chess Champion","author":"F-H Hsu","year":"2002","unstructured":"Hsu F-H (2002) Behind deep blue: building the computer that defeated the World Chess Champion. Princeton University Press, Princeton"},{"issue":"1","key":"9112_CR47","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","volume":"3","author":"R Jacobs","year":"1991","unstructured":"Jacobs R, Jordan M, Nowlan S, Hinton G (1991) Adaptive mixture of local experts. Neural Comput 3(1): 79\u201387","journal-title":"Neural Comput"},{"key":"9112_CR48","volume-title":"AI game programming wisdom 2","author":"N Kirby","year":"2004","unstructured":"Kirby N (2004) Getting around the limits of machine learning. In: Rabin S(eds) AI game programming wisdom 2, 1st edn. Charles River Media, Hingham","edition":"1"},{"key":"9112_CR49","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/BF00337288","volume":"43","author":"T Kohonen","year":"1982","unstructured":"Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43: 59\u201369","journal-title":"Biol Cybern"},{"key":"9112_CR50","volume-title":"Genetic programming: on the programming of computers by means of natural selection","author":"JP Koza","year":"1992","unstructured":"Koza JP (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge"},{"key":"9112_CR51","unstructured":"Laird JE, Van Lent M (1999) Developing an artificial intelligence engine. In: Proceedings of the 1999 game developers conference, pp 577\u2013588"},{"key":"9112_CR52","first-page":"15","volume":"22","author":"JE Laird","year":"2001","unstructured":"Laird JE, Van Lent M (2001) Human level AI\u2019s killer application: interactive computer games. AI Mag 22: 15\u201325","journal-title":"AI Mag"},{"key":"9112_CR53","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1109\/TEVC.2004.823466","volume":"8","author":"SJ Louis","year":"2004","unstructured":"Louis SJ, McDonnell J (2004) Learning with case-injected genetic algorithms. IEEE Trans Evol Comput 8: 316\u2013327","journal-title":"IEEE Trans Evol Comput"},{"key":"9112_CR54","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1109\/TEVC.2005.856209","volume":"9","author":"SJ Louis","year":"2005","unstructured":"Louis SJ, Miles C (2005) Playing to learn: case-injected genetic algorithms for learning to play computer games. IEEE Trans Evol Comput 9: 669\u2013681","journal-title":"IEEE Trans Evol Comput"},{"key":"9112_CR55","doi-asserted-by":"crossref","unstructured":"Lucas SM (2004) Cellz: A simple dynamic game for testing evolutionary algorithms. In: Proceedings of the 2004 IEEE congress on evolutionary computation. IEEE Press, pp 1007\u20131014","DOI":"10.1109\/CEC.2004.1330972"},{"key":"9112_CR56","unstructured":"Lucas SM (2005) Evolving a neural network location evaluator to play Ms. Pac-Man. In: Kendall G, Lucas SM (eds) Proceedings of the 2005 IEEE symposium on computational intelligence and games. IEEE Press, pp 203\u2013210"},{"key":"9112_CR57","doi-asserted-by":"crossref","unstructured":"Lucas SM, Kendall G (2006) Evolutionary computation and games. IEEE Comput Intell Mag. February, pp 10\u201318","DOI":"10.1109\/MCI.2006.1597057"},{"key":"9112_CR58","doi-asserted-by":"crossref","unstructured":"Lucas SM, Togelius J (2007) Point-to-point car racing: an initial study of evolution versus temporal difference learning. In: Proceedings of the 2007 IEEE symposium on computational intelligence and games. IEEE Press, pp 260\u2013267","DOI":"10.1109\/CIG.2007.368107"},{"key":"9112_CR59","unstructured":"MacNamee B, Cunningham P (2001) Proposal for an agent architecture for proactive persistent non player characters. In: Proceedings of the 12th Irish conference on artificial intelligence and cognitive science, pp 221\u2013232"},{"key":"9112_CR60","first-page":"28","volume":"2","author":"B MacNamee","year":"2003","unstructured":"MacNamee B, Cunningham P (2003) Creating socially interactive non player characters: the\u00a0\u03bc-SIC system. Int J Intell Games Simul 2: 28\u201335","journal-title":"Int J Intell Games Simul"},{"key":"9112_CR61","first-page":"72","volume-title":"Challenges in game artificial intelligence: papers from the 2004 AAAI workshop","author":"C Maderia","year":"2004","unstructured":"Maderia C, Corruble V, Ramalho G, Ratitch B (2004) Bootstrapping the learning process for semi-automated design of a challenging game AI. In: Fu D, Henke S, Orkin J (eds) Challenges in game artificial intelligence: papers from the 2004 AAAI workshop. AAAI Press, Menlo Park, pp 72\u201376"},{"key":"9112_CR62","first-page":"121","volume-title":"Proceedings of the 2nd artificial intelligence and interactive digital entertainment conference","author":"C Maderia","year":"2006","unstructured":"Maderia C, Corruble V, Ramalho G (2006) Designing a reinforcement learning-based adaptive AI for large-scale strategy games. In: Laird J, Schaeffer J (eds) Proceedings of the 2nd artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 121\u2013123"},{"key":"9112_CR63","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1145\/219717.219808","volume":"38","author":"P Maes","year":"1995","unstructured":"Maes P (1995) Artificial life meets entertainment: lifelike autonomous agents. Commun ACM 38: 108\u2013114","journal-title":"Commun ACM"},{"key":"9112_CR64","volume-title":"AI game programming wisdom","author":"J Manslow","year":"2002","unstructured":"Manslow J (2002) Learning and adaptation. In: Rabin S (eds) AI game programming wisdom, 1st edn. Charles River Media, Hingham","edition":"1"},{"key":"9112_CR65","unstructured":"McGlinchey S (2003) Learning of AI players from game observation data. In: Mehdi Q, Gough N, Natkin S (eds) Proceedings of the 4th international conference on intelligent games and simulation. Eurosis, pp 106\u2013110"},{"key":"9112_CR66","doi-asserted-by":"crossref","unstructured":"Miconi T, Channon A (2006) The N-strikes-out algorithm: a steady-state algorithm for coevolution. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 1639\u20131646","DOI":"10.1109\/CEC.2006.1688505"},{"key":"9112_CR67","unstructured":"Miikkulainen R, Bryant BD, Cornelius R, Karpov IV, Stanley KO, Yong CH (2006) Computational intelligence in games. In: Yen GY, Fogel DB (eds) Computational intelligence: principles and practice. IEEE Computational Intelligence Society, Piscataway, pp 155\u2013191"},{"key":"9112_CR68","doi-asserted-by":"crossref","unstructured":"Miles C, Louis SJ (2006) Towards the co-evolution of influence map tree based strategy game players. In: Louis SJ, Kendall G (eds) Proceedings of the 2006 IEEE symposium on computational intelligence and games. IEEE Press, pp 75\u201382","DOI":"10.1109\/CIG.2006.311684"},{"key":"9112_CR69","doi-asserted-by":"crossref","unstructured":"Miles C, Louis SJ, Cole N, McDonnell J (2004a) Learning to play like a human: case injected genetic algorithms for strategic computer gaming. In: Proceedings of the 2004 IEEE congress on evolutionary computing. IEEE Press, pp 1441\u20131448","DOI":"10.1109\/CEC.2004.1331066"},{"key":"9112_CR70","doi-asserted-by":"crossref","unstructured":"Miles C, Louis SJ, Drewes R (2004b) Trap avoidance in strategic computer game playing with case injected genetic algorithms. In: Proceedings of the 2004 genetic and evolutionary computation conference. ACM Press, pp 1365\u20131376","DOI":"10.1007\/978-3-540-24854-5_130"},{"key":"9112_CR71","doi-asserted-by":"crossref","unstructured":"Miles C, Quiroz J, Leigh R, Louis SJ (2007) Co-evolving influence map tree based strategy game players. In: Proceedings of the 2007 IEEE symposium on computational intelligence and games. IEEE Press, pp 88\u201395","DOI":"10.1109\/CIG.2007.368083"},{"key":"9112_CR72","first-page":"105","volume-title":"Proceedings of the 1st artificial intelligence and interactive digital entertainment conference","author":"J Orkin","year":"2005","unstructured":"Orkin J (2005) Agent architecture considerations for real-time planning in games. In: Young M, Laird J (eds) Proceedings of the 1st artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 105\u2013110"},{"key":"9112_CR73","unstructured":"Parker G, Rawlins G (1996) Cyclic genetic algorithms for the locomotion of hexapod robots. In: Proceedings of the 2nd world automation congress, pp 617\u2013622"},{"key":"9112_CR74","doi-asserted-by":"crossref","unstructured":"Parker GB, Doherty TS, Parker M (2005a) Evolution and prioritization of survival strategies for a simulated robot in xpilot. In: Proceedings of the 2005 IEEE congress on evolutionary computing. IEEE Press, pp 2410\u20132415","DOI":"10.1109\/CEC.2005.1554995"},{"key":"9112_CR75","doi-asserted-by":"crossref","unstructured":"Parker G, Parker M, Johnson S (2005b) Evolving autonomous agent control in the xpilot environment. In: Proceedings of the 2005 IEEE congress on evolutionary computation. IEEE Press, pp 2416\u20132421","DOI":"10.1109\/CEC.2005.1554996"},{"key":"9112_CR76","doi-asserted-by":"crossref","unstructured":"Parker GB, Doherty TS, Parker M (2006) Generation of unconstrained looping programs for control of xpilot agents. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 1216\u20131223","DOI":"10.1109\/CEC.2006.1688448"},{"key":"9112_CR77","doi-asserted-by":"crossref","unstructured":"Parker M, Parker GB (2006a) Learning control for xpilot agents in the core. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 800\u2013807","DOI":"10.1109\/CEC.2006.1688393"},{"key":"9112_CR78","doi-asserted-by":"crossref","unstructured":"Parker GB, Parker M (2006b) The incremental evolution of attack agents in xpilot. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 969\u2013975","DOI":"10.1109\/CEC.2006.1688415"},{"key":"9112_CR79","doi-asserted-by":"crossref","unstructured":"Parker M, Parker GB (2006c) Using a queue genetic algorithm to evolve xpilot control strategies on a distributed system. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 1202\u20131207","DOI":"10.1109\/CEC.2006.1688446"},{"key":"9112_CR80","doi-asserted-by":"crossref","unstructured":"Parker M, Parker GB (2007a) The Evolution of multi-layer neural networks for the control of xpilot agents. In: Proceedings of the 2007 IEEE symposium on computational intelligence and games. IEEE Press, pp 232\u2013237","DOI":"10.1109\/CIG.2007.368103"},{"key":"9112_CR81","doi-asserted-by":"crossref","unstructured":"Parker GB, Parker M (2007b) Evolving parameters for xpilot combat agents. In: Proceedings of the 2007 IEEE symposium on computational intelligence and games. IEEE Press, pp 238\u2013243","DOI":"10.1109\/CIG.2007.368104"},{"key":"9112_CR82","unstructured":"Pfeiffer M (2004) Reinforcement learning of strategies for settlers of catan. In: Mehdi Q, Gough N, Natkin, S, Al-Dabass D (eds) Proceedings of the 5th international conference on computer games: artificial intelligence, design and education. The University of Wolverhampton, pp 384\u2013388"},{"key":"9112_CR83","unstructured":"Ponsen M, Spronck P (2004) Improving adaptive game AI with evolutionary learning. In: Mehdi Q, Gough N, Natkin, S, Al-Dabass D (eds) Proceedings of the 5th international conference on computer games: artificial intelligence, design and education. The University of Wolverhampton, pp 389\u2013396"},{"key":"9112_CR84","unstructured":"Ponsen M, Spronck P, Tuyls K (2006) Hierarchical reinforcement learning with deictic representation in a computer game. In: Schobbens P-Y, Vanhoof W, Schwanen G (eds) Proceedings of the 18th Belgium-Netherlands conference on artificial intelligence. University of Namur, pp 251\u2013258"},{"key":"9112_CR85","first-page":"36","volume":"8","author":"D Pottinger","year":"2000","unstructured":"Pottinger D (2000) The Future of AI in games. Game Dev 8: 36\u201339","journal-title":"Game Dev"},{"key":"9112_CR86","volume-title":"Parallel distributed processing: explorations in the microstructure of cognition, vol. 1","author":"DE Rumelhart","year":"1989","unstructured":"Rumelhart DE, Hinton GE, Williams RJ (1989) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: explorations in the microstructure of cognition, vol. 1. MIT Press, Cambridge"},{"key":"9112_CR87","unstructured":"Rummery GA, Niranjan M (1994) On-line Q-learning using connectionist systems. Technical Report CUED\/F-INFENG\/TR 166, Cambridge University, UK"},{"issue":"3","key":"9112_CR88","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1147\/rd.33.0210","volume":"3","author":"AL Samuel","year":"1959","unstructured":"Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3(3): 211\u2013229","journal-title":"IBM J Res Dev"},{"issue":"6","key":"9112_CR89","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1147\/rd.116.0601","volume":"11","author":"AL Samuel","year":"1967","unstructured":"Samuel AL (1967) Some studies in machine learning using the game of checkers. ii \u2013 recent progress. IBM J Res Dev 11(6): 601\u2013617","journal-title":"IBM J Res Dev"},{"key":"9112_CR90","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2733-3","volume-title":"One jump ahead: challenging human supremacy in checkers","author":"J Schaeffer","year":"1997","unstructured":"Schaeffer J (1997) One jump ahead: challenging human supremacy in checkers. Springer, Berlin"},{"key":"9112_CR91","first-page":"189","volume-title":"Advances in computers: 50","author":"J Schaeffer","year":"2000","unstructured":"Schaeffer J (2000) The games computers (and people) play. In: Zelkowitz M (eds) Advances in computers: 50. Academic Press, San Diego, pp 189\u2013266"},{"key":"9112_CR92","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0004-3702(01)00165-5","volume":"134","author":"J Schaeffer","year":"2002","unstructured":"Schaeffer J, Vanden Herik HJ (2002) Games, computers, and artificial intelligence. Artif Intell 134: 1\u20137","journal-title":"Artif Intell"},{"key":"9112_CR93","volume-title":"AI game engine programming","author":"B Schwab","year":"2004","unstructured":"Schwab B (2004) AI game engine programming, 1st edn. Charles River Media, Hingham","edition":"1"},{"key":"9112_CR94","first-page":"20","volume":"2","author":"P Spronck","year":"2003","unstructured":"Spronck P, Sprinkhuizen-Kuyper I, Postma E (2003a) Improving opponent intelligence through offline evolutionary learning. Int J Intell Games Sim 2: 20\u201327","journal-title":"Int J Intell Games Sim"},{"key":"9112_CR95","unstructured":"Spronck P, Sprinkhuizen-Kuyper I, Postma E (2003b) Online adaptation of game opponent AI in simulation and in practice. In: Mehdi Q, Gough N, Natkin S (eds) Proceedings of the 4th international conference on intelligent games and simulation. Eurosis, pp 93\u2013100"},{"key":"9112_CR96","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10994-006-6205-6","volume":"63","author":"P Spronck","year":"2006","unstructured":"Spronck P, Ponsen M, Sprinkhuizen-Kuyper I, Postma E (2006) Adaptive game AI with dynamic scripting. Mach Learn 63: 217\u2013248","journal-title":"Mach Learn"},{"key":"9112_CR97","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1162\/106365602320169811","volume":"10","author":"KO Stanley","year":"2002","unstructured":"Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10: 99\u2013127","journal-title":"Evol Comput"},{"key":"9112_CR98","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1109\/TEVC.2005.856210","volume":"9","author":"KO Stanley","year":"2005","unstructured":"Stanley KO, Bryant BD, Miikkulainen R (2005a) Real-time neuroevolution in the NERO video game. IEEE Trans Evol Comput 9: 653\u2013668","journal-title":"IEEE Trans Evol Comput"},{"key":"9112_CR99","volume-title":"Proceedings of the 2005 IEEE symposium on computational intelligence and games","author":"KO Stanley","year":"2005","unstructured":"Stanley KO, Bryant BD, Miikkulainen R (2005b) Evolving neural network agents in the NERO video game. In: Kendall G, Lucas SM (eds) Proceedings of the 2005 IEEE symposium on computational intelligence and games. IEEE Press, Piscataway, NJ"},{"key":"9112_CR100","first-page":"9","volume":"3","author":"RS Sutton","year":"1988","unstructured":"Sutton RS (1988) Learning to predict by the method of temporal differences. Mach Learn 3: 9\u201344","journal-title":"Mach Learn"},{"key":"9112_CR101","first-page":"1038","volume-title":"Advances in neural information processing systems 8","author":"RS Sutton","year":"1996","unstructured":"Sutton RS (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) Advances in neural information processing systems 8. MIT Press, Cambridge, pp 1038\u20131044"},{"key":"9112_CR102","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"key":"9112_CR103","doi-asserted-by":"crossref","first-page":"2936","DOI":"10.1162\/neco.2006.18.12.2936","volume":"18","author":"I Szita","year":"2006","unstructured":"Szita I, L\u00f5rincz A (2006) Learning tetris using noisy cross-entropy method. Neural Comput 18: 2936\u20132941","journal-title":"Neural Comput"},{"key":"9112_CR104","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1613\/jair.2368","volume":"30","author":"I Szita","year":"2007","unstructured":"Szita I, L\u00f5rincz A (2007) Learning to play using low-complexity rule-based policies: illustrations through Ms. Pac-Man. J Artif Intell Res 30: 659\u2013684","journal-title":"J Artif Intell Res"},{"issue":"3","key":"9112_CR105","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","volume":"38","author":"G Tesauro","year":"1995","unstructured":"Tesauro G (1995) Temporal difference learning and td-gammon. Commun ACM 38(3): 58\u201368","journal-title":"Commun ACM"},{"key":"9112_CR106","unstructured":"Thurau C, Bauckhage C, Sagerer G (2003) Combining self-organizing maps and multilayer perceptrons to learn bot-behavior for a commercial game. In: Mehdi Q, Gough N, Natkin S (eds) Proceedings of the 4th international conference on intelligent games and simulation. Eurosis, pp 119\u2013123"},{"key":"9112_CR107","doi-asserted-by":"crossref","unstructured":"Togelius J, Lucas SM (2005) Evolving controllers for simulated car racing. In: Proceedings of the 2005 IEEE congress on evolutionary computation. IEEE Press, pp 1906\u20131913","DOI":"10.1109\/CEC.2005.1554920"},{"key":"9112_CR108","doi-asserted-by":"crossref","unstructured":"Togelius J, Lucas SM (2006) Evolving robust and specialized car racing skills. In: Proceedings of the 2006 IEEE congress on evolutionary computation. IEEE Press, pp 1187\u20131194","DOI":"10.1109\/CEC.2006.1688444"},{"key":"9112_CR109","doi-asserted-by":"crossref","unstructured":"Togelius J, Burrow P, Lucas SM (2007a) Multi-population competitive co-evolutions of car racing controllers. In: Proceedings of the 2007 IEEE congress on evolutionary computation. IEEE Press, pp 4043\u20134050","DOI":"10.1109\/CEC.2007.4424998"},{"key":"9112_CR110","doi-asserted-by":"crossref","unstructured":"Togelius J, Nardi RD, Lucas SM (2007b) Towards automatic personalized content creation for racing games. In: Proceedings of the 2007 IEEE symposium on computational intelligence and games. IEEE Press, pp 252\u2013259","DOI":"10.1109\/CIG.2007.368106"},{"key":"9112_CR111","first-page":"287","volume-title":"Game programming gems, vol. 2","author":"P Tozour","year":"2001","unstructured":"Tozour P (2001) Influence mapping. In: DeLoura M (eds) Game programming gems, vol. 2. Charles River Media, Hingham, pp 287\u2013297"},{"key":"9112_CR112","volume-title":"AI game programming wisdom","author":"P Tozour","year":"2002","unstructured":"Tozour P (2002) The evolution of game AI. In: Rabin S (eds) AI game programming wisdom, 1st edn. Charles River Media, Hingham","edition":"1"},{"key":"9112_CR113","first-page":"279","volume":"8","author":"C Watkins","year":"1992","unstructured":"Watkins C, Dayan P (1992) Q-learning. Mach Learn 8: 279\u2013292","journal-title":"Mach Learn"},{"key":"9112_CR114","doi-asserted-by":"crossref","unstructured":"Whiteson S, Stone P, Stanley KO, Miikkulainen R, Kohl N (2005) Automatic feature selection in neuroevolution. In: Proceedings of the 2005 genetic and evolutionary computation conference. ACM Press, pp 1225\u20131232","DOI":"10.1145\/1068009.1068210"},{"key":"9112_CR115","unstructured":"Wolpert D, Sill J, Tumer K (2001) Reinforcement learning in distributed domains: beyond team games. In: Proceedings of the 17th international joint conference on artificial intelligence. Morgan Kaufmann, pp 819\u2013824"},{"key":"9112_CR116","first-page":"28","volume":"5","author":"S Woodcock","year":"1998","unstructured":"Woodcock S (1998) Game AI: the state of the industry. Game Dev 5: 28\u201335","journal-title":"Game Dev"},{"key":"9112_CR117","first-page":"34","volume":"6","author":"S Woodcock","year":"1999","unstructured":"Woodcock S (1999) Game AI: The state of the industry. Game Dev 6: 34\u201343","journal-title":"Game Dev"},{"key":"9112_CR118","first-page":"24","volume":"7","author":"S Woodcock","year":"2000","unstructured":"Woodcock S (2000) Game AI: the state of the industry. Game Dev 7: 24\u201332","journal-title":"Game Dev"},{"key":"9112_CR119","first-page":"36","volume":"8","author":"S Woodcock","year":"2001","unstructured":"Woodcock S (2001) Game AI: the state of the industry 2000\u20132001. Game Dev 8: 36\u201344","journal-title":"Game Dev"},{"key":"9112_CR120","first-page":"26","volume":"9","author":"S Woodcock","year":"2002","unstructured":"Woodcock S (2002) Game AI: the state of the industry 2001\u20132002. Game Dev 9: 26\u201331","journal-title":"Game Dev"},{"key":"9112_CR121","unstructured":"Yannakakis GN, Hallam J (2004) Evolving opponents for interesting interactive computer games. In: Schaal S, Ijspeert A, Billard A, Vijayakumar S, Hallam J, Meyer J-A (eds) From animals to animats 8: proceedings of the 8th international conference on simulation of adaptive behaviour. MIT Press, pp 499\u2013508"},{"key":"9112_CR122","doi-asserted-by":"crossref","unstructured":"Yannakakis GN, Levine J, Hallam J (2004) An evolutionary approach for interactive computer games. In: Proceedings of the 2004 IEEE congress on evolutionary computation. IEEE Press, pp 986\u2013993","DOI":"10.1109\/CEC.2004.1330969"},{"key":"9112_CR123","unstructured":"Yannakakis GN, Levine J, Hallam J, Papageorgiou M (2003) Performance, robustness and effort cost comparison of machine learning mechanisms in FlatLand. In: Proceedings of the 11th Mediterranean conference on control and automation. Rhodes, 18\u201320 June 2003"},{"key":"9112_CR124","first-page":"98","volume-title":"Proceedings of the second artificial intelligence and interactive digital entertainment conference","author":"CH Yong","year":"2006","unstructured":"Yong CH, Stanley KO, Miikkulainen R, Karpov IV (2006) Incorporating advice into evolution of neural networks. In: Laird J, Schaeffer J (eds) Proceedings of the second artificial intelligence and interactive digital entertainment conference. AAAI Press, Menlo Park, pp 98\u2013104"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-009-9112-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10462-009-9112-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-009-9112-y","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,11]],"date-time":"2025-02-11T16:35:04Z","timestamp":1739291704000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10462-009-9112-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,4]]},"references-count":124,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2008,4]]}},"alternative-id":["9112"],"URL":"https:\/\/doi.org\/10.1007\/s10462-009-9112-y","relation":{},"ISSN":["0269-2821","1573-7462"],"issn-type":[{"value":"0269-2821","type":"print"},{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,4]]}}}