{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T10:39:18Z","timestamp":1761129558006},"reference-count":34,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Evolutionary Computation"],"published-print":{"date-parts":[[2018,9]]},"abstract":"<jats:p> Algorithms that learn through environmental interaction and delayed rewards, or reinforcement learning (RL), increasingly face the challenge of scaling to dynamic, high-dimensional, and partially observable environments. Significant attention is being paid to frameworks from deep learning, which scale to high-dimensional data by decomposing the task through multilayered neural networks. While effective, the representation is complex and computationally demanding. In this work, we propose a framework based on genetic programming which adaptively complexifies policies through interaction with the task. We make a direct comparison with several deep reinforcement learning frameworks in the challenging Atari video game environment as well as more traditional reinforcement learning frameworks based on a priori engineered features. Results indicate that the proposed approach matches the quality of deep learning while being a minimum of three orders of magnitude simpler with respect to model complexity. This results in real-time operation of the champion RL agent without recourse to specialized hardware support. Moreover, the approach is capable of evolving solutions to multiple game titles simultaneously with no additional computational cost. In this case, agent behaviours for an individual game as well as single agents capable of playing all games emerge from the same evolutionary run. <\/jats:p>","DOI":"10.1162\/evco_a_00232","type":"journal-article","created":{"date-parts":[[2018,6,22]],"date-time":"2018-06-22T15:57:10Z","timestamp":1529683030000},"page":"347-380","source":"Crossref","is-referenced-by-count":32,"title":["Emergent Solutions to High-Dimensional Multitask Reinforcement Learning"],"prefix":"10.1162","volume":"26","author":[{"given":"Stephen","family":"Kelly","sequence":"first","affiliation":[{"name":"Department of Computer Science, Dalhousie University, 6050 University Avenue, Halifax, NS, B3H 4R2, Canada"}]},{"given":"Malcolm I.","family":"Heywood","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Dalhousie University, 6050 University Avenue, Halifax, NS, B3H 4R2, Canada"}]}],"member":"281","reference":[{"key":"B1","first-page":"47:253","author":"Bellemare M. G.","year":"2012","journal-title":"Journal of Artificial Intelligence Research"},{"key":"B2","first-page":"864","author":"Bellemare M. G.","year":"2012","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1023\/A:1012978805372"},{"key":"B4","volume-title":"Linear genetic programming","author":"Brameier M.","year":"2007","edition":"1"},{"issue":"1","key":"B6","first-page":"1","volume":"7","author":"Dem\u0161ar J","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"B7","first-page":"97","author":"Doucette J. A.","year":"2012","journal-title":"Proceedings of the ACM Genetic and Evolutionary Computation Conference"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2013.2294713"},{"key":"B9","author":"Hausknecht M.","year":"2015","journal-title":"AAAI Workshop on Learning for General Competency in Video Games"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025124423708"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511921803"},{"key":"B12","first-page":"3110","author":"Kelly S.","year":"2014","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B13","first-page":"75","author":"Kelly S.","year":"2014","journal-title":"European Conference on Genetic Programming"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-55696-3_5"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1145\/3071178.3071303"},{"key":"B16","first-page":"3245","author":"Kelly S.","year":"2012","journal-title":"IEEE Congress on Evolutionary Computation"},{"key":"B17","volume-title":"Genetic programming theory and practice XVI","author":"Kelly S.","year":"2018"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-27645-3_18"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00025"},{"key":"B21","first-page":"485","author":"Liang Y.","year":"2016","journal-title":"Proceedings of the ACM International Conference on Autonomous Agents and Multiagent Systems"},{"key":"B22","first-page":"9:331","author":"Lichodzijewski P.","year":"2008","journal-title":"Genetic Programming and Evolvable Machines"},{"key":"B23","first-page":"863","author":"Lichodzijewski P.","year":"2008","journal-title":"Proceedings of the ACM Genetic and Evolutionary Computation Conference"},{"key":"B24","first-page":"853","author":"Lichodzijewski P.","year":"2010","journal-title":"Proceedings of the ACM Genetic and Evolutionary Computation Conference"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-7747-2_3"},{"key":"B26","first-page":"1928","author":"Mnih V.","year":"2016","journal-title":"Proceedings of the 33rd International Conference on Machine Learning"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"B28","author":"Nair A.","year":"2015","journal-title":"International Conference on Machine Learning\u2014Deep Learning Workshop"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1177\/105971239700500306"},{"key":"B31","first-page":"265","author":"Pepels T.","year":"2012","journal-title":"IEEE Symposium on Computational Intelligence in Games"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2015.2390615"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77553-1_9"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-27645-3_17"},{"key":"B35","first-page":"1708","author":"Thomason R.","year":"2007","journal-title":"Proceedings of the ACM Genetic and Evolutionary Computation Conference"},{"key":"B36","first-page":"2094","author":"van Hasselt H.","year":"2016","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B37","first-page":"1403","author":"Wu S.","year":"2011","journal-title":"Proceedings of the ACM Genetic and Evolutionary Computation Conference"}],"container-title":["Evolutionary Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/evco_a_00232","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:59:03Z","timestamp":1615586343000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/evco\/article\/26\/3\/347-380\/1067"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9]]},"references-count":34,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2018,9]]}},"alternative-id":["10.1162\/evco_a_00232"],"URL":"https:\/\/doi.org\/10.1162\/evco_a_00232","relation":{},"ISSN":["1063-6560","1530-9304"],"issn-type":[{"value":"1063-6560","type":"print"},{"value":"1530-9304","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9]]}}}