{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T14:16:18Z","timestamp":1766067378695,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":25,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,7]],"date-time":"2021-07-07T00:00:00Z","timestamp":1625616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,7]]},"DOI":"10.1145\/3449726.3463217","type":"proceedings-article","created":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T14:52:26Z","timestamp":1625755946000},"page":"1483-1487","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Novelty and MCTS"],"prefix":"10.1145","author":[{"given":"Hendrik","family":"Baier","sequence":"first","affiliation":[{"name":"Centrum Wiskunde & Informatica, Amsterdam, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Kaisers","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde & Informatica, Amsterdam, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,8]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"A survey on intrinsic motivation in reinforcement learning. CoRR abs\/1908.06976","author":"Aubret Arthur","year":"2019","unstructured":"Arthur Aubret , La\u00ebtitia Matignon , and Salima Hassas . 2019. A survey on intrinsic motivation in reinforcement learning. CoRR abs\/1908.06976 ( 2019 ). arXiv:1908.06976 Arthur Aubret, La\u00ebtitia Matignon, and Salima Hassas. 2019. A survey on intrinsic motivation in reinforcement learning. CoRR abs\/1908.06976 (2019). arXiv:1908.06976"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CoG47356.2020.9231603"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2021\/555"},{"key":"e_1_3_2_1_4_1","volume-title":"Unifying Count-Based Exploration and Intrinsic Motivation. In Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS","author":"Bellemare Marc G.","year":"2016","unstructured":"Marc G. Bellemare , Sriram Srinivasan , Georg Ostrovski , Tom Schaul , David Saxton , and R\u00e9mi Munos . 2016 . Unifying Count-Based Exploration and Intrinsic Motivation. In Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS 2016), Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). 1471--1479. Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and R\u00e9mi Munos. 2016. Unifying Count-Based Exploration and Intrinsic Motivation. In Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS 2016), Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). 1471--1479."},{"key":"e_1_3_2_1_5_1","volume-title":"Exploration by Random Network Distillation. CoRR abs\/1810.12894","author":"Burda Yuri","year":"2018","unstructured":"Yuri Burda , Harrison Edwards , Amos J. Storkey , and Oleg Klimov . 2018. Exploration by Random Network Distillation. CoRR abs\/1810.12894 ( 2018 ). arXiv:1810.12894 Yuri Burda, Harrison Edwards, Amos J. Storkey, and Oleg Klimov. 2018. Exploration by Random Network Distillation. CoRR abs\/1810.12894 (2018). arXiv:1810.12894"},{"key":"e_1_3_2_1_6_1","volume-title":"Generalized Monte-Carlo Tree Search Extensions for General Game Playing. In Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI","author":"Finnsson Hilmar","year":"2012","unstructured":"Hilmar Finnsson . 2012 . Generalized Monte-Carlo Tree Search Extensions for General Game Playing. In Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI 2012), J\u00f6rg Hoffmann and Bart Selman (Eds.). AAAI Press. Hilmar Finnsson. 2012. Generalized Monte-Carlo Tree Search Extensions for General Game Playing. In Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI 2012), J\u00f6rg Hoffmann and Bart Selman (Eds.). AAAI Press."},{"key":"e_1_3_2_1_7_1","volume-title":"Learning Simulation Control in General Game-Playing Agents. In Twenty-Fourth AAAI Conference on Artificial Intelligence, (AAAI","author":"Finnsson Hilmar","year":"2010","unstructured":"Hilmar Finnsson and Yngvi Bj\u00f6rnsson . 2010 . Learning Simulation Control in General Game-Playing Agents. In Twenty-Fourth AAAI Conference on Artificial Intelligence, (AAAI 2010), Maria Fox and David Poole (Eds.). AAAI Press. Hilmar Finnsson and Yngvi Bj\u00f6rnsson. 2010. Learning Simulation Control in General Game-Playing Agents. In Twenty-Fourth AAAI Conference on Artificial Intelligence, (AAAI 2010), Maria Fox and David Poole (Eds.). AAAI Press."},{"key":"e_1_3_2_1_8_1","volume-title":"Width-Based Planning for General Video-Game Playing. In Eleventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE","author":"Geffner Tomas","year":"2015","unstructured":"Tomas Geffner and Hector Geffner . 2015 . Width-Based Planning for General Video-Game Playing. In Eleventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2015), Arnav Jhala and Nathan Sturtevant (Eds.). 23--29. Tomas Geffner and Hector Geffner. 2015. Width-Based Planning for General Video-Game Playing. In Eleventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2015), Arnav Jhala and Nathan Sturtevant (Eds.). 23--29."},{"key":"e_1_3_2_1_9_1","volume-title":"Combining Online and Offline Knowledge in UCT. In Twenty-Fourth International Conference on Machine Learning (ICML 2007) (ACM International Conference Proceeding Series","volume":"280","author":"Gelly Sylvain","year":"2007","unstructured":"Sylvain Gelly and David Silver . 2007 . Combining Online and Offline Knowledge in UCT. In Twenty-Fourth International Conference on Machine Learning (ICML 2007) (ACM International Conference Proceeding Series , Vol. 227), Zoubin Ghahramani (Ed.). ACM, 273-- 280 . Sylvain Gelly and David Silver. 2007. Combining Online and Offline Knowledge in UCT. In Twenty-Fourth International Conference on Machine Learning (ICML 2007) (ACM International Conference Proceeding Series, Vol. 227), Zoubin Ghahramani (Ed.). ACM, 273--280."},{"key":"e_1_3_2_1_10_1","volume-title":"Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS","author":"Katz Michael","year":"2017","unstructured":"Michael Katz , Nir Lipovetzky , Dany Moshkovich , and Alexander Tuisov . 2017 . Adapting Novelty to Classical Planning as Heuristic Search . In Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS 2017), Laura Barbulescu, Jeremy Frank, Mausam, and Stephen F. Smith (Eds.). 172--180. Michael Katz, Nir Lipovetzky, Dany Moshkovich, and Alexander Tuisov. 2017. Adapting Novelty to Classical Planning as Heuristic Search. In Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS 2017), Laura Barbulescu, Jeremy Frank, Mausam, and Stephen F. Smith (Eds.). 172--180."},{"key":"e_1_3_2_1_11_1","volume-title":"Bandit Based Monte-Carlo Planning. In 17th European Conference on Machine Learning (ECML","author":"Kocsis Levente","year":"2006","unstructured":"Levente Kocsis and Csaba Szepesv\u00e1ri . 2006 . Bandit Based Monte-Carlo Planning. In 17th European Conference on Machine Learning (ECML 2016). 282--293. Levente Kocsis and Csaba Szepesv\u00e1ri. 2006. Bandit Based Monte-Carlo Planning. In 17th European Conference on Machine Learning (ECML 2016). 282--293."},{"volume-title":"Artificial Life XI: Eleventh International Conference on the Synthesis and Simulation of Living Systems, Seth Bullock, Jason Noble, Richard A. Watson, and Mark A. Bedau (Eds.). 329--336","author":"Lehman Joel","key":"e_1_3_2_1_12_1","unstructured":"Joel Lehman and Kenneth O. Stanley . 2008. Exploiting Open-Endedness to Solve Problems Through the Search for Novelty . In Artificial Life XI: Eleventh International Conference on the Synthesis and Simulation of Living Systems, Seth Bullock, Jason Noble, Richard A. Watson, and Mark A. Bedau (Eds.). 329--336 . Joel Lehman and Kenneth O. Stanley. 2008. Exploiting Open-Endedness to Solve Problems Through the Search for Novelty. In Artificial Life XI: Eleventh International Conference on the Synthesis and Simulation of Living Systems, Seth Bullock, Jason Noble, Richard A. Watson, and Mark A. Bedau (Eds.). 329--336."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00025"},{"volume-title":"General Game Playing Using Search for Novelty. Student research project","author":"Leong Wilkins","key":"e_1_3_2_1_14_1","unstructured":"Wilkins Leong . 2017. General Game Playing Using Search for Novelty. Student research project . University of Melbourne , Melbourne, Australia . Wilkins Leong. 2017. General Game Playing Using Search for Novelty. Student research project. University of Melbourne, Melbourne, Australia."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2015.2402393"},{"key":"e_1_3_2_1_16_1","volume-title":"Width and Serialization of Classical Planning Problems. In 20th European Conference on Artificial Intelligence (ECAI 2012)","volume":"545","author":"Lipovetzky Nir","year":"2012","unstructured":"Nir Lipovetzky and Hector Geffner . 2012 . Width and Serialization of Classical Planning Problems. In 20th European Conference on Artificial Intelligence (ECAI 2012) (Frontiers in Artificial Intelligence and Applications , Vol. 242), Luc De Raedt, Christian Bessiere, Didier Dubois, Patrick Doherty, Paolo Frasconi, Fredrik Heintz, and Peter J. F. Lucas (Eds.). 540-- 545 . Nir Lipovetzky and Hector Geffner. 2012. Width and Serialization of Classical Planning Problems. In 20th European Conference on Artificial Intelligence (ECAI 2012) (Frontiers in Artificial Intelligence and Applications, Vol. 242), Luc De Raedt, Christian Bessiere, Didier Dubois, Patrick Doherty, Paolo Frasconi, Fredrik Heintz, and Peter J. F. Lucas (Eds.). 540--545."},{"key":"e_1_3_2_1_17_1","volume-title":"Classical Planning with Simulators: Results on the Atari Video Games. In Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI","author":"Lipovetzky Nir","year":"2015","unstructured":"Nir Lipovetzky , Miquel Ram\u00edrez , and Hector Geffner . 2015 . Classical Planning with Simulators: Results on the Atari Video Games. In Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), Qiang Yang and Michael J. Wooldridge (Eds.). 1610--1616. Nir Lipovetzky, Miquel Ram\u00edrez, and Hector Geffner. 2015. Classical Planning with Simulators: Results on the Atari Video Games. In Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), Qiang Yang and Michael J. Wooldridge (Eds.). 1610--1616."},{"key":"e_1_3_2_1_18_1","volume-title":"Count-Based Exploration with the Successor Representation. In Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI","author":"Machado Marlos C.","year":"2020","unstructured":"Marlos C. Machado , Marc G. Bellemare , and Michael Bowling . 2020 . Count-Based Exploration with the Successor Representation. In Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020). 5125--5133. Marlos C. Machado, Marc G. Bellemare, and Michael Bowling. 2020. Count-Based Exploration with the Successor Representation. In Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020). 5125--5133."},{"key":"e_1_3_2_1_19_1","volume-title":"Count-Based Exploration in Feature Space for Reinforcement Learning. In Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI","author":"Martin Jarryd","year":"2017","unstructured":"Jarryd Martin , Suraj Narayanan Sasikumar , Tom Everitt , and Marcus Hutter . 2017 . Count-Based Exploration in Feature Space for Reinforcement Learning. In Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), Carles Sierra (Ed.). 2471--2478. Jarryd Martin, Suraj Narayanan Sasikumar, Tom Everitt, and Marcus Hutter. 2017. Count-Based Exploration in Feature Space for Reinforcement Learning. In Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), Carles Sierra (Ed.). 2471--2478."},{"key":"e_1_3_2_1_20_1","volume-title":"Count-Based Exploration with Neural Density Models. In 34th International Conference on Machine Learning (ICML 2017)","volume":"2730","author":"Ostrovski Georg","unstructured":"Georg Ostrovski , Marc G. Bellemare , A\u00e4ron van den Oord, and R\u00e9mi Munos. 2017 . Count-Based Exploration with Neural Density Models. In 34th International Conference on Machine Learning (ICML 2017) (Proceedings of Machine Learning Research , Vol. 70), Doina Precup and Yee Whye Teh (Eds.). 2721-- 2730 . Georg Ostrovski, Marc G. Bellemare, A\u00e4ron van den Oord, and R\u00e9mi Munos. 2017. Count-Based Exploration with Neural Density Models. In 34th International Conference on Machine Learning (ICML 2017) (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). 2721--2730."},{"key":"e_1_3_2_1_21_1","volume-title":"Chess and Shogi by Planning with a Learned Model. CoRR abs\/1911.08265","author":"Schrittwieser Julian","year":"2019","unstructured":"Julian Schrittwieser , Ioannis Antonoglou , Thomas Hubert , Karen Simonyan , Laurent Sifre , Simon Schmitt , Arthur Guez , Edward Lockhart , Demis Hassabis , Thore Graepel , Timothy P. Lillicrap , and David Silver . 2019. Mastering Atari , Go , Chess and Shogi by Planning with a Learned Model. CoRR abs\/1911.08265 ( 2019 ). arXiv:1911.08265 Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy P. Lillicrap, and David Silver. 2019. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. CoRR abs\/1911.08265 (2019). arXiv:1911.08265"},{"key":"e_1_3_2_1_22_1","volume-title":"Blind Search for Atari-Like Online Planning Revisited. In Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI","author":"Shleyfman Alexander","year":"2016","unstructured":"Alexander Shleyfman , Alexander Tuisov , and Carmel Domshlak . 2016 . Blind Search for Atari-Like Online Planning Revisited. In Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2016), Subbarao Kambhampati (Ed.). 3251--3257. Alexander Shleyfman, Alexander Tuisov, and Carmel Domshlak. 2016. Blind Search for Atari-Like Online Planning Revisited. In Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2016), Subbarao Kambhampati (Ed.). 3251--3257."},{"key":"e_1_3_2_1_23_1","volume-title":"A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science 362, 6419","author":"Silver David","year":"2018","unstructured":"David Silver , Thomas Hubert , Julian Schrittwieser , Ioannis Antonoglou , Matthew Lai , Arthur Guez , Marc Lanctot , Laurent Sifre , Dharshan Kumaran , Thore Graepel , Timothy Lillicrap , Karen Simonyan , and Demis Hassabis . 2018. A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science 362, 6419 ( 2018 ), 1140--1144. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. 2018. A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science 362, 6419 (2018), 1140--1144."},{"key":"e_1_3_2_1_24_1","volume-title":"Mastering the Game of Go without Human Knowledge. Nature 550, 7676","author":"Silver David","year":"2017","unstructured":"David Silver , Julian Schrittwieser , Karen Simonyan , Ioannis Antonoglou , Aja Huang , Arthur Guez , Thomas Hubert , Lucas Baker , Matthew Lai , Adrian Bolton , Yutian Chen , Timothy Lillicrap , Fan Hui , Laurent Sifre , George van den Driessche , Thore Graepel , and Demis Hassabis . 2017. Mastering the Game of Go without Human Knowledge. Nature 550, 7676 ( 2017 ), 354--359. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the Game of Go without Human Knowledge. Nature 550, 7676 (2017), 354--359."},{"key":"e_1_3_2_1_25_1","volume-title":"Thirtieth Annual Conference on Neural Information Processing Systems (NIPS","author":"Tang Haoran","year":"2017","unstructured":"Haoran Tang , Rein Houthooft , Davis Foote , Adam Stooke , Xi Chen , Yan Duan , John Schulman , Filip De Turck , and Pieter Abbeel . 2017 . #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning . In Thirtieth Annual Conference on Neural Information Processing Systems (NIPS 2017), Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2753--2762. Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbeel. 2017. #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning. In Thirtieth Annual Conference on Neural Information Processing Systems (NIPS 2017), Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2753--2762."}],"event":{"name":"GECCO '21: Genetic and Evolutionary Computation Conference","sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"],"location":"Lille France","acronym":"GECCO '21"},"container-title":["Proceedings of the Genetic and Evolutionary Computation Conference Companion"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449726.3463217","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449726.3463217","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:02Z","timestamp":1750191422000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449726.3463217"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,7]]},"references-count":25,"alternative-id":["10.1145\/3449726.3463217","10.1145\/3449726"],"URL":"https:\/\/doi.org\/10.1145\/3449726.3463217","relation":{},"subject":[],"published":{"date-parts":[[2021,7,7]]},"assertion":[{"value":"2021-07-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}