{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:22:25Z","timestamp":1750220545108,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":38,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,6,26]],"date-time":"2021-06-26T00:00:00Z","timestamp":1624665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100002632","name":"Association Nationale Recherche Technologie","doi-asserted-by":"publisher","award":["2018\/0726"],"award-info":[{"award-number":["2018\/0726"]}],"id":[{"id":"10.13039\/100002632","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,6,26]]},"DOI":"10.1145\/3449639.3459314","type":"proceedings-article","created":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T17:50:43Z","timestamp":1624297843000},"page":"154-162","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Sparse reward exploration via novelty search and emitters"],"prefix":"10.1145","author":[{"given":"Giuseppe","family":"Paolo","sequence":"first","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandre","family":"Coninx","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephane","family":"Doncieux","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alban","family":"Laflaqui\u00e8re","sequence":"additional","affiliation":[{"name":"SoftBank Robotics Europe, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,6,26]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"OpenAI Pieter Abbeel, and Wojciech Zaremba","author":"Andrychowicz Marcin","year":"2017","unstructured":"Marcin Andrychowicz , Filip Wolski , Alex Ray , Jonas Schneider , Rachel Fong , Peter Welinder , Bob McGrew , Josh Tobin , OpenAI Pieter Abbeel, and Wojciech Zaremba . 2017 . Hindsight experience replay. In Advances in Neural Information Processing Systems . 5048--5058. Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. 2017. Hindsight experience replay. In Advances in Neural Information Processing Systems. 5048--5058."},{"key":"e_1_3_2_2_2_1","volume-title":"Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems 29","author":"Bellemare Marc","year":"2016","unstructured":"Marc Bellemare , Sriram Srinivasan , Georg Ostrovski , Tom Schaul , David Saxton , and Remi Munos . 2016. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems 29 ( 2016 ), 1471--1479. Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. 2016. Unifying count-based exploration and intrinsic motivation. Advances in neural information processing systems 29 (2016), 1471--1479."},{"key":"e_1_3_2_2_3_1","volume-title":"Evolution strategies-A comprehensive introduction. Natural computing 1, 1","author":"Beyer Hans-Georg","year":"2002","unstructured":"Hans-Georg Beyer and Hans-Paul Schwefel . 2002. Evolution strategies-A comprehensive introduction. Natural computing 1, 1 ( 2002 ), 3--52. Hans-Georg Beyer and Hans-Paul Schwefel. 2002. Evolution strategies-A comprehensive introduction. Natural computing 1, 1 (2002), 3--52."},{"volume-title":"Advances in Neural Information Processing (NeurIPS'19). Curran Associates","author":"Blaes Sebastian","unstructured":"Sebastian Blaes , Marin Vlastelica , Jia-Jie Zhu , and Georg Martius . 2019. Control What You Can: Intrinsically Motivated Task-Planning Agent . In Advances in Neural Information Processing (NeurIPS'19). Curran Associates , Inc ., 12520--12531. Sebastian Blaes, Marin Vlastelica, Jia-Jie Zhu, and Georg Martius. 2019. Control What You Can: Intrinsically Motivated Task-Planning Agent. In Advances in Neural Information Processing (NeurIPS'19). Curran Associates, Inc., 12520--12531.","key":"e_1_3_2_2_4_1"},{"key":"e_1_3_2_2_5_1","volume-title":"Xavier Giroi Nieto, and Jordi Torres","author":"Campos V\u00edctor","year":"2020","unstructured":"V\u00edctor Campos , Alexander Trott , Caiming Xiong , Richard Socher , Xavier Giroi Nieto, and Jordi Torres . 2020 . Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills . arXiv preprint arXiv:2002.03647 (2020). V\u00edctor Campos, Alexander Trott, Caiming Xiong, Richard Socher, Xavier Giroi Nieto, and Jordi Torres. 2020. Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills. arXiv preprint arXiv:2002.03647 (2020)."},{"key":"e_1_3_2_2_6_1","volume-title":"QD-RL: Efficient Mixing of Quality and Diversity in Reinforcement Learning. arXiv preprint arXiv:2006.08505","author":"Cideron Geoffrey","year":"2020","unstructured":"Geoffrey Cideron , Thomas Pierrot , Nicolas Perrin , Karim Beguir , and Olivier Sigaud . 2020. QD-RL: Efficient Mixing of Quality and Diversity in Reinforcement Learning. arXiv preprint arXiv:2006.08505 ( 2020 ). Geoffrey Cideron, Thomas Pierrot, Nicolas Perrin, Karim Beguir, and Olivier Sigaud. 2020. QD-RL: Efficient Mixing of Quality and Diversity in Reinforcement Learning. arXiv preprint arXiv:2006.08505 (2020)."},{"key":"e_1_3_2_2_7_1","volume-title":"International Conference on Machine Learning. PMLR, 1039--1048","author":"Colas C\u00e9dric","year":"2018","unstructured":"C\u00e9dric Colas , Olivier Sigaud , and Pierre-Yves Oudeyer . 2018 . Gep-pg: Decoupling exploration and exploitation in deep reinforcement learning algorithms . In International Conference on Machine Learning. PMLR, 1039--1048 . C\u00e9dric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. 2018. Gep-pg: Decoupling exploration and exploitation in deep reinforcement learning algorithms. In International Conference on Machine Learning. PMLR, 1039--1048."},{"key":"e_1_3_2_2_8_1","volume-title":"Joel Lehman, Kenneth Stanley, and Jeff Clune.","author":"Conti Edoardo","year":"2018","unstructured":"Edoardo Conti , Vashisht Madhavan , Felipe Petroski Such , Joel Lehman, Kenneth Stanley, and Jeff Clune. 2018 . Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. In Advances in neural information processing systems. 5027--5038. Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth Stanley, and Jeff Clune. 2018. Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. In Advances in neural information processing systems. 5027--5038."},{"key":"e_1_3_2_2_9_1","volume-title":"Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters. arXiv preprint arXiv:2007.05352","author":"Cully Antoine","year":"2020","unstructured":"Antoine Cully . 2020. Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters. arXiv preprint arXiv:2007.05352 ( 2020 ). Antoine Cully. 2020. Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters. arXiv preprint arXiv:2007.05352 (2020)."},{"key":"e_1_3_2_2_10_1","volume-title":"Robots that can adapt like animals. Nature 521, 7553","author":"Cully Antoine","year":"2015","unstructured":"Antoine Cully , Jeff Clune , Danesh Tarapore , and Jean-Baptiste Mouret . 2015. Robots that can adapt like animals. Nature 521, 7553 ( 2015 ), 503. Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. 2015. Robots that can adapt like animals. Nature 521, 7553 (2015), 503."},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_11_1","DOI":"10.1109\/TEVC.2017.2704781"},{"key":"e_1_3_2_2_12_1","volume-title":"A fast and elitist multiobjective genetic algorithm: NSGA-II","author":"Deb Kalyanmoy","year":"2002","unstructured":"Kalyanmoy Deb , Amrit Pratap , Sameer Agarwal , and TAMT Meyarivan . 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II . IEEE transactions on evolutionary computation 6, 2 ( 2002 ), 182--197. Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2 (2002), 182--197."},{"key":"e_1_3_2_2_13_1","volume-title":"Attraction-repulsion actor-critic for continuous control reinforcement learning. arXiv preprint arXiv:1909.07543","author":"Doan Thang","year":"2019","unstructured":"Thang Doan , BogdanMazoure, Moloud Abdar , Audrey Durand , Joelle Pineau , and R Devon Hjelm . 2019. Attraction-repulsion actor-critic for continuous control reinforcement learning. arXiv preprint arXiv:1909.07543 ( 2019 ). Thang Doan, BogdanMazoure, Moloud Abdar, Audrey Durand, Joelle Pineau, and R Devon Hjelm. 2019. Attraction-repulsion actor-critic for continuous control reinforcement learning. arXiv preprint arXiv:1909.07543 (2019)."},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_14_1","DOI":"10.1145\/3321707.3321752"},{"key":"e_1_3_2_2_15_1","volume-title":"First return, then explore. Nature 590, 7847","author":"Ecoffet Adrien","year":"2021","unstructured":"Adrien Ecoffet , Joost Huizinga , Joel Lehman , Kenneth O Stanley , and Jeff Clune . 2021. First return, then explore. Nature 590, 7847 ( 2021 ), 580--586. Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune. 2021. First return, then explore. Nature 590, 7847 (2021), 580--586."},{"key":"e_1_3_2_2_16_1","volume-title":"Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070","author":"Eysenbach Benjamin","year":"2018","unstructured":"Benjamin Eysenbach , Abhishek Gupta , Julian Ibarz , and Sergey Levine . 2018. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070 ( 2018 ). Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. 2018. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070 (2018)."},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_17_1","DOI":"10.1145\/3377930.3390232"},{"key":"e_1_3_2_2_18_1","volume-title":"Intrinsically motivated goal exploration processes with automatic curriculum learning. arXiv preprint arXiv:1708.02190","author":"Forestier S\u00e9bastien","year":"2017","unstructured":"S\u00e9bastien Forestier , R\u00e9my Portelas , Yoan Mollard , and Pierre-Yves Oudeyer . 2017. Intrinsically motivated goal exploration processes with automatic curriculum learning. arXiv preprint arXiv:1708.02190 ( 2017 ). S\u00e9bastien Forestier, R\u00e9my Portelas, Yoan Mollard, and Pierre-Yves Oudeyer. 2017. Intrinsically motivated goal exploration processes with automatic curriculum learning. arXiv preprint arXiv:1708.02190 (2017)."},{"key":"e_1_3_2_2_19_1","volume-title":"curiosity, and attention: computational and neural mechanisms. Trends in cognitive sciences 17, 11","author":"Gottlieb Jacqueline","year":"2013","unstructured":"Jacqueline Gottlieb , Pierre-Yves Oudeyer , Manuel Lopes , and Adrien Baranes . 2013. Information-seeking , curiosity, and attention: computational and neural mechanisms. Trends in cognitive sciences 17, 11 ( 2013 ), 585--593. Jacqueline Gottlieb, Pierre-Yves Oudeyer, Manuel Lopes, and Adrien Baranes. 2013. Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends in cognitive sciences 17, 11 (2013), 585--593."},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Gravina Daniele","year":"2016","unstructured":"Daniele Gravina , Antonios Liapis , and Georgios Yannakakis . 2016 . Surprise search: Beyond objectives and novelty . In Proceedings of the Genetic and Evolutionary Computation Conference 2016. ACM, 677--684. Daniele Gravina, Antonios Liapis, and Georgios Yannakakis. 2016. Surprise search: Beyond objectives and novelty. In Proceedings of the Genetic and Evolutionary Computation Conference 2016. ACM, 677--684."},{"key":"e_1_3_2_2_21_1","volume-title":"The CMA evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772","author":"Hansen Nikolaus","year":"2016","unstructured":"Nikolaus Hansen . 2016. The CMA evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772 ( 2016 ). Nikolaus Hansen. 2016. The CMA evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772 (2016)."},{"key":"e_1_3_2_2_22_1","volume-title":"Population-guided parallel policy search for reinforcement learning. arXiv preprint arXiv:2001.02907","author":"Jung Whiyoung","year":"2020","unstructured":"Whiyoung Jung , Giseung Park , and Youngchul Sung . 2020. Population-guided parallel policy search for reinforcement learning. arXiv preprint arXiv:2001.02907 ( 2020 ). Whiyoung Jung, Giseung Park, and Youngchul Sung. 2020. Population-guided parallel policy search for reinforcement learning. arXiv preprint arXiv:2001.02907 (2020)."},{"unstructured":"Shauharda Khadka and Kagan Tumer. 2018. Evolution-guided policy gradient in reinforcement learning. In Advances in Neural Information Processing Systems. 1188--1200.  Shauharda Khadka and Kagan Tumer. 2018. Evolution-guided policy gradient in reinforcement learning. In Advances in Neural Information Processing Systems. 1188--1200.","key":"e_1_3_2_2_23_1"},{"unstructured":"Joel Lehman and Kenneth O Stanley. 2008. Exploiting open-endedness to solve problems through the search for novelty.. In ALIFE. 329--336.  Joel Lehman and Kenneth O Stanley. 2008. Exploiting open-endedness to solve problems through the search for novelty.. In ALIFE. 329--336.","key":"e_1_3_2_2_24_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_25_1","DOI":"10.1145\/2001576.2001606"},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_26_1","DOI":"10.1109\/DEVLRN.2017.8329814"},{"key":"e_1_3_2_2_27_1","volume-title":"Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909","author":"Mouret Jean-Baptiste","year":"2015","unstructured":"Jean-Baptiste Mouret and Jeff Clune . 2015. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909 ( 2015 ). Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909 (2015)."},{"unstructured":"Ashvin V Nair Vitchyr Pong Murtaza Dalal Shikhar Bahl Steven Lin and Sergey Levine. 2018. Visual reinforcement learning with imagined goals. In Advances in Neural Information Processing Systems. 9191--9200.  Ashvin V Nair Vitchyr Pong Murtaza Dalal Shikhar Bahl Steven Lin and Sergey Levine. 2018. Visual reinforcement learning with imagined goals. In Advances in Neural Information Processing Systems. 9191--9200.","key":"e_1_3_2_2_28_1"},{"key":"e_1_3_2_2_29_1","volume-title":"Unsupervised Learning and Exploration of Reachable Outcome Space. algorithms 24","author":"Paolo Giuseppe","year":"2019","unstructured":"Giuseppe Paolo , Alban Laflaquiere , Alexandre Coninx , and Stephane Doncieux . 2019. Unsupervised Learning and Exploration of Reachable Outcome Space. algorithms 24 ( 2019 ), 25. Giuseppe Paolo, Alban Laflaquiere, Alexandre Coninx, and Stephane Doncieux. 2019. Unsupervised Learning and Exploration of Reachable Outcome Space. algorithms 24 (2019), 25."},{"key":"e_1_3_2_2_30_1","volume-title":"Effective diversity in population-based reinforcement learning. arXiv preprint arXiv:2002.00632","author":"Parker-Holder Jack","year":"2020","unstructured":"Jack Parker-Holder , Aldo Pacchiano , Krzysztof Choromanski , and Stephen Roberts . 2020. Effective diversity in population-based reinforcement learning. arXiv preprint arXiv:2002.00632 ( 2020 ). Jack Parker-Holder, Aldo Pacchiano, Krzysztof Choromanski, and Stephen Roberts. 2020. Effective diversity in population-based reinforcement learning. arXiv preprint arXiv:2002.00632 (2020)."},{"key":"e_1_3_2_2_31_1","volume-title":"CEM-RL: Combining evolutionary and gradient-based methods for policy search. arXiv preprint arXiv:1810.01222","author":"Pourchot Alo\u00efs","year":"2018","unstructured":"Alo\u00efs Pourchot and Olivier Sigaud . 2018. CEM-RL: Combining evolutionary and gradient-based methods for policy search. arXiv preprint arXiv:1810.01222 ( 2018 ). Alo\u00efs Pourchot and Olivier Sigaud. 2018. CEM-RL: Combining evolutionary and gradient-based methods for policy search. arXiv preprint arXiv:1810.01222 (2018)."},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_32_1","DOI":"10.3389\/frobt.2016.00040"},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_33_1","DOI":"10.1016\/j.neunet.2019.01.011"},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_34_1","DOI":"10.1371\/journal.pone.0162235"},{"volume-title":"Reinforcement learning: An introduction","author":"Sutton Richard S","unstructured":"Richard S Sutton and Andrew G Barto . 2018. Reinforcement learning: An introduction . MIT press . Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.","key":"e_1_3_2_2_35_1"},{"key":"e_1_3_2_2_36_1","volume-title":"Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel.","author":"Tang Haoran","year":"2017","unstructured":"Haoran Tang , Rein Houthooft , Davis Foote , Adam Stooke , OpenAI Xi Chen , Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel. 2017 . # exploration: A study of count-based exploration for deep reinforcement learning. In Advances in neural information processing systems. 2753--2762. Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, OpenAI Xi Chen, Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel. 2017. # exploration: A study of count-based exploration for deep reinforcement learning. In Advances in neural information processing systems. 2753--2762."},{"unstructured":"Alexander Trott Stephan Zheng Caiming Xiong and Richard Socher. 2019. Keeping your distance: Solving sparse reward tasks using self-balancing shaped rewards. In Advances in Neural Information Processing Systems. 10376--10386.  Alexander Trott Stephan Zheng Caiming Xiong and Richard Socher. 2019. Keeping your distance: Solving sparse reward tasks using self-balancing shaped rewards. In Advances in Neural Information Processing Systems. 10376--10386.","key":"e_1_3_2_2_37_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_2_38_1","DOI":"10.1109\/ICGTSPICC.2016.7955308"}],"event":{"sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"],"acronym":"GECCO '21","name":"GECCO '21: Genetic and Evolutionary Computation Conference","location":"Lille France"},"container-title":["Proceedings of the Genetic and Evolutionary Computation Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449639.3459314","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449639.3459314","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:08Z","timestamp":1750195688000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449639.3459314"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,26]]},"references-count":38,"alternative-id":["10.1145\/3449639.3459314","10.1145\/3449639"],"URL":"https:\/\/doi.org\/10.1145\/3449639.3459314","relation":{},"subject":[],"published":{"date-parts":[[2021,6,26]]},"assertion":[{"value":"2021-06-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}