{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T06:04:12Z","timestamp":1790834652302,"version":"4.1.0"},"reference-count":108,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,20]],"date-time":"2023-09-20T00:00:00Z","timestamp":1695168000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Evol. Learn. Optim."],"published-print":{"date-parts":[[2023,9,30]]},"abstract":"<jats:p>Deep neuroevolution and deep Reinforcement Learning have received a lot of attention over the past few years. Some works have compared them, highlighting their pros and cons, but an emerging trend combines them so as to benefit from the best of both worlds. In this article, we provide a survey of this emerging trend by organizing the literature into related groups of works and casting all the existing combinations in each group into a generic framework. We systematically cover all easily available papers irrespective of their publication status, focusing on the combination mechanisms rather than on the experimental results. In total, we cover 45 algorithms more recent than 2017. We hope this effort will favor the growth of the domain by facilitating the understanding of the relationships between the methods, leading to deeper analyses, outlining missing useful comparisons and suggesting new combinations of mechanisms.<\/jats:p>","DOI":"10.1145\/3569096","type":"journal-article","created":{"date-parts":[[2022,10,26]],"date-time":"2022-10-26T10:13:07Z","timestamp":1666779187000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":50,"title":["Combining Evolution and Deep Reinforcement Learning for Policy Search: A Survey"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8544-0229","authenticated-orcid":false,"given":"Olivier","family":"Sigaud","sequence":"first","affiliation":[{"name":"Sorbonne Universit\u00e9, CNRS, Institut des Syst\u00e8mes Intelligents et de Robotique Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,9,20]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"5048","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems (NeurIPS\u201917)","author":"Andrychowicz Marcin","year":"2017","unstructured":"Marcin Andrychowicz, Dwight Crow, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba. 2017. Hindsight experience replay. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems (NeurIPS\u201917), Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5048\u20135058."},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.585888"},{"key":"e_1_3_1_4_2","first-page":"277","volume-title":"Learning for Dynamics and Control","author":"Bharadhwaj Homanga","year":"2020","unstructured":"Homanga Bharadhwaj, Kevin Xie, and Florian Shkurti. 2020. Model-predictive control via cross-entropy and gradient-based optimization. In Learning for Dynamics and Control. PMLR, 277\u2013286."},{"key":"e_1_3_1_5_2","first-page":"3283","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI\u201920), The 32nd Innovative Applications of Artificial Intelligence Conference (IAAI\u201920), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201920)","author":"Bodnar Cristian","year":"2020","unstructured":"Cristian Bodnar, Ben Day, and Pietro Li\u00f3. 2020. Proximal distilled evolutionary reinforcement learning. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI\u201920), The 32nd Innovative Applications of Artificial Intelligence Conference (IAAI\u201920), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201920). AAAI Press, 3283\u20133290."},{"key":"e_1_3_1_6_2","first-page":"1754","volume-title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918)","author":"Chang Simyung","year":"2018","unstructured":"Simyung Chang, John Yang, Jaeseok Choi, and Nojun Kwak. 2018. Genetic-gated networks for deep reinforcement learning. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918), Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 1754\u20131763."},{"key":"e_1_3_1_7_2","unstructured":"Gang Chen. 2019. Merging deterministic policy gradient estimations with varied bias-variance tradeoff for effective deep reinforcement learning. arXiv:1911.10527. Retrieved from https:\/\/arxiv.org\/abs\/1911.10527."},{"key":"e_1_3_1_8_2","first-page":"10299","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems (NeurIPS\u201919)","author":"Choromanski Krzysztof","year":"2019","unstructured":"Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder, Yunhao Tang, and Vikas Sindhwani. 2019. From complexity to simplicity: Adaptive ES-active subspaces for blackbox optimization. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems (NeurIPS\u201919), Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d\u2019Alch\u00e9-Buc, Emily B. Fox, and Roman Garnett (Eds.). 10299\u201310309. https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/88bade49e98db8790df275fcebb37a13-Abstract.html."},{"key":"e_1_3_1_9_2","series-title":"Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),","first-page":"969","volume":"80","author":"Choromanski Krzysztof","year":"2018","unstructured":"Krzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E. Turner, and Adrian Weller. 2018. Structured evolution with compact architectures for scalable policy optimization. In Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),Proceedings of Machine Learning Research, Vol. 80, Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 969\u2013977."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/197"},{"key":"e_1_3_1_11_2","article-title":"Deep reinforcement learning in a handful of trials using probabilistic dynamics models","volume":"31","author":"Chua Kurtland","year":"2018","unstructured":"Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. 2018. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in Neural Information Processing Systems 31 (2018).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_12_2","unstructured":"Geoffrey Cideron Thomas Pierrot Nicolas Perrin Karim Beguir and Olivier Sigaud. 2020. QD-RL: Efficient mixing of quality and diversity in reinforcement learning. arXiv:2006.08505. Retrieved from https:\/\/arxiv.org\/abs\/2006.08505."},{"key":"e_1_3_1_13_2","series-title":"Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),","first-page":"1038","volume":"80","author":"Colas C\u00e9dric","year":"2018","unstructured":"C\u00e9dric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. 2018. GEP-PG: Decoupling exploration and exploitation in deep reinforcement learning algorithms. In Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),Proceedings of Machine Learning Research, Vol. 80, Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 1038\u20131047."},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/3285273"},{"key":"e_1_3_1_15_2","first-page":"5032","volume-title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918)","author":"Conti Edoardo","year":"2018","unstructured":"Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth O. Stanley, and Jeff Clune. 2018. Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918), Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 5032\u20135043."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2017.2704781"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/978-981-16-3802-2_28","volume-title":"Proceedings of International Conference on Computational Intelligence","author":"Dereventsov Anton","year":"2022","unstructured":"Anton Dereventsov, Clayton G. Webster, and Joseph Daws. 2022. An adaptive stochastic gradient-free approach for high-dimensional blackbox optimization. In Proceedings of International Conference on Computational Intelligence. Springer, 333\u2013348."},{"key":"e_1_3_1_18_2","unstructured":"Thang Doan Bogdan Mazoure Audrey Durand Joelle Pineau and R. Devon Hjelm. 2019. Attraction-repulsion actor-critic for continuous control reinforcement learning. arXiv:1909.07543. Retrieved from https:\/\/arxiv.org\/abs\/1909.07543."},{"key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3321707.3321752","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Doncieux Stephane","year":"2019","unstructured":"Stephane Doncieux, Alban Laflaqui\u00e8re, and Alexandre Coninx. 2019. Novelty search: A theoretical perspective. In Proceedings of the Genetic and Evolutionary Computation Conference. 99\u2013106."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2018.03.011"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-64583-0_26"},{"key":"e_1_3_1_22_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921)","author":"Franke J\u00f6rg K. H.","year":"2021","unstructured":"J\u00f6rg K. H. Franke, Gregor K\u00f6hler, Andr\u00e9 Biedenkapp, and Frank Hutter. 2021. Sample-efficient automated deep reinforcement learning. In Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921). OpenReview.net."},{"key":"e_1_3_1_23_2","series-title":"Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),","first-page":"1582","volume":"80","author":"Fujimoto Scott","year":"2018","unstructured":"Scott Fujimoto, Herke van Hoof, and David Meger. 2018. Addressing function approximation error in actor-critic methods. In Proceedings of the 35th International Conference on Machine Learning (ICML\u201918),Proceedings of Machine Learning Research, Vol. 80, Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 1582\u20131591."},{"key":"e_1_3_1_24_2","volume-title":"Proceedings of the 6th International Conference on Learning Representations (ICLR\u201918)","author":"Gangwani Tanmay","year":"2018","unstructured":"Tanmay Gangwani and Jian Peng. 2018. Policy optimization by genetic distillation. In Proceedings of the 6th International Conference on Learning Representations (ICLR\u201918). OpenReview.net. https:\/\/openreview.net\/forum?id=ByOnmlWC-."},{"key":"e_1_3_1_25_2","unstructured":"Jake Grigsby Jin Yong Yoo and Yanjun Qi. 2021. Towards automatic actor-critic solutions to continuous control. arXiv:2106.08918. Retrieved from https:\/\/arxiv.org\/abs\/2106.08918."},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Agrim Gupta Silvio Savarese Surya Ganguli and Li Fei-Fei. 2021. Embodied intelligence via learning and evolution. arXiv:2102.02202. Retrieved from https:\/\/arxiv.org\/abs\/2102.02202.","DOI":"10.1038\/s41467-021-25874-z"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1162\/artl_a_00301"},{"key":"e_1_3_1_28_2","unstructured":"Tuomas Haarnoja Aurick Zhou Kristian Hartikainen George Tucker Sehoon Ha Jie Tan Vikash Kumar Henry Zhu Abhishek Gupta Pieter Abbeel et\u00a0al. 2018. Soft actor-critic algorithms and applications. arXiv:1812.05905. Retrieved from https:\/\/arxiv.org\/abs\/1812.05905."},{"key":"e_1_3_1_29_2","first-page":"2555","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Hafner Danijar","year":"2019","unstructured":"Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. 2019. Learning latent dynamics for planning from pixels. In Proceedings of the International Conference on Machine Learning. PMLR, 2555\u20132565."},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1145\/3449726.3459475","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference Companion","author":"Hallawa Ahmed","year":"2021","unstructured":"Ahmed Hallawa, Thorsten Born, Anke Schmeink, Guido Dartmann, Arne Peine, Lukas Martin, Giovanni Iacca, AE Eiben, and Gerd Ascheid. 2021. Evo-RL: Evolutionary-driven reinforcement learning. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. 153\u2013154."},{"key":"e_1_3_1_31_2","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1145\/3067695.3084202","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference Companion","author":"Hallawa Ahmed","year":"2017","unstructured":"Ahmed Hallawa, Jaro De Roose, Martin Andraud, Marian Verhelst, and Gerd Ascheid. 2017. Instinct-driven dynamic hardware reconfiguration: Evolutionary algorithm optimized compression for autonomous sensory agents. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. 1727\u20131734."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-737550-2.50020-8"},{"key":"e_1_3_1_33_2","first-page":"5405","volume-title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918)","author":"Houthooft Rein","year":"2018","unstructured":"Rein Houthooft, Yuhua Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, and Pieter Abbeel. 2018. Evolved policy gradients. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918), Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 5405\u20135414."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aau6249"},{"key":"e_1_3_1_35_2","unstructured":"Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M. Czarnecki Jeff Donahue Ali Razavi Oriol Vinyals Tim Green Iain Dunning Karen Simonyan et\u00a0al. 2017. Population-based training of neural networks. arXiv:1711.09846."},{"key":"e_1_3_1_36_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920)","author":"Jung Whiyoung","year":"2020","unstructured":"Whiyoung Jung, Giseung Park, and Youngchul Sung. 2020. Population-guided parallel policy search for reinforcement learning. In Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920). OpenReview.net."},{"key":"e_1_3_1_37_2","unstructured":"Dmitry Kalashnikov Alex Irpan Peter Pastor Julian Ibarz Alexander Herzog Eric Jang Deirdre Quillen Ethan Holly Mrinal Kalakrishnan Vincent Vanhoucke et\u00a0al. 2018. QT-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. arXiv:1806.10293. Retrieved from https:\/\/arxiv.org\/abs\/1806.10293."},{"key":"e_1_3_1_38_2","series-title":"Proceedings of the 36th International Conference on Machine Learning (ICML\u201919),","first-page":"3341","volume":"97","author":"Khadka Shauharda","year":"2019","unstructured":"Shauharda Khadka, Somdeb Majumdar, Tarek Nassar, Zach Dwiel, Evren Tumer, Santiago Miret, Yinyin Liu, and Kagan Tumer. 2019. Collaborative evolutionary reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning (ICML\u201919),Proceedings of Machine Learning Research, Vol. 97, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 3341\u20133350."},{"key":"e_1_3_1_39_2","first-page":"1196","volume-title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918)","author":"Khadka Shauharda","year":"2018","unstructured":"Shauharda Khadka and Kagan Tumer. 2018. Evolution-guided policy gradient in reinforcement learning. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918), Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 1196\u20131208."},{"key":"e_1_3_1_40_2","first-page":"28","volume-title":"Proceedings of the 1st IEEE International Conference on Robotic Computing (IRC\u201917)","author":"Khamassi Mehdi","year":"2017","unstructured":"Mehdi Khamassi, George Velentzas, Theodore Tsitsimis, and Costas Tzafestas. 2017. Active exploration and parameterized reinforcement learning applied to a simulated human-robot interaction task. In Proceedings of the 1st IEEE International Conference on Robotic Computing (IRC\u201917). IEEE, 28\u201335."},{"key":"e_1_3_1_41_2","first-page":"203","volume-title":"Proceedings of the IEEE Symposium on Computational Intelligence and Games","author":"Kim Kyung-Joong","year":"2007","unstructured":"Kyung-Joong Kim, Heejin Choi, and Sung-Bae Cho. 2007. Hybrid of evolution and reinforcement learning for othello players. In Proceedings of the IEEE Symposium on Computational Intelligence and Games. IEEE, 203\u2013209."},{"key":"e_1_3_1_42_2","unstructured":"Namyong Kim Hyunsuk Baek and Hayong Shin. 2020. PGPS: Coupling policy gradient with population-based search (unpublished)."},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.5555\/183459"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1162\/evco.1999.7.2.125"},{"key":"e_1_3_1_45_2","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Lee Kyunghyun","year":"2020","unstructured":"Kyunghyun Lee, Byeong-Uk Lee, Ukcheol Shin, and In So Kweon. 2020. An efficient asynchronous method for integrating evolutionary and gradient-based policy search. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.)."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3205455.3205473"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00025"},{"key":"e_1_3_1_48_2","unstructured":"Yuheng Lei Jianyu Chen Shengbo Eben Li and Sifa Zheng. 2022. Zeroth-order actor-critic. arXiv:2201.12518. Retrieved from https:\/\/arxiv.org\/abs\/2201.12518."},{"key":"e_1_3_1_49_2","volume-title":"Proceedings of the Conference on Artificial Life","author":"Leite Abe","year":"2020","unstructured":"Abe Leite, Madhavun Candadai, and Eduardo J. Izquierdo. 2020. Reinforcement learning beyond the Bellman equation: Exploring critic objectives using evolution. In Proceedings of the Conference on Artificial Life(ALIFE\u201920)."},{"key":"e_1_3_1_50_2","volume-title":"Proceedings of the 4th International Conference on Learning Representations (ICLR\u201916)","author":"Lillicrap Timothy P.","year":"2016","unstructured":"Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations (ICLR\u201916), Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1509.02971"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/205"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014352"},{"key":"e_1_3_1_53_2","article-title":"Diversity evolutionary policy deep reinforcement learning","author":"Liu Jian","year":"2021","unstructured":"Jian Liu and Liming Feng. 2021. Diversity evolutionary policy deep reinforcement learning. Comput. Intell. Neurosci. (2021).","journal-title":"Comput. Intell. Neurosci."},{"key":"e_1_3_1_54_2","first-page":"5627","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS\u201918)","author":"Liu Qihao","year":"2018","unstructured":"Qihao Liu, Yujia Wang, and Xiaofeng Liu. 2018. PNS: Population-guided novelty search for reinforcement learning in hard exploration environments. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS\u201918). IEEE, 5627\u20135634."},{"key":"e_1_3_1_55_2","volume-title":"Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI\u201917)","author":"Liu Yang","year":"2017","unstructured":"Yang Liu, Prajit Ramachandran, Qiang Liu, and Jian Peng. 2017. Stein variational policy gradient. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI\u201917), Gal Elidan, Kristian Kersting, and Alexander T. Ihler (Eds.). AUAI Press."},{"key":"e_1_3_1_56_2","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.ins.2020.12.017","article-title":"Recruitment-imitation mechanism for evolutionary reinforcement learning","volume":"553","author":"L\u00fc Shuai","year":"2021","unstructured":"Shuai L\u00fc, Shuai Han, Wenbo Zhou, and Junwei Zhang. 2021. Recruitment-imitation mechanism for evolutionary reinforcement learning. Inf. Sci. 553 (2021), 172\u2013188.","journal-title":"Inf. Sci."},{"key":"e_1_3_1_57_2","first-page":"854","volume-title":"Proceedings of the Conference on Robot Learning","author":"Luck Kevin Sebastian","year":"2020","unstructured":"Kevin Sebastian Luck, Heni Ben Amor, and Roberto Calandra. 2020. Data-efficient co-adaptation of morphology and behaviour with deep reinforcement learning. In Proceedings of the Conference on Robot Learning. PMLR, 854\u2013869."},{"key":"e_1_3_1_58_2","unstructured":"Yan Ma Tianxing Liu Bingsheng Wei Yi Liu Kang Xu and Wei Li. 2022. Evolutionary action selection for gradient-based policy learning. arXiv:2201.04286. Retrieved from https:\/\/arxiv.org\/abs\/2201.04286."},{"key":"e_1_3_1_59_2","series-title":"Proceedings of the 36th International Conference on Machine Learning (ICML\u201919),","first-page":"4264","volume":"97","author":"Maheswaranathan Niru","year":"2019","unstructured":"Niru Maheswaranathan, Luke Metz, George Tucker, Dami Choi, and Jascha Sohl-Dickstein. 2019. Guided evolutionary strategies: Augmenting random search with surrogate gradients. In Proceedings of the 36th International Conference on Machine Learning (ICML\u201919),Proceedings of Machine Learning Research, Vol. 97, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 4264\u20134273."},{"key":"e_1_3_1_60_2","unstructured":"Amjad Yousef Majid Serge Saaybi Tomas van Rietbergen Vincent Francois-Lavet R. Venkatesha Prasad and Chris Verhoeven. 2021. Deep reinforcement learning versus evolution strategies: A comparative survey. arXiv:\/2110.01411. Retrieved from https:\/\/arxiv.org\/abs\/2110.01411."},{"key":"e_1_3_1_61_2","first-page":"1805","volume-title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918)","author":"Mania Horia","year":"2018","unstructured":"Horia Mania, Aurelia Guy, and Benjamin Recht. 2018. Simple random search of static linear policies is competitive for reinforcement learning. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS\u201918), Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 1805\u20131814."},{"key":"e_1_3_1_62_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921)","author":"Marchesini Enrico","year":"2021","unstructured":"Enrico Marchesini, Davide Corsi, and Alessandro Farinelli. 2021. Genetic soft updates for policy evolution in deep reinforcement learning. In Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921). OpenReview.net."},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/1570256.1570428"},{"key":"e_1_3_1_64_2","first-page":"1928","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1928\u20131937."},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2020.101731"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449639.3459304"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.3390\/sym13030471"},{"key":"e_1_3_1_69_2","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Parker-Holder Jack","year":"2020","unstructured":"Jack Parker-Holder, Aldo Pacchiano, Krzysztof Marcin Choromanski, and Stephen J. Roberts. 2020. Effective diversity in population based reinforcement learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.)."},{"key":"e_1_3_1_70_2","doi-asserted-by":"crossref","unstructured":"Jack Parker-Holder Raghu Rajan Xingyou Song Andr\u00e9 Biedenkapp Yingjie Miao Theresa Eimer Baohe Zhang Vu Nguyen Roberto Calandra Aleksandra Faust et\u00a0al. 2022. Automated reinforcement learning (AutoRL): A survey and open problems. arXiv:2201.03916. Retrieved from https:\/\/arxiv.org\/abs\/2201.03916.","DOI":"10.1613\/jair.1.13596"},{"key":"e_1_3_1_71_2","unstructured":"Xue Bin Peng Aviral Kumar Grace Zhang and Sergey Levine. 2019. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv:1910.00177. Retrieved from https:\/\/arxiv.org\/abs\/1910.00177."},{"key":"e_1_3_1_72_2","unstructured":"Thomas Pierrot Valentin Mac\u00e9 Geoffrey Cideron Nicolas Perrin Karim Beguir and Olivier Sigaud. 2020. Sample efficient quality diversity for neural continuous control (unpublished)."},{"key":"e_1_3_1_73_2","unstructured":"Thomas Pierrot Nicolas Perrin and Olivier Sigaud. 2018. First-order and second-order variants of the gradient descent in a unified framework. arXiv:1810.08102. Retrieved from https:\/\/arxiv.org\/abs\/1810.08102."},{"key":"e_1_3_1_74_2","unstructured":"Cristina Pinneri Shambhuraj Sawant Sebastian Blaes Jan Achterhold Joerg Stueckler Michal Rolinek and Georg Martius. 2020. Sample-efficient cross-entropy method for real-time planning. arXiv:2008.06389. Retrieved from https:\/\/arxiv.org\/abs\/2008.06389."},{"key":"e_1_3_1_75_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921)","author":"Pinneri Cristina","year":"2021","unstructured":"Cristina Pinneri, Shambhuraj Sawant, Sebastian Blaes, and Georg Martius. 2021. Extracting strong policies for robotics tasks from zero-order trajectory optimizers. In Proceedings of the 9th International Conference on Learning Representations (ICLR\u201921). OpenReview.net."},{"key":"e_1_3_1_76_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)","author":"Pourchot Alo\u00efs","year":"2019","unstructured":"Alo\u00efs Pourchot and Olivier Sigaud. 2019. CEM-RL: Combining evolutionary and gradient-based methods for policy search. In Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919). OpenReview.net."},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2016.00040"},{"key":"e_1_3_1_78_2","doi-asserted-by":"crossref","unstructured":"Hong Qian and Yang Yu. 2021. Derivative-free reinforcement learning: A review. arXiv:2102.05710. Retrieved from https:\/\/arxiv.org\/abs\/2102.05710.","DOI":"10.1007\/s11704-020-0241-4"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010091220143"},{"key":"e_1_3_1_80_2","unstructured":"Tim Salimans Jonathan Ho Xi Chen Szymon Sidor and Ilya Sutskever. 2017. Evolution strategies as a scalable alternative to reinforcement learning. arXiv:1703.03864. Retrieved from https:\/\/arxiv.org\/abs\/1703.03864."},{"key":"e_1_3_1_81_2","series-title":"Proceedings of the 32nd International Conference on Machine Learning (ICML\u201915),","first-page":"1889","volume":"37","author":"Schulman John","year":"2015","unstructured":"John Schulman, Sergey Levine, Pieter Abbeel, Michael I. Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In Proceedings of the 32nd International Conference on Machine Learning (ICML\u201915),JMLR Workshop and Conference Proceedings, Vol. 37, Francis R. Bach and David M. Blei (Eds.). JMLR.org, 1889\u20131897."},{"key":"e_1_3_1_82_2","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347. Retrieved from https:\/\/arxiv.org\/abs\/1707.06347."},{"key":"e_1_3_1_83_2","first-page":"596","volume-title":"Proceedings of the 3rd IEEE International Conference on Robotic Computing (IRC\u201919)","author":"Sehgal Adarsh","year":"2019","unstructured":"Adarsh Sehgal, Hung La, Sushil Louis, and Hai Nguyen. 2019. Deep reinforcement learning using genetic algorithm for parameter optimization. In Proceedings of the 3rd IEEE International Conference on Robotic Computing (IRC\u201919). IEEE, 596\u2013601."},{"key":"e_1_3_1_84_2","doi-asserted-by":"crossref","unstructured":"Adarsh Sehgal Nicholas Ward Hung Manh La Christos Papachristos and Sushil Louis. 2022. GA-DRL: Genetic algorithm-based function optimizer in deep reinforcement learning for robotic manipulation tasks. arXiv:2203.00141. Retrieved from https:\/\/arxiv.org\/abs\/2203.00141.","DOI":"10.1109\/IRC55401.2022.00022"},{"key":"e_1_3_1_85_2","first-page":"267","volume-title":"Proceedings of the Conference on Robot Learning","author":"Shao Lin","year":"2021","unstructured":"Lin Shao, Yifan You, Mengyuan Yan, Shenli Yuan, Qingyun Sun, and Jeannette Bohg. 2021. GRAC: Self-guided and self-regularized actor-critic. In Proceedings of the Conference on Robot Learning. PMLR, 267\u2013276."},{"key":"e_1_3_1_86_2","unstructured":"Longxiang Shi Shijian Li Longbing Cao Long Yang Gang Zheng and Gang Pan. 2019. FiDi-RL: Incorporating deep reinforcement learning with finite-difference policy search for efficient learning of continuous control. arXiv:1907.00526. Retrieved from https:\/\/arxiv.org\/abs\/1907.00526."},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3008735"},{"key":"e_1_3_1_88_2","unstructured":"Zhenyang Shi and Surya P. N. Singh. 2021. Soft actor-critic with cross-entropy policy optimization. arXiv:2112.11115. Retrieved from https:\/\/arxiv.org\/abs\/2112.11115."},{"key":"e_1_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.01.011"},{"issue":"11","key":"e_1_3_1_90_2","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1007\/s00500-007-0164-0","article-title":"Learning classifier systems: A survey","volume":"11","author":"Sigaud Olivier","year":"2007","unstructured":"Olivier Sigaud and S. W. Wilson. 2007. Learning classifier systems: A survey. J. Soft Comput. 11, 11 (2007), 1065\u20131078.","journal-title":"J. Soft Comput."},{"key":"e_1_3_1_91_2","unstructured":"Riley Simmons-Edler Ben Eisner Eric Mitchell Sebastian Seung and Daniel Lee. 2019. Q-learning for continuous actions with cross-entropy guided policies. arXiv:1903.10605. Retrieved from https:\/\/arxiv.org\/abs\/1903.10605."},{"key":"e_1_3_1_92_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1558-5646.1953.tb00069.x"},{"key":"e_1_3_1_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449726.3463171"},{"key":"e_1_3_1_94_2","unstructured":"Hao Sun Ziping Xu Yuhang Song Meng Fang Jiechao Xiong Bo Dai and Bolei Zhou. 2020. Zeroth-order supervised policy improvement. arXiv:2006.06600. Retrieved from https:\/\/arxiv.org\/abs\/2006.06600."},{"key":"e_1_3_1_95_2","unstructured":"Karush Suri Xiao Qi Shi Konstantinos N. Plataniotis and Yuri A. Lawryshyn. 2020. Maximum mutation reinforcement learning for scalable control. arXiv:2007.13690. Retrieved from https:\/\/arxiv.org\/abs\/2007.13690."},{"key":"e_1_3_1_96_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.5555\/3463952.3464104"},{"key":"e_1_3_1_98_2","unstructured":"Yunhao Tang and Krzysztof Choromanski. 2020. Online hyper-parameter tuning in off-policy learning via evolutionary strategies. arXiv:2006.07554. Retrieved from https:\/\/arxiv.org\/abs\/2006.07554."},{"key":"e_1_3_1_99_2","unstructured":"Rohan Tangri Danilo P. Mandic and Anthony G. Constantinides. 2022. Pearl: Parallel evolutionary and reinforcement learning library. arXiv:2201.09568. Retrieved from https:\/\/arxiv.org\/abs\/2201.09568."},{"key":"e_1_3_1_100_2","doi-asserted-by":"crossref","unstructured":"Bryon Tjanaka Matthew C. Fontaine Julian Togelius and Stefanos Nikolaidis. 2022. Approximating gradients for differentiable quality diversity in reinforcement learning. arXiv:2202.03666. Retrieved from https:\/\/arxiv.org\/abs\/2202.03666.","DOI":"10.1145\/3512290.3528705"},{"key":"e_1_3_1_101_2","first-page":"761","volume-title":"Artificial Life Conference Proceedings","author":"Todd Graham","year":"2020","unstructured":"Graham Todd, Madhavun Candadai, and Eduardo J. Izquierdo. 2020. Interaction between evolution and learning in nk fitness landscapes. In Artificial Life Conference Proceedings. MIT Press, 761\u2013767."},{"key":"e_1_3_1_102_2","article-title":"Exploring model-based planning with policy networks","author":"Wang Tingwu","year":"2019","unstructured":"Tingwu Wang and Jimmy Ba. 2019. Exploring model-based planning with policy networks. arXiv:1906.08649. Retrieved from https:\/\/arxiv.org\/abs\/1906.08649.","journal-title":"arXiv:1906.08649"},{"key":"e_1_3_1_103_2","doi-asserted-by":"crossref","unstructured":"Yuxing Wang Tiantian Zhang Yongzhe Chang Bin Liang Xueqian Wang and Bo Yuan. 2022. A surrogate-assisted controller for expensive evolutionary reinforcement learning. arXiv:2201.00129. Retrieved from https:\/\/arxiv.org\/abs\/2201.00129.","DOI":"10.1016\/j.ins.2022.10.134"},{"key":"e_1_3_1_104_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917)","author":"Wang Ziyu","year":"2017","unstructured":"Ziyu Wang, Victor Bapst, Nicolas Heess, Volodymyr Mnih, R\u00e9mi Munos, Koray Kavukcuoglu, and Nando de Freitas. 2017. Sample efficient actor-critic with experience replay. In Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917). OpenReview.net."},{"key":"e_1_3_1_105_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"e_1_3_1_106_2","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/2871.001.0001","volume-title":"Evolution and Learning: The Baldwin Effect Reconsidered","author":"Weber Bruce H.","year":"2003","unstructured":"Bruce H. Weber and David J. Depew. 2003. Evolution and Learning: The Baldwin Effect Reconsidered. MIT Press."},{"key":"e_1_3_1_107_2","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2638566"},{"key":"e_1_3_1_108_2","unstructured":"Jiaxing Zhang Hoang Tran and Guannan Zhang. 2020. Accelerating reinforcement learning with a directional-gaussian-smoothing evolution strategy. arXiv:2002.09077. Retrieved from https:\/\/arxiv.org\/abs\/2002.09077."},{"key":"e_1_3_1_109_2","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Zheng Han","year":"2020","unstructured":"Han Zheng, Pengfei Wei, Jing Jiang, Guodong Long, Qinghua Lu, and Chengqi Zhang. 2020. Cooperative heterogeneous deep reinforcement learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.)."}],"container-title":["ACM Transactions on Evolutionary Learning and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569096","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3569096","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T13:51:41Z","timestamp":1750168301000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569096"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,20]]},"references-count":108,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,9,30]]}},"alternative-id":["10.1145\/3569096"],"URL":"https:\/\/doi.org\/10.1145\/3569096","relation":{},"ISSN":["2688-299X","2688-3007"],"issn-type":[{"value":"2688-299X","type":"print"},{"value":"2688-3007","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,20]]},"assertion":[{"value":"2022-06-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-18","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}