{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T02:09:03Z","timestamp":1772676543944,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":22,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T00:00:00Z","timestamp":1657238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Paris Ile-de-France region","award":["DIM RFSI AlgoSelect"],"award-info":[{"award-number":["DIM RFSI AlgoSelect"]}]},{"name":"Spanish Ministry of Science and Innovation (MICINN)","award":["BEAGAL 18\/0005"],"award-info":[{"award-number":["BEAGAL 18\/0005"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,8]]},"DOI":"10.1145\/3512290.3528799","type":"proceedings-article","created":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T13:59:57Z","timestamp":1658152797000},"page":"867-875","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Analyzing the impact of undersampling on the benchmarking and configuration of evolutionary algorithms"],"prefix":"10.1145","author":[{"given":"Diederick","family":"Vermetten","sequence":"first","affiliation":[{"name":"Leiden Institute for Advanced Computer Science, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Wang","sequence":"additional","affiliation":[{"name":"Leiden Institute for Advanced Computer Science, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manuel","family":"L\u00f3pez-Iba\u00f1ez","sequence":"additional","affiliation":[{"name":"University of M\u00e1laga, M\u00e1laga, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carola","family":"Doerr","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"B\u00e4ck","sequence":"additional","affiliation":[{"name":"Leiden Institute for Advanced Computer Science, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,7,8]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Manuel L\u00f3pez-Ib\u00e1\u00f1ez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise.","author":"Bartz-Beielstein Thomas","year":"2020","unstructured":"Thomas Bartz-Beielstein , Carola Doerr , Daan van den Berg , Jakob Bossek , Sowmya Chandrasekaran , Tome Eftimov , Andreas Fischbach , Pascal Kerschke , William La Cava , Manuel L\u00f3pez-Ib\u00e1\u00f1ez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise. 2020 . Benchmarking in Optimization : Best Practice and Open Issues. Arxiv preprint arXiv:2007.03488 [cs.NE] (2020). https:\/\/arxiv.org\/abs\/2007.03488 Thomas Bartz-Beielstein, Carola Doerr, Daan van den Berg, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, William La Cava, Manuel L\u00f3pez-Ib\u00e1\u00f1ez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise. 2020. Benchmarking in Optimization: Best Practice and Open Issues. Arxiv preprint arXiv:2007.03488 [cs.NE] (2020). https:\/\/arxiv.org\/abs\/2007.03488"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10732-020-09454-w"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.4524959"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449726.3463167"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.05281"},{"key":"e_1_3_2_1_6_1","volume-title":"The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results","author":"Ellis Paul D","unstructured":"Paul D Ellis . 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results . Cambridge university press . Paul D Ellis. 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge university press."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ITA50056.2020.9244948"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2020.1808977"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICEC.1996.542381"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2019.2921598"},{"key":"e_1_3_2_1_12_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Sanjoy Dasgupta and David McAllester (Eds.)","volume":"28","author":"Karnin Zohar","year":"2013","unstructured":"Zohar Karnin , Tomer Koren , and Oren Somekh . 2013 . Almost optimal exploration in multi-armed bandits . In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Sanjoy Dasgupta and David McAllester (Eds.) , Vol. 28 . 1238--1246. http:\/\/jmlr.org\/proceedings\/papers\/v28\/ Zohar Karnin, Tomer Koren, and Oren Somekh. 2013. Almost optimal exploration in multi-armed bandits. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Sanjoy Dasgupta and David McAllester (Eds.), Vol. 28. 1238--1246. http:\/\/jmlr.org\/proceedings\/papers\/v28\/"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00242"},{"key":"e_1_3_2_1_14_1","first-page":"1","article-title":"Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization","volume":"18","author":"Li Lisha","year":"2018","unstructured":"Lisha Li , Kevin Jamieson , Giulia DeSalvo , Afshin Rostamizadeh , and Ameet Talwalkar . 2018 . Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization . Journal of Machine Learning Research 18 , 185 (2018), 1 -- 52 . Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. Journal of Machine Learning Research 18, 185 (2018), 1--52.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3466624"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.orp.2016.09.002"},{"key":"e_1_3_2_1_17_1","first-page":"1","article-title":"The Racing Algorithm: Model Selection for Lazy Learners","volume":"11","author":"Maron O.","year":"1997","unstructured":"O. Maron and A. W. Moore . 1997 . The Racing Algorithm: Model Selection for Lazy Learners . Artificial Intelligence Research 11 , 1 -- 5 (1997), 193--225. O. Maron and A. W. Moore. 1997. The Racing Algorithm: Model Selection for Lazy Learners. Artificial Intelligence Research 11, 1--5 (1997), 193--225.","journal-title":"Artificial Intelligence Research"},{"key":"e_1_3_2_1_18_1","volume-title":"Nevergrad: A gradient-free optimization platform. https:\/\/GitHub.com\/FacebookResearch\/Nevergrad.","author":"Rapin J.","year":"2018","unstructured":"J. Rapin and O. Teytaud . 2018 . Nevergrad: A gradient-free optimization platform. https:\/\/GitHub.com\/FacebookResearch\/Nevergrad. J. Rapin and O. Teytaud. 2018. Nevergrad: A gradient-free optimization platform. https:\/\/GitHub.com\/FacebookResearch\/Nevergrad."},{"key":"e_1_3_2_1_19_1","unstructured":"Sander van Rijn. 2018. Modular CMA-ES framework from [20] v0.3.0. https:\/\/github.com\/sjvrijn\/ModEA. Available also as pypi package at https:\/\/pypi.org\/project\/ModEA\/0.3.0\/.  Sander van Rijn. 2018. Modular CMA-ES framework from [20] v0.3.0. https:\/\/github.com\/sjvrijn\/ModEA. Available also as pypi package at https:\/\/pypi.org\/project\/ModEA\/0.3.0\/."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSCI.2016.7850138"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377930.3389831"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.5925410"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.18857486.v1"}],"event":{"name":"GECCO '22: Genetic and Evolutionary Computation Conference","location":"Boston Massachusetts","acronym":"GECCO '22","sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"]},"container-title":["Proceedings of the Genetic and Evolutionary Computation Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512290.3528799","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3512290.3528799","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:30Z","timestamp":1750186830000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512290.3528799"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,8]]},"references-count":22,"alternative-id":["10.1145\/3512290.3528799","10.1145\/3512290"],"URL":"https:\/\/doi.org\/10.1145\/3512290.3528799","relation":{},"subject":[],"published":{"date-parts":[[2022,7,8]]},"assertion":[{"value":"2022-07-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}