{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:26:05Z","timestamp":1750220765312,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,6,25]],"date-time":"2020-06-25T00:00:00Z","timestamp":1593043200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Paris Ile de France region","award":["DIM RFSI AlgoSelect"],"award-info":[{"award-number":["DIM RFSI AlgoSelect"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,6,25]]},"DOI":"10.1145\/3377930.3389831","type":"proceedings-article","created":{"date-parts":[[2020,6,29]],"date-time":"2020-06-29T19:29:12Z","timestamp":1593458952000},"page":"903-912","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Integrated vs. sequential approaches for selecting and tuning CMA-ES variants"],"prefix":"10.1145","author":[{"given":"Diederick","family":"Vermetten","sequence":"first","affiliation":[{"name":"Leiden Institute for Advanced Computer Science, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Wang","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carola","family":"Doerr","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"B\u00e4ck","sequence":"additional","affiliation":[{"name":"Leiden Institute for Advanced Computer Science, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,6,26]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2015.7257124"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001576.2001694"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2005.1554902"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/11740698_26"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40137-4_4"},{"key":"e_1_3_2_1_6_1","volume-title":"SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization. CoRR abs\/1006.4645","author":"Bartz-Beielstein Thomas","year":"2010","unstructured":"Thomas Bartz-Beielstein . 2010 . SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization. CoRR abs\/1006.4645 (2010). arXiv:1006.4645 http:\/\/arxiv.org\/abs\/1006.4645 Thomas Bartz-Beielstein. 2010. SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization. CoRR abs\/1006.4645 (2010). arXiv:1006.4645 http:\/\/arxiv.org\/abs\/1006.4645"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3071178.3071343"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15844-5_2"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Leslie P\u00e9rez C\u00e1ceres Manuel L\u00f3pez-Ib\u00e1\u00f1ez Holger Hoos and Thomas St\u00fctzle. 2017. An Experimental Study of Adaptive Capping in irace. In LION Roberto Battiti Dmitri E. Kvasov and Yaroslav D. Sergeyev (Eds.). 235--250.  Leslie P\u00e9rez C\u00e1ceres Manuel L\u00f3pez-Ib\u00e1\u00f1ez Holger Hoos and Thomas St\u00fctzle. 2017. An Experimental Study of Adaptive Capping in irace. In LION Roberto Battiti Dmitri E. Kvasov and Yaroslav D. Sergeyev (Eds.). 235--250.","DOI":"10.1007\/978-3-319-69404-7_17"},{"key":"e_1_3_2_1_10_1","volume-title":"IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. arXiv e-prints:1810.05281 (Oct","author":"Doerr Carola","year":"2018","unstructured":"Carola Doerr , Hao Wang , Furong Ye , Sander van Rijn , and Thomas B\u00e4ck . 2018. IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. arXiv e-prints:1810.05281 (Oct . 2018 ). arXiv:1810.05281 https:\/\/arxiv.org\/abs\/1810.05281 Carola Doerr, Hao Wang, Furong Ye, Sander van Rijn, and Thomas B\u00e4ck. 2018. IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. arXiv e-prints:1810.05281 (Oct. 2018). arXiv:1810.05281 https:\/\/arxiv.org\/abs\/1810.05281"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.l.11420"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.771166"},{"key":"e_1_3_2_1_13_1","volume-title":"Proc. of International Conference on Machine Learning (ICML'18)","author":"Falkner Stefan","year":"2018","unstructured":"Stefan Falkner , Aaron Klein , and Frank Hutter . 2018 . BOHB: Robust and Efficient Hyperparameter Optimization at Scale . In Proc. of International Conference on Machine Learning (ICML'18) . 1436--1445. Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and Efficient Hyperparameter Optimization at Scale. In Proc. of International Conference on Machine Learning (ICML'18). 1436--1445."},{"key":"e_1_3_2_1_14_1","unstructured":"Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in neural information processing systems. 2962--2970.  Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in neural information processing systems. 2962--2970."},{"key":"e_1_3_2_1_15_1","volume-title":"CMA-ES with Two-Point Step-Size Adaptation. arXiv:0805.0231 [cs] (May","author":"Hansen Nikolaus","year":"2008","unstructured":"Nikolaus Hansen . 2008. CMA-ES with Two-Point Step-Size Adaptation. arXiv:0805.0231 [cs] (May 2008 ). http:\/\/arxiv.org\/abs\/0805.0231 arXiv: 0805.0231. Nikolaus Hansen. 2008. CMA-ES with Two-Point Step-Size Adaptation. arXiv:0805.0231 [cs] (May 2008). http:\/\/arxiv.org\/abs\/0805.0231 arXiv: 0805.0231."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1570256.1570333"},{"key":"e_1_3_2_1_17_1","volume-title":"COCO: Performance Assessment. arXiv:1605.03560 [cs] (May","author":"Hansen Nikolaus","year":"2016","unstructured":"Nikolaus Hansen , Anne Auger , Dimo Brockhoff , Dejan Tu\u0161ar , and Tea Tu\u0161ar . 2016 . COCO: Performance Assessment. arXiv:1605.03560 [cs] (May 2016). http:\/\/arxiv.org\/abs\/1605.03560 arXiv: 1605.03560. Nikolaus Hansen, Anne Auger, Dimo Brockhoff, Dejan Tu\u0161ar, and Tea Tu\u0161ar. 2016. COCO: Performance Assessment. arXiv:1605.03560 [cs] (May 2016). http:\/\/arxiv.org\/abs\/1605.03560 arXiv: 1605.03560."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1830761.1830790"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICEC.1996.542381"},{"volume-title":"Sequential model-based optimization for general algorithm configuration","author":"Hutter Frank","key":"e_1_3_2_1_20_1","unstructured":"Frank Hutter , Holger H Hoos , and Kevin Leyton-Brown . 2011. Sequential model-based optimization for general algorithm configuration . In LION. Springer , 507--523. Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2011. Sequential model-based optimization for general algorithm configuration. In LION. Springer, 507--523."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319619.3326905"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2006.1688662"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00242"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2016.7748359"},{"key":"e_1_3_2_1_25_1","volume-title":"Graham","author":"Kleinberg Robert","year":"2019","unstructured":"Robert Kleinberg , Kevin Leyton-Brown , Brendan Lucier , and Devon R . Graham . 2019 . Procrastinating with Confidence : Near-Optimal, Anytime, Adaptive Algorithm Configuration. In Proc. of Neural Information Processing Systems (NeurPS '19). 8881--8891. Robert Kleinberg, Kevin Leyton-Brown, Brendan Lucier, and Devon R. Graham. 2019. Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration. In Proc. of Neural Information Processing Systems (NeurPS'19). 8881--8891."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3122034"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99259-4_1"},{"key":"e_1_3_2_1_28_1","volume-title":"Hyperband: A novel bandit-based approach to hyperparameter optimization. arXiv preprint arXiv:1603.06560","author":"Li Lisha","year":"2016","unstructured":"Lisha Li , Kevin Jamieson , Giulia DeSalvo , Afshin Rostamizadeh , and Ameet Talwalkar . 2016 . Hyperband: A novel bandit-based approach to hyperparameter optimization. arXiv preprint arXiv:1603.06560 (2016). Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2016. Hyperband: A novel bandit-based approach to hyperparameter optimization. arXiv preprint arXiv:1603.06560 (2016)."},{"key":"e_1_3_2_1_29_1","volume-title":"Studies in Computational Intelligence","volume":"54","author":"Lobo Fernando G.","year":"2007","unstructured":"Fernando G. Lobo , Cl\u00e1udio F. Lima , and Zbigniew Michalewicz ( Eds .). 2007 . Parameter Setting in Evolutionary Algorithms . Studies in Computational Intelligence , Vol. 54 . Springer. Fernando G. Lobo, Cl\u00e1udio F. Lima, and Zbigniew Michalewicz (Eds.). 2007. Parameter Setting in Evolutionary Algorithms. Studies in Computational Intelligence, Vol. 54. Springer."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.orp.2016.09.002"},{"key":"e_1_3_2_1_32_1","unstructured":"Manuel L\u00f3pez-Ib\u00e1\u00f1ez and Leslie P\u00e9rez C\u00e1ceres. [n. d.]. The irace Package: Iterated Race for Automatic Algorithm Configuration. http:\/\/iridia.ulb.ac.be\/irace\/..  Manuel L\u00f3pez-Ib\u00e1\u00f1ez and Leslie P\u00e9rez C\u00e1ceres. [n. d.]. The irace Package: Iterated Race for Automatic Algorithm Configuration. http:\/\/iridia.ulb.ac.be\/irace\/.."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001576.2001690"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2015.7257143"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3321707.3321861"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319619.3326913"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0065-2458(08)60520-3"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"crossref","unstructured":"Chris Thornton Frank Hutter Holger H Hoos and Kevin Leyton-Brown. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In ACM SIGKDD. ACM 847--855.  Chris Thornton Frank Hutter Holger H Hoos and Kevin Leyton-Brown. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In ACM SIGKDD. ACM 847--855.","DOI":"10.1145\/2487575.2487629"},{"key":"e_1_3_2_1_39_1","unstructured":"Sander van Rijn. 2018. Modular CMA-ES framework from [41] v0.3.0. https:\/\/github.com\/sjvrijn\/ModEA. Pypi package: https:\/\/pypi.org\/project\/ModEA\/0.3.0\/.  Sander van Rijn. 2018. Modular CMA-ES framework from [41] v0.3.0. https:\/\/github.com\/sjvrijn\/ModEA. Pypi package: https:\/\/pypi.org\/project\/ModEA\/0.3.0\/."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99253-2_5"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSCI.2016.7850138"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3071178.3071205"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Diederick Vermetten Sander van Rijn Thomas B\u00e4ck and Carola Doerr. 2019. Github repository for Online Selection of CMA-ES Variants. (2019). https:\/\/github.com\/Dvermetten\/Online_CMA-ES_Selection.  Diederick Vermetten Sander van Rijn Thomas B\u00e4ck and Carola Doerr. 2019. Github repository for Online Selection of CMA-ES Variants. (2019). https:\/\/github.com\/Dvermetten\/Online_CMA-ES_Selection.","DOI":"10.1145\/3321707.3321803"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3321707.3321803"},{"key":"e_1_3_2_1_45_1","unstructured":"Diederick Vermetten Hao Wang Thomas B\u00e4ck and Carola Doerr. 2020. Github repository with the data used in this paper. https:\/\/github.com\/Dvermetten\/CMA_ES_hyperparam.  Diederick Vermetten Hao Wang Thomas B\u00e4ck and Carola Doerr. 2020. Github repository with the data used in this paper. https:\/\/github.com\/Dvermetten\/CMA_ES_hyperparam."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2554850.2555089"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2018.8477956"},{"volume-title":"A new acquisition function for Bayesian optimization based on the moment-generating function","author":"Wang Hao","key":"e_1_3_2_1_48_1","unstructured":"Hao Wang , Bas van Stein , Michael Emmerich , and Thomas B\u00e4ck . 2017. A new acquisition function for Bayesian optimization based on the moment-generating function . In SMC. IEEE , 507--512. Hao Wang, Bas van Stein, Michael Emmerich, and Thomas B\u00e4ck. 2017. A new acquisition function for Bayesian optimization based on the moment-generating function. In SMC. IEEE, 507--512."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.585893"},{"key":"e_1_3_2_1_50_1","volume-title":"Twenty-Fourth AAAI Conference on Artificial Intelligence.","author":"Xu Lin","year":"2010","unstructured":"Lin Xu , Holger Hoos , and Kevin Leyton-Brown . 2010 . Hydra: Automatically configuring algorithms for portfolio-based selection . In Twenty-Fourth AAAI Conference on Artificial Intelligence. Lin Xu, Holger Hoos, and Kevin Leyton-Brown. 2010. Hydra: Automatically configuring algorithms for portfolio-based selection. In Twenty-Fourth AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_2_1_51_1","volume-title":"RCRA workshop on experimental evaluation of algorithms for solving problems with combinatorial explosion at the international joint conference on artificial intelligence (IJCAI).","author":"Xu Lin","year":"2011","unstructured":"Lin Xu , Frank Hutter , Holger H Hoos , and Kevin Leyton-Brown . 2011 . Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming . In RCRA workshop on experimental evaluation of algorithms for solving problems with combinatorial explosion at the international joint conference on artificial intelligence (IJCAI). Lin Xu, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2011. Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming. In RCRA workshop on experimental evaluation of algorithms for solving problems with combinatorial explosion at the international joint conference on artificial intelligence (IJCAI)."}],"event":{"name":"GECCO '20: Genetic and Evolutionary Computation Conference","sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"],"location":"Canc\u00fan Mexico","acronym":"GECCO '20"},"container-title":["Proceedings of the 2020 Genetic and Evolutionary Computation Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3377930.3389831","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3377930.3389831","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:07Z","timestamp":1750200067000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3377930.3389831"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,25]]},"references-count":50,"alternative-id":["10.1145\/3377930.3389831","10.1145\/3377930"],"URL":"https:\/\/doi.org\/10.1145\/3377930.3389831","relation":{},"subject":[],"published":{"date-parts":[[2020,6,25]]},"assertion":[{"value":"2020-06-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}