{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T06:12:07Z","timestamp":1779343927109,"version":"3.51.4"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T00:00:00Z","timestamp":1722211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Sorbonne Center for Artificial Intelligence (SCAI) of Sorbonne University","award":["IDEX SUPER 11-IDEX-0004"],"award-info":[{"award-number":["IDEX SUPER 11-IDEX-0004"]}]},{"name":"ANR T-ERC project VARIATION","award":["ANR-22-ERCS-0003-01"],"award-info":[{"award-number":["ANR-22-ERCS-0003-01"]}]},{"name":"PRIME programme of the German Academic Exchange Service (DAAD) with funds from the German Federal Ministry of Education and Research"},{"name":"European Union\u2013NextGenerationEU under the Italian Ministry of University and Research (MUR) National Innovation Ecosystem","award":["ECS00000041\u2013VITALITY\u2013CUP J13C22000430001"],"award-info":[{"award-number":["ECS00000041\u2013VITALITY\u2013CUP J13C22000430001"]}]},{"name":"INdAM\u2013GNCS Project 2023","award":["CUP\u2013E53C22001930001"],"award-info":[{"award-number":["CUP\u2013E53C22001930001"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Evol. Learn. Optim."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Bayesian Optimization (BO) is a class of surrogate-based black-box optimization heuristics designed to efficiently locate high-quality solutions for problems that are expensive to evaluate, and therefore allow only small evaluation budgets. BO is particularly popular for solving numerical optimization problems in industry, where the evaluation of objective functions often relies on time-consuming simulations or physical experiments. However, many industrial problems depend on a large number of parameters. This poses a challenge for BO algorithms, whose performance is often reported to suffer when the dimension grows beyond 15 decision variables. Although many new algorithms have been proposed to address this, it remains unclear which one is best suited for a specific optimization problem. In this work, we compare five state-of-the-art high-dimensional BO algorithms with vanilla BO, CMA-ES, and random search on the 24 BBOB functions of the COCO environment at increasing dimensionality, ranging from 10 to 60 variables. Our results confirm the superiority of BO over CMA-ES for limited evaluation budgets and suggest that the most promising approach to improve BO is the use of trust regions. However, we also observe significant performance differences for different function landscapes and budget exploitation phases, indicating improvement potential, e.g., through hybridization of algorithmic components.<\/jats:p>","DOI":"10.1145\/3670683","type":"journal-article","created":{"date-parts":[[2024,6,15]],"date-time":"2024-06-15T13:03:38Z","timestamp":1718456618000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7389-2555","authenticated-orcid":false,"given":"Maria Laura","family":"Santoni","sequence":"first","affiliation":[{"name":"Sorbonne Universit\u00e9, CNRS, LIP6, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6841-7409","authenticated-orcid":false,"given":"Elena","family":"Raponi","sequence":"additional","affiliation":[{"name":"Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9783-608X","authenticated-orcid":false,"given":"Renato De","family":"Leone","sequence":"additional","affiliation":[{"name":"School of Science and Technology, University of Camerino, Camerino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4981-3227","authenticated-orcid":false,"given":"Carola","family":"Doerr","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, CNRS, LIP6, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,29]]},"reference":[{"key":"e_1_3_4_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-14714-2_9"},{"key":"e_1_3_4_3_1","doi-asserted-by":"publisher","unstructured":"R. Bellman. 1966. Dynamic Programming. Science 153 3731 (July 1966) 34\u201337. DOI: 10.1126\/science.153.3731.34","DOI":"10.1126\/science.153.3731.34"},{"key":"e_1_3_4_4_1","volume-title":"Reproducing Kernel Hilbert Spaces in Probability and Statistics","author":"Berlinet Alain","year":"2011","unstructured":"Alain Berlinet and Christine Thomas-Agnan. 2011. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer Science & Business Media."},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3545611"},{"key":"e_1_3_4_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10472-015-9463-9"},{"key":"e_1_3_4_7_1","unstructured":"Jingfan Chen Guanghui Zhu Rong Gu Chunfeng Yuan and Yihua Huang. 2020. Semi-Supervised Embedding Learning for High-Dimensional Bayesian Optimization. arXiv:2005.14601. Retrieved from https:\/\/arxiv.org\/abs\/2005.14601"},{"key":"e_1_3_4_8_1","doi-asserted-by":"publisher","unstructured":"Alexander I. Cowen-Rivers Wenlong Lyu Rasul Tutunov Zhi Wang Antoine Grosnit Ryan Rhys Griffiths Alexandre Max Maraval Hao Jianye Jun Wang Jan Peters and Haitham Bou Ammar. 2022. HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation. arXiv:2012.03826. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2012.03826","DOI":"10.48550\/arXiv.2012.03826"},{"key":"e_1_3_4_9_1","first-page":"2453","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Delbridge Ian","year":"2020","unstructured":"Ian Delbridge, David Bindel, and Andrew Gordon Wilson. 2020. Randomly Projected Additive Gaussian Processes for Regression. In Proceedings of the International Conference on Machine Learning. PMLR, 2453\u20132463."},{"key":"e_1_3_4_10_1","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1032280214"},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-022-01245-w"},{"key":"e_1_3_4_12_1","unstructured":"Carola Doerr Hao Wang Furong Ye Sander van Rijn and Thomas B\u00e4ck. 2018. IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. arXiv:1810.05281. Retrieved from https:\/\/iohprofiler.github.io\/"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","unstructured":"Nicolas Durrande David Ginsbourger and Olivier Roustant. 2011. Additive Kernels for Gaussian Process Modeling. arXiv:1103.4023. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1103.4023","DOI":"10.48550\/arXiv.1103.4023"},{"key":"e_1_3_4_14_1","first-page":"493","volume-title":"Proceedings of the Uncertainty in Artificial Intelligence","author":"Eriksson David","year":"2021","unstructured":"David Eriksson and Martin Jankowiak. 2021. High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces. In Proceedings of the Uncertainty in Artificial Intelligence. PMLR, 493\u2013503."},{"key":"e_1_3_4_15_1","first-page":"5496","article-title":"Scalable Global Optimization via Local Bayesian Optimization","volume":"32","author":"Eriksson David","year":"2019","unstructured":"David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias Poloczek. 2019. Scalable Global Optimization via Local Bayesian Optimization. Advances in Neural Information Processing Systems 32, 5496\u20135507.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/9780470770801"},{"key":"e_1_3_4_17_1","doi-asserted-by":"publisher","unstructured":"Peter I. Frazier. 2018. A Tutorial on Bayesian Optimization. arXiv:1807.02811. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1807.02811","DOI":"10.48550\/arXiv.1807.02811"},{"key":"e_1_3_4_18_1","doi-asserted-by":"publisher","DOI":"10.1017\/9781108348973"},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00158-019-02458-6"},{"key":"e_1_3_4_20_1","doi-asserted-by":"publisher","DOI":"10.1039\/C9SC04026A"},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-32494-1_4"},{"key":"e_1_3_4_22_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.2559634"},{"key":"e_1_3_4_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2020.1808977"},{"key":"e_1_3_4_24_1","unstructured":"Nikolaus Hansen Steffen Finck Raymond Ros and Anne Auger. 2009. Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Technical Report RR-6829. INRIA. Retrieved from https:\/\/hal.inria.fr\/inria-00362633\/document"},{"key":"e_1_3_4_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30217-9_29"},{"key":"e_1_3_4_26_1","doi-asserted-by":"publisher","DOI":"10.1162\/106365603321828970"},{"key":"e_1_3_4_27_1","doi-asserted-by":"publisher","DOI":"10.1162\/106365601750190398"},{"key":"e_1_3_4_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2649-2"},{"key":"e_1_3_4_29_1","unstructured":"Tim Head M. Kumar H. Nahrstaedt G. Louppe and I. Shcherbatyi. 2017. scikit-optimize: Sequential Model-Based Optimization in Python. Retrieved from https:\/\/scikit-optimize.github.io\/stable\/auto_examples\/bayesian-optimization.html"},{"key":"e_1_3_4_30_1","first-page":"1470","volume-title":"NIPS Workshop on Bayesian Optimization, Proceedings of the 34th International Conference on Machine Learning - Volume 70","author":"Hern\u00e1ndez-Lobato Jos\u00e9 Miguel","year":"2016","unstructured":"Jos\u00e9 Miguel Hern\u00e1ndez-Lobato, Edward Pyzer-Knapp, Alan Aspuru-Guzik, and Ryan P. Adams. 2016. Distributed Thompson Sampling for Large-Scale Accelerated Exploration of Chemical Space. In NIPS Workshop on Bayesian Optimization, Proceedings of the 34th International Conference on Machine Learning - Volume 70, 1470\u20131479."},{"key":"e_1_3_4_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305533"},{"key":"e_1_3_4_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2464576.2501592"},{"key":"e_1_3_4_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"e_1_3_4_34_1","doi-asserted-by":"publisher","unstructured":"Haifeng Jin Qingquan Song and Xia Hu. 2019. Auto-Keras: An Efficient Neural Architecture Search System. arXiv:1806.10282. DOI: 10.48550\/arXiv.1806.10282","DOI":"10.48550\/arXiv.1806.10282"},{"key":"e_1_3_4_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2965418"},{"key":"e_1_3_4_36_1","doi-asserted-by":"publisher","unstructured":"Aaron Klein Stefan Falkner Simon Bartels Philipp Hennig and Frank Hutter. 2017. Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets. arXiv:1605.07079. DOI: 10.48550\/arXiv.1605.07079","DOI":"10.48550\/arXiv.1605.07079"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","unstructured":"Lars Kotthoff Hud Wahab and Patrick Johnson. 2021. Bayesian Optimization in Materials Science: A Survey. arXiv:2108.00002. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2108.00002","DOI":"10.48550\/arXiv.2108.00002"},{"key":"e_1_3_4_38_1","doi-asserted-by":"publisher","DOI":"10.2514\/6.2018-1656"},{"key":"e_1_3_4_39_1","first-page":"1546","article-title":"Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization","volume":"33","author":"Letham Ben","year":"2020","unstructured":"Ben Letham, Roberto Calandra, Akshara Rai, and Eytan Bakshy. 2020a. Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization. Advances in Neural Information Processing Systems 33, 1546\u20131558.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_40_1","doi-asserted-by":"publisher","unstructured":"Benjamin Letham Roberto Calandra Akshara Rai and Eytan Bakshy. 2020b. Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization. arXiv:2001.11659. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2001.11659","DOI":"10.48550\/arXiv.2001.11659"},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512290.3528712"},{"key":"e_1_3_4_42_1","first-page":"2900","volume-title":"Advances in Neural Information Processing Systems","author":"Malkomes Gustavo","year":"2016","unstructured":"Gustavo Malkomes, Charles Schaff, and Roman Garnett. 2016. Bayesian Optimization for Automated Model Selection. In Advances in Neural Information Processing Systems, Vol. 29. Curran Associates, Inc, 2900\u20132908."},{"key":"e_1_3_4_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISA52424.2021.9555522"},{"key":"e_1_3_4_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2008.03.026"},{"key":"e_1_3_4_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-009-9130-2"},{"key":"e_1_3_4_46_1","volume-title":"Bayesian Approach to Global Optimization: Theory and Applications","author":"Mockus Jonas","year":"2012","unstructured":"Jonas Mockus. 2012. Bayesian Approach to Global Optimization: Theory and Applications, Vol. 37. Springer Science & Business Media."},{"key":"e_1_3_4_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/AIKE.2019.00060"},{"key":"e_1_3_4_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/b98874"},{"key":"e_1_3_4_49_1","first-page":"11586","article-title":"Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces","volume":"35","author":"Papenmeier Leonard","year":"2022","unstructured":"Leonard Papenmeier, Luigi Nardi, and Matthias Poloczek. 2022. Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces. Advances in Neural Information Processing Systems 35, 11586\u201311601.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2019.02.002"},{"key":"e_1_3_4_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compstruct.2021.114646"},{"key":"e_1_3_4_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58112-1_12"},{"key":"e_1_3_4_53_1","doi-asserted-by":"publisher","DOI":"10.1080\/0305215X.2015.1082350"},{"key":"e_1_3_4_54_1","unstructured":"Malek Ben Salem Fran\u00e7ois Bachoc Olivier Roustant Fabrice Gamboa and Lionel Tomaso. 2019. Sequential Dimension Reduction for Learning Features of Expensive Black-Box Functions."},{"key":"e_1_3_4_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2006.1688506"},{"key":"e_1_3_4_56_1","volume-title":"Engineering Design via Surrogate Modelling: A Practical Guide","author":"Sobester Andr\u00e1s","year":"2008","unstructured":"Andr\u00e1s Sobester, Alexander Forrester, and Andy Keane. 2008. Engineering Design via Surrogate Modelling: A Practical Guide. John Wiley & Sons."},{"key":"e_1_3_4_57_1","doi-asserted-by":"publisher","unstructured":"Ryan Turner David Eriksson Michael McCourt Juha Kiili Eero Laaksonen Zhen Xu and Isabelle Guyon. 2021. Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020. arXiv:2104.10201. DOI: 10.48550\/arXiv.2104.10201","DOI":"10.48550\/arXiv.2104.10201"},{"key":"e_1_3_4_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICGTSPICC.2016.7955308"},{"key":"e_1_3_4_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510426"},{"key":"e_1_3_4_60_1","first-page":"745","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics","author":"Wang Zi","year":"2018","unstructured":"Zi Wang, Clement Gehring, Pushmeet Kohli, and Stefanie Jegelka. 2018. Batched Large-Scale Bayesian Optimization in High-Dimensional Spaces. In Proceedings of the International Conference on Artificial Intelligence and Statistics. PMLR, 745\u2013754."},{"key":"e_1_3_4_61_1","doi-asserted-by":"publisher","unstructured":"Ziyu Wang Frank Hutter Masrour Zoghi David Matheson and Nando de Freitas. 2016. Bayesian Optimization in a Billion Dimensions via Random Embeddings. arXiv:1301.1942. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1301.1942","DOI":"10.48550\/arXiv.1301.1942"},{"key":"e_1_3_4_62_1","doi-asserted-by":"publisher","unstructured":"Nathan Wycoff Mickael Binois and Stefan M. Wild. 2019. Sequential Learning of Active Subspaces. arXiv:1907.11572. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1907.11572","DOI":"10.48550\/arXiv.1907.11572"},{"key":"e_1_3_4_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-64712-4"},{"key":"e_1_3_4_64_1","volume-title":"Theory of Global Random Search","author":"Zhigljavsky Anatoly A.","year":"2012","unstructured":"Anatoly A. Zhigljavsky. 2012. Theory of Global Random Search, Vol. 65. Springer Science & Business Media."},{"key":"e_1_3_4_65_1","first-page":"43347","volume-title":"Proceedings of the 40th International Conference on Machine Learning","volume":"202","author":"Ziomek Juliusz Krzysztof","year":"2023","unstructured":"Juliusz Krzysztof Ziomek and Haitham Bou Ammar. 2023. Are Random Decompositions all we need in High Dimensional Bayesian Optimisation? In Proceedings of the 40th International Conference on Machine Learning, Vol. 202. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 43347\u201343368. Retrieved from https:\/\/proceedings.mlr.press\/v202\/ziomek23a.html"}],"container-title":["ACM Transactions on Evolutionary Learning and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3670683","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3670683","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:20Z","timestamp":1750294700000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3670683"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,29]]},"references-count":64,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3670683"],"URL":"https:\/\/doi.org\/10.1145\/3670683","relation":{},"ISSN":["2688-3007"],"issn-type":[{"value":"2688-3007","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,29]]},"assertion":[{"value":"2023-07-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}