{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T02:35:54Z","timestamp":1777430154532,"version":"3.51.4"},"reference-count":16,"publisher":"MIT Press","issue":"3","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We present IOHexperimenter, the experimentation module of the IOHprofiler project. IOHexperimenter aims at providing an easy-to-use and customizable toolbox for benchmarking iterative optimization heuristics such as local search, evolutionary and genetic algorithms, and Bayesian optimization techniques. IOHexperimenter can be used as a stand-alone tool or as part of a benchmarking pipeline that uses other modules of the IOHprofiler environment.<\/jats:p>\n               <jats:p>IOHexperimenter provides an efficient interface between optimization problems and their solvers while allowing for granular logging of the optimization process. Its logs are fully compatible with existing tools for interactive data analysis, which significantly speeds up the deployment of a benchmarking pipeline. The main components of IOHexperimenter are the environment to build customized problem suites and the various logging options that allow users to steer the granularity of the data records.<\/jats:p>","DOI":"10.1162\/evco_a_00342","type":"journal-article","created":{"date-parts":[[2023,7,24]],"date-time":"2023-07-24T17:57:06Z","timestamp":1690221426000},"page":"205-210","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":22,"title":["IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics"],"prefix":"10.1162","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1169-1962","authenticated-orcid":true,"given":"Jacob","family":"de Nobel","sequence":"first","affiliation":[{"name":"LIACS, Leiden University, the Netherlands j.p.de.nobel@liacs.leidenuniv.nl"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8707-4189","authenticated-orcid":true,"given":"Furong","family":"Ye","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, the Netherlands f.ye@liacs.leidenuniv.nl"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3040-7162","authenticated-orcid":true,"given":"Diederick","family":"Vermetten","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, the Netherlands d.l.vermetten@liacs.leidenuniv.nl"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4933-5181","authenticated-orcid":true,"given":"Hao","family":"Wang","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, the Netherlands h.wang@liacs.leidenuniv.nl"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4981-3227","authenticated-orcid":true,"given":"Carola","family":"Doerr","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, CNRS, LIP6, Paris, France Carola.Doerr@lip6.fr"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6768-1478","authenticated-orcid":true,"given":"Thomas","family":"B\u00e4ck","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, the Netherlands t.h.w.baeck@liacs.leidenuniv.nl"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,9,3]]},"reference":[{"key":"2024090312543662100_B1","doi-asserted-by":"crossref","first-page":"1365","DOI":"10.1145\/3449726.3463155","article-title":"Towards large scale automated algorithm design by integrating modular benchmarking frameworks","author":"Aziz-Alaoui","year":"2021","journal-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)"},{"key":"2024090312543662100_B2","doi-asserted-by":"crossref","first-page":"1375","DOI":"10.1145\/3449726.3463167","article-title":"Tuning as a means of assessing the benefits of new ideas in interplay with existing algorithmic modules","author":"de Nobel","year":"2021","journal-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)"},{"key":"2024090312543662100_B3","article-title":"IOHprofiler: A benchmarking and profiling tool for iterative optimization heuristics","author":"Doerr","year":"2018","journal-title":"CoRR"},{"key":"2024090312543662100_B4","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2019.106027","article-title":"Benchmarking discrete optimization heuristics with IOHprofiler","volume":"88","author":"Doerr","year":"2020","journal-title":"Applied Soft Computing"},{"key":"2024090312543662100_B5","first-page":"1689","article-title":"Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009","author":"Hansen","year":"2010","journal-title":"Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO)"},{"issue":"1","key":"2024090312543662100_B6","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1080\/10556788.2020.1808977","article-title":"COCO: A platform for comparing continuous optimizers in a black-box setting","volume":"36","author":"Hansen","year":"2021","journal-title":"Optimization Methods and Software"},{"key":"2024090312543662100_B7","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1007\/978-3-031-14714-2_4","article-title":"Per-run algorithm selection with warm-starting using trajectory-based features","volume":"13398","author":"Kostovska","year":"2022","journal-title":"Proceedings of Parallel Problem Solving from Nature"},{"key":"2024090312543662100_B8","article-title":"Benchmark functions for the CEC 2013 special session and competition on large-scale global optimization","author":"Li","year":"2013"},{"key":"2024090312543662100_B9","article-title":"Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization","author":"Liang","year":"2013"},{"key":"2024090312543662100_B10","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1007\/978-3-031-30229-9_25","article-title":"BBOB instance analysis: Landscape properties and algorithm performance across problem instances","volume":"13989","author":"Long","year":"2023","journal-title":"Proceedings of the International Conference on the Applications of Evolutionary Computation"},{"key":"2024090312543662100_B11","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1145\/3449726.3461433","article-title":"Evolutionary submodular optimisation","author":"Neumann","year":"2021","journal-title":"Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO)"},{"key":"2024090312543662100_B12","article-title":"Nevergrad\u2014A gradient-free optimization platform","author":"Rapin","year":"2018"},{"key":"2024090312543662100_B13","article-title":"Competition on single objective bound constrained numerical optimization","author":"Suganthan","year":"2020"},{"issue":"1","key":"2024090312543662100_B14","doi-asserted-by":"publisher","first-page":"3:1","DOI":"10.1145\/3510426","article-title":"IOHanalyzer: Detailed performance analyses for iterative optimization heuristics","volume":"2","author":"Wang","year":"2022","journal-title":"ACM Transactions on Evolutionary Learning and Optimization"},{"key":"2024090312543662100_B15","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2020.106269","article-title":"Selecting a diverse set of benchmark instances from a tunable model problem for black-box discrete optimization algorithms","volume":"92","author":"Weise","year":"2020","journal-title":"Applied Soft Computing"},{"issue":"6","key":"2024090312543662100_B16","doi-asserted-by":"publisher","first-page":"1526","DOI":"10.1109\/TEVC.2022.3159087","article-title":"Automated configuration of genetic algorithms by tuning for anytime performance","volume":"26","author":"Ye","year":"2022","journal-title":"IEEE Transactions on Evolutionary Computation"}],"container-title":["Evolutionary Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/evco\/article-pdf\/32\/3\/205\/2463042\/evco_a_00342.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/evco\/article-pdf\/32\/3\/205\/2463042\/evco_a_00342.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T12:54:49Z","timestamp":1725368089000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/evco\/article\/32\/3\/205\/116949\/IOHexperimenter-Benchmarking-Platform-for"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":16,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,9,3]]},"published-print":{"date-parts":[[2024,9,3]]}},"URL":"https:\/\/doi.org\/10.1162\/evco_a_00342","relation":{},"ISSN":["1530-9304"],"issn-type":[{"value":"1530-9304","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}