{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T05:52:46Z","timestamp":1783749166459,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":24,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,7,13]],"date-time":"2019-07-13T00:00:00Z","timestamp":1562976000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["Z2-8177"],"award-info":[{"award-number":["Z2-8177"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Agence Nationale de la Recherche","award":["ANR-11-LABX-0056-LMH"],"award-info":[{"award-number":["ANR-11-LABX-0056-LMH"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,7,13]]},"DOI":"10.1145\/3321707.3321868","type":"proceedings-article","created":{"date-parts":[[2019,7,3]],"date-time":"2019-07-03T13:48:04Z","timestamp":1562161684000},"page":"718-726","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":38,"title":["Mixed-integer benchmark problems for single- and bi-objective optimization"],"prefix":"10.1145","author":[{"given":"Tea","family":"Tu\u0161ar","sequence":"first","affiliation":[{"name":"Jo\u017eef Stefan Institute, Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dimo","family":"Brockhoff","sequence":"additional","affiliation":[{"name":"Inria, IP Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikolaus","family":"Hansen","sequence":"additional","affiliation":[{"name":"Inria, IP Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,13]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS","author":"Bergstra J.","year":"2011","unstructured":"J. Bergstra , R. Bardenet , Y. Bengio , and B. Kegl . 2011. Algorithms for hyper-parameter optimization . In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS 2011 ). 2546--2554. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kegl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS 2011). 2546--2554."},{"key":"e_1_3_2_1_2_1","unstructured":"D. Brockhoff T. Tu\u0161ar A. Auger and N. Hansen. 2019. Using well-understood single-objective functions in multiobjective black-box optimization test suites. ArXiv e-prints arXiv:1604.00359v3 (2019).  D. Brockhoff T. Tu\u0161ar A. Auger and N. Hansen. 2019. Using well-understood single-objective functions in multiobjective black-box optimization test suites. ArXiv e-prints arXiv:1604.00359v3 (2019)."},{"key":"e_1_3_2_1_3_1","unstructured":"D. Brockhoff T. Tu\u0161ar D. Tu\u0161ar T. Wagner N. Hansen and A. Auger. 2016. Biobjective performance assessment with the COCO platform. ArXiv e-prints arXiv:1605.01746 (2016).  D. Brockhoff T. Tu\u0161ar D. Tu\u0161ar T. Wagner N. Hansen and A. Auger. 2016. Biobjective performance assessment with the COCO platform. ArXiv e-prints arXiv:1605.01746 (2016)."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.15.1.114.15159"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amc.2009.02.044"},{"key":"e_1_3_2_1_6_1","volume-title":"Technical Report 2009\/20. Research Center PPE, Austria","author":"Finck S.","year":"2009","unstructured":"S. Finck , N. Hansen , R. Ros , and A. Auger . 2009 . Real-parameter black-box optimization benchmarking 2009: Presentation of the noiseless functions. Technical Report 2009\/20. Research Center PPE, Austria . http:\/\/coco.lri.fr\/downloads\/download15.03\/bbobdocfunctions.pdf Updated February 2010. S. Finck, N. Hansen, R. Ros, and A. Auger. 2009. Real-parameter black-box optimization benchmarking 2009: Presentation of the noiseless functions. Technical Report 2009\/20. Research Center PPE, Austria. http:\/\/coco.lri.fr\/downloads\/download15.03\/bbobdocfunctions.pdf Updated February 2010."},{"key":"e_1_3_2_1_7_1","unstructured":"N. Hansen Y. Akimoto and P. Baudis. 2019. CMA-ES\/pycma on Github. Zenodo. (Feb. 2019).  N. Hansen Y. Akimoto and P. Baudis. 2019. CMA-ES\/pycma on Github. Zenodo. (Feb. 2019)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"N. Hansen and A. Auger. 2014. Principled design of continuous stochastic search: From theory to practice. In Theory and principled methods for the design of metaheuristics Y. Borenstein and A. Moraglio (Eds.). Springer Berlin Heidelberg 145--180.  N. Hansen and A. Auger. 2014. Principled design of continuous stochastic search: From theory to practice. In Theory and principled methods for the design of metaheuristics Y. Borenstein and A. Moraglio (Eds.). Springer Berlin Heidelberg 145--180.","DOI":"10.1007\/978-3-642-33206-7_8"},{"key":"e_1_3_2_1_9_1","volume-title":"COCO: Performance assessment. ArXiv e-prints arXiv:1605.03560","author":"Hansen N.","year":"2016","unstructured":"N. Hansen , A Auger , D. Brockhoff , D. Tu\u0161ar , and T. Tu\u0161ar . 2016 . COCO: Performance assessment. ArXiv e-prints arXiv:1605.03560 (2016). N. Hansen, A Auger, D. Brockhoff, D. Tu\u0161ar, and T. Tu\u0161ar. 2016. COCO: Performance assessment. ArXiv e-prints arXiv:1605.03560 (2016)."},{"key":"e_1_3_2_1_10_1","volume-title":"COCO: A platform for comparing continuous optimizers in a black-box setting. ArXiv e-prints arXiv:1603.08785","author":"Hansen N.","year":"2016","unstructured":"N. Hansen , A. Auger , O. Mersmann , T. Tu\u0161ar , and D. Brockhoff . 2016 . COCO: A platform for comparing continuous optimizers in a black-box setting. ArXiv e-prints arXiv:1603.08785 (2016). N. Hansen, A. Auger, O. Mersmann, T. Tu\u0161ar, and D. Brockhoff. 2016. COCO: A platform for comparing continuous optimizers in a black-box setting. ArXiv e-prints arXiv:1603.08785 (2016)."},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the International Conference on Learning and Intelligent Optimization (LION 8) (Lecture Notes in Computer Science)","volume":"8426","author":"Hutter F.","unstructured":"F. Hutter , M. L\u00f3pez-Ib\u00e1nez , C. Fawcett , M. Lindauer , H. H. Hoos , K. Leyton-Brown , and T. St\u00fctzle . 2014. AClib: A benchmark library for algorithm configuration . In Proceedings of the International Conference on Learning and Intelligent Optimization (LION 8) (Lecture Notes in Computer Science) , Vol. 8426 . Springer, 36--40. F. Hutter, M. L\u00f3pez-Ib\u00e1nez, C. Fawcett, M. Lindauer, H. H. Hoos, K. Leyton-Brown, and T. St\u00fctzle. 2014. AClib: A benchmark library for algorithm configuration. In Proceedings of the International Conference on Learning and Intelligent Optimization (LION 8) (Lecture Notes in Computer Science), Vol. 8426. Springer, 36--40."},{"key":"e_1_3_2_1_12_1","unstructured":"E. Jones T. Oliphant P. Peterson etal 2001-. SciPy: Open source scientific tools for Python. (2001-). http:\/\/www.scipy.org\/ {Online; accessed 5. 2. 2019}.  E. Jones T. Oliphant P. Peterson et al. 2001-. SciPy: Open source scientific tools for Python. (2001-). http:\/\/www.scipy.org\/ {Online; accessed 5. 2. 2019}."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"S. A. Kauffman. 1993. The Origins of Order: Self-Organization and Selection in Evolution. OUP USA.  S. A. Kauffman. 1993. The Origins of Order: Self-Organization and Selection in Evolution. OUP USA.","DOI":"10.1007\/978-94-015-8054-0_8"},{"key":"e_1_3_2_1_14_1","volume-title":"Mixed integer evolution strategies for parameter optimization. Evolutionary computation 21, 1","author":"Li R.","year":"2013","unstructured":"R. Li , M. T. M. Emmerich , J. Eggermont , T. B\u00e4ck , M. Sch\u00fctz , J. Dijkstra , and J. H. C. Reiber . 2013. Mixed integer evolution strategies for parameter optimization. Evolutionary computation 21, 1 ( 2013 ), 29--64. R. Li, M. T. M. Emmerich, J. Eggermont, T. B\u00e4ck, M. Sch\u00fctz, J. Dijkstra, and J. H. C. Reiber. 2013. Mixed integer evolution strategies for parameter optimization. Evolutionary computation 21, 1 (2013), 29--64."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/11844297_5"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2013.2281531"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2010.05.007"},{"key":"e_1_3_2_1_18_1","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation (CEC","author":"McClymont K.","year":"2011","unstructured":"K. McClymont and E. Keedwell . 2011. Benchmark multi-objective optimisation test problems with mixed encodings . In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2011 ). IEEE, 2131--2138. K. McClymont and E. Keedwell. 2011. Benchmark multi-objective optimisation test problems with mixed encodings. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2011). IEEE, 2131--2138."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1137\/080724083"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11081-015-9281-2"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008202821328"},{"key":"e_1_3_2_1_22_1","unstructured":"P. N. Suganthan N. Hansen J. J. Liang K. Deb Y.-P. Chen A. Auger and S. Tiwari. 2005. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report 2005005. IIT Kanpur India.  P. N. Suganthan N. Hansen J. J. Liang K. Deb Y.-P. Chen A. Auger and S. Tiwari. 2005. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report 2005005. IIT Kanpur India."},{"key":"e_1_3_2_1_23_1","volume-title":"COCO: The bi-objective black-box optimization benchmarking (bbob-biobj) test suite. ArXiv e-prints arXiv:1604.00359","author":"Tu\u0161ar T.","year":"2016","unstructured":"T. Tu\u0161ar , D. Brockhoff , N. Hansen , and A. Auger . 2016 . COCO: The bi-objective black-box optimization benchmarking (bbob-biobj) test suite. ArXiv e-prints arXiv:1604.00359 (2016). T. Tu\u0161ar, D. Brockhoff, N. Hansen, and A. Auger. 2016. COCO: The bi-objective black-box optimization benchmarking (bbob-biobj) test suite. ArXiv e-prints arXiv:1604.00359 (2016)."},{"key":"e_1_3_2_1_24_1","volume-title":"Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN XV) (Lecture Notes in Computer Science)","volume":"11101","author":"Varelas K.","unstructured":"K. Varelas , A. Auger , D. Brockhoff , N. Hansen , O. A. ElHara , Y. Semet , R. Kassab , and F. Barbaresco . 2018. A comparative study of large-scale variants of CMA-ES . In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN XV) (Lecture Notes in Computer Science) , Vol. 11101 . Springer, 3--15. K. Varelas, A. Auger, D. Brockhoff, N. Hansen, O. A. ElHara, Y. Semet, R. Kassab, and F. Barbaresco. 2018. A comparative study of large-scale variants of CMA-ES. In Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN XV) (Lecture Notes in Computer Science), Vol. 11101. Springer, 3--15."}],"event":{"name":"GECCO '19: Genetic and Evolutionary Computation Conference","location":"Prague Czech Republic","acronym":"GECCO '19","sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"]},"container-title":["Proceedings of the Genetic and Evolutionary Computation Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321707.3321868","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3321707.3321868","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:29Z","timestamp":1750206329000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321707.3321868"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,13]]},"references-count":24,"alternative-id":["10.1145\/3321707.3321868","10.1145\/3321707"],"URL":"https:\/\/doi.org\/10.1145\/3321707.3321868","relation":{},"subject":[],"published":{"date-parts":[[2019,7,13]]},"assertion":[{"value":"2019-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}