{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T23:41:03Z","timestamp":1780616463887,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":19,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,7,6]],"date-time":"2018-07-06T00:00:00Z","timestamp":1530835200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["P2-0098"],"award-info":[{"award-number":["P2-0098"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]},{"name":"European Commision","award":["692286"],"award-info":[{"award-number":["692286"]}]},{"DOI":"10.13039\/501100000921","name":"European Cooperation in Science and Technology","doi-asserted-by":"publisher","award":["CA15140"],"award-info":[{"award-number":["CA15140"]}],"id":[{"id":"10.13039\/501100000921","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,7,6]]},"DOI":"10.1145\/3205651.3208232","type":"proceedings-article","created":{"date-parts":[[2018,7,6]],"date-time":"2018-07-06T19:30:44Z","timestamp":1530905444000},"page":"1329-1336","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["The impact of statistics for benchmarking in evolutionary computation research"],"prefix":"10.1145","author":[{"given":"Tome","family":"Eftimov","sequence":"first","affiliation":[{"name":"Jo\u017eef Stefan Institute, Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Koro\u0161ec","sequence":"additional","affiliation":[{"name":"Jo\u017eef Stefan Institute, Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7,6]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2739482.2768467"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2739482.2768468"},{"key":"e_1_3_2_1_3_1","volume-title":"d.}. Black-Box Benchmarking","author":"Box Optimization Competition Comp Black","year":"2015","unstructured":"BB Comp Black Box Optimization Competition , {n. d.}. Black-Box Benchmarking 2015 . http:\/\/coco.gforge.inria.fr\/doku.php?id=bbob-2015. ({n. d.}). Accessed : 2016-02-01. BBComp Black Box Optimization Competition, {n. d.}. Black-Box Benchmarking 2015. http:\/\/coco.gforge.inria.fr\/doku.php?id=bbob-2015. ({n. d.}). Accessed: 2016-02-01."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2739482.2768470"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Wayne W Daniel and Chad Lee Cross. 1995. Biostatistics: a foundation for analysis in the health sciences. (1995).  Wayne W Daniel and Chad Lee Cross. 1995. Biostatistics: a foundation for analysis in the health sciences. (1995).","DOI":"10.2307\/2533362"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/1248547.1248548"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2011.02.002"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2017.07.015"},{"key":"e_1_3_2_1_9_1","first-page":"1","article-title":"Comparing distributions: the two-sample Anderson-Darling test as an alternative to the Kolmogorov-Smirnoff test","volume":"6","author":"Engmann Sonja","year":"2011","unstructured":"Sonja Engmann and Denis Cousineau . 2011 . Comparing distributions: the two-sample Anderson-Darling test as an alternative to the Kolmogorov-Smirnoff test . Journal of Applied Quantitative Methods 6 , 3 (2011), 1 -- 17 . Sonja Engmann and Denis Cousineau. 2011. Comparing distributions: the two-sample Anderson-Darling test as an alternative to the Kolmogorov-Smirnoff test. Journal of Applied Quantitative Methods 6, 3 (2011), 1--17.","journal-title":"Journal of Applied Quantitative Methods"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10732-008-9080-4"},{"key":"e_1_3_2_1_11_1","volume-title":"Vyacheslav Lyubchich, Weiwen Miao, and Kimihiro Noguchi.","author":"Gastwirth Joseph L","year":"2015","unstructured":"Joseph L Gastwirth , Yulia R Gel , W L Wallace Hui , Vyacheslav Lyubchich, Weiwen Miao, and Kimihiro Noguchi. 2015 . law-stat: Tools for Biostatistics, Public Policy , and Law. https:\/\/CRAN.R-project.org\/package=lawstat R package version 3.0. Joseph L Gastwirth, Yulia R Gel, W L Wallace Hui, Vyacheslav Lyubchich, Weiwen Miao, and Kimihiro Noguchi. 2015. law-stat: Tools for Biostatistics, Public Policy, and Law. https:\/\/CRAN.R-project.org\/package=lawstat R package version 3.0."},{"key":"e_1_3_2_1_12_1","unstructured":"Nikolaus Hansen Anne Auger Steffen Finck and Raymond Ros. 2010. Real-parameter black-box optimization benchmarking 2010: Experimental setup. (2010).  Nikolaus Hansen Anne Auger Steffen Finck and Raymond Ros. 2010. Real-parameter black-box optimization benchmarking 2010: Experimental setup. (2010)."},{"key":"e_1_3_2_1_13_1","volume-title":"COCO: A platform for comparing continuous optimizers in a black-box setting. arXiv preprint arXiv:1603.08785","author":"Hansen Nikolaus","year":"2016","unstructured":"Nikolaus Hansen , Anne Auger , Olaf Mersmann , Tea Tusar , and Dimo Brockhoff . 2016 . COCO: A platform for comparing continuous optimizers in a black-box setting. arXiv preprint arXiv:1603.08785 (2016). Nikolaus Hansen, Anne Auger, Olaf Mersmann, Tea Tusar, and Dimo Brockhoff. 2016. COCO: A platform for comparing continuous optimizers in a black-box setting. arXiv preprint arXiv:1603.08785 (2016)."},{"key":"e_1_3_2_1_14_1","volume-title":"Multiple testing. Part II. Step-down procedures for control of the family-wise error rate. Statistical applications in genetics and molecular biology 3, 1","author":"Mark J","year":"2004","unstructured":"Mark J Laan van der, Sandrine Dudoit , and Katherine S Pollard . 2004. Multiple testing. Part II. Step-down procedures for control of the family-wise error rate. Statistical applications in genetics and molecular biology 3, 1 ( 2004 ), 1--33. Mark J Laan van der, Sandrine Dudoit, and Katherine S Pollard. 2004. Multiple testing. Part II. Step-down procedures for control of the family-wise error rate. Statistical applications in genetics and molecular biology 3, 1 (2004), 1--33."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001576.2001690"},{"key":"e_1_3_2_1_16_1","volume-title":"International Conference on Parallel Problem Solving from Nature. Springer, 73--82","author":"Mersmann Olaf","year":"2010","unstructured":"Olaf Mersmann , Mike Preuss , and Heike Trautmann . 2010 . Bench-marking evolutionary algorithms: Towards exploratory landscape analysis . In International Conference on Parallel Problem Solving from Nature. Springer, 73--82 . Olaf Mersmann, Mike Preuss, and Heike Trautmann. 2010. Bench-marking evolutionary algorithms: Towards exploratory landscape analysis. In International Conference on Parallel Problem Solving from Nature. Springer, 73--82."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2739482.2768469"},{"key":"e_1_3_2_1_18_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2015","unstructured":"R Core Team . 2015 . R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing , Vienna, Austria . https:\/\/www.R-project.org\/ R Core Team. 2015. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/"},{"key":"e_1_3_2_1_19_1","unstructured":"Fritz Scholz and Angie Zhu. 2016. kSamples: K-Sample Rank Tests and their Combinations. https:\/\/CRAN.R-project.org\/package=kSamples R package version 1.2-4.  Fritz Scholz and Angie Zhu. 2016. kSamples: K-Sample Rank Tests and their Combinations. https:\/\/CRAN.R-project.org\/package=kSamples R package version 1.2-4."}],"event":{"name":"GECCO '18: Genetic and Evolutionary Computation Conference","location":"Kyoto Japan","acronym":"GECCO '18","sponsor":["SIGEVO ACM Special Interest Group on Genetic and Evolutionary Computation"]},"container-title":["Proceedings of the Genetic and Evolutionary Computation Conference Companion"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3205651.3208232","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3205651.3208232","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:07:13Z","timestamp":1750212433000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3205651.3208232"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,6]]},"references-count":19,"alternative-id":["10.1145\/3205651.3208232","10.1145\/3205651"],"URL":"https:\/\/doi.org\/10.1145\/3205651.3208232","relation":{},"subject":[],"published":{"date-parts":[[2018,7,6]]},"assertion":[{"value":"2018-07-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}