{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:00:53Z","timestamp":1750309253087,"version":"3.41.0"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T00:00:00Z","timestamp":1722211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ELKARTEK program under project KONFLOT","award":["KK-2022\/00100"],"award-info":[{"award-number":["KK-2022\/00100"]}]},{"name":"Spanish Ministry of Economy and Competitiveness through BCAM Severo Ochoa excellence accreditation","award":["SEV-2023-2026"],"award-info":[{"award-number":["SEV-2023-2026"]}]},{"name":"Edinburgh Napier University through EPSRC","award":["EP\/R035733\/1"],"award-info":[{"award-number":["EP\/R035733\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Evol. Learn. Optim."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>\n            Lengthy evaluation times are common in many optimization problems such as direct policy search tasks, especially when they involve conducting evaluations in the physical world, for example, in robotics applications. Often when evaluating solution over a fixed time period, it becomes clear that the objective value will not increase with additional computation time (e.g., when a two-wheeled robot continuously spins on the spot). In such cases, it makes sense to stop the evaluation early to save computation time. However, most approaches to stop the evaluation are problem specific and need to be specifically designed for the task at hand. Therefore, we propose an early stopping method for direct policy search. The proposed method only looks at the objective value at each timestep and requires no problem-specific knowledge. We test the introduced stopping criterion in five direct policy search environments drawn from games, robotics, and classic control domains and show that it can save up to\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(75\\%\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            of the computation time. We also compare it with problem-specific stopping criteria and show that it performs comparably, while being more generally applicable.\n          <\/jats:p>","DOI":"10.1145\/3653024","type":"journal-article","created":{"date-parts":[[2024,3,20]],"date-time":"2024-03-20T14:46:28Z","timestamp":1710945988000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Generalized Early Stopping in Evolutionary Direct Policy Search"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8044-0334","authenticated-orcid":false,"given":"Etor","family":"Arza","sequence":"first","affiliation":[{"name":"Basque Center for Applied Mathematics, Bilbao, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1749-9154","authenticated-orcid":false,"given":"L\u00e9ni K.","family":"Le Goff","sequence":"additional","affiliation":[{"name":"Edinburgh Napier University, Edinburgh, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5405-4413","authenticated-orcid":false,"given":"Emma","family":"Hart","sequence":"additional","affiliation":[{"name":"Edinburgh Napier University, Edinburgh, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,29]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00163-020-00336-7"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00163-020-00336-7"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2022.2084405"},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1983.6313077"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1162\/artl.2010.Bongard.024"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2010.2096540"},{"key":"e_1_3_2_8_1","unstructured":"Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540 [cs]."},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2020.06.045"},{"key":"e_1_3_2_10_1","volume-title":"Practical Nonparametric Statistics","author":"Conover William Jay","year":"1980","unstructured":"William Jay Conover. 1980. Practical Nonparametric Statistics. Wiley."},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2015.2419431"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2021.105615"},{"key":"e_1_3_2_13_1","unstructured":"Thibaud De Souza. 2014. The blind game designer\u2014Darwinism in a fast pace casual action game. https:\/\/www.gamedeveloper.com\/design\/the-blind-game-designer---darwinism-in-a-fast-pace-casual-action-game-"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.biophys.37.092707.153558"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12065-014-0110-x"},{"key":"e_1_3_2_16_1","unstructured":"Johann Dreo and Manuel L\u00f3pez-Ib\u00e1\u00f1ez. 2021. Extensible logging and empirical attainment function for IOHexperimenter. arXiv:2109.13773."},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2024.3352450"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-66494-7_2"},{"key":"e_1_3_2_19_1","first-page":"1437","volume-title":"Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80)","author":"Falkner Stefan","year":"2018","unstructured":"Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and efficient hyperparameter optimization at scale. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). Jennifer Dy and Andreas Krause (Eds.), PMLR, 1437\u20131446."},{"key":"e_1_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0967-0661(02)00081-3"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2006.1679"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17146-8"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2021.0117"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CIG.2009.5286468"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.949487"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2861"},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5"},{"key":"e_1_3_2_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2017.12.030"},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1143997.1144082"},{"key":"e_1_3_2_30_1","first-page":"1238","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Karnin Zohar","year":"2013","unstructured":"Zohar Karnin, Tomer Koren, and Oren Somekh. 2013. Almost optimal exploration in multi-armed bandits. In Proceedings of the International Conference on Machine Learning. PMLR, 1238\u20131246."},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-021-04210-0"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-014-0371-0"},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","unstructured":"S\u00e9bastien Le Digabel and Stefan M. Wild. 2015. A taxonomy of constraints in simulation-based optimization. arXiv:1505.07881. Retrieved from 10.48550\/ARXIV.1505.07881","DOI":"10.48550\/ARXIV.1505.07881"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1162\/isal_a_00299"},{"key":"e_1_3_2_35_1","unstructured":"L\u00e9ni K. Le Goff Edgar Buchanan Emma Hart Agoston E. Eiben Wei Li Matteo De Carlo Alan F. Winfield Matthew F. Hale Robert Woolley Mike Angus Jon Timmis and Andy M. Tyrrell. 2021. Morpho-evolution with learning using a controller archive as an inheritance mechanism. arXiv:2104.04269 [cs]."},{"key":"e_1_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00025"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3242042"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocs.2015.11.004"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.orp.2016.09.002"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_41_1","doi-asserted-by":"publisher","DOI":"10.1137\/120882834"},{"key":"e_1_3_2_42_1","doi-asserted-by":"publisher","DOI":"10.3390\/psf2021003006"},{"key":"e_1_3_2_43_1","doi-asserted-by":"publisher","DOI":"10.2514\/6.2006-7114"},{"key":"e_1_3_2_44_1","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347."},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.1162\/106365602320169811"},{"key":"e_1_3_2_46_1","doi-asserted-by":"publisher","DOI":"10.5555\/911176"},{"key":"e_1_3_2_47_1","doi-asserted-by":"publisher","DOI":"10.1162\/isal_a_00295"},{"key":"e_1_3_2_48_1","unstructured":"Vivek Verma. 2020. Applying neural networks and neuroevolution of augmenting topologies to play Super Mario Bros. https:\/\/vixra.org\/abs\/2008.0130"},{"key":"e_1_3_2_49_1","unstructured":"Yijia Wang Matthias Poloczek and Daniel R. Jiang. 2022. Subgoal-based exploration via Bayesian optimization. arXiv:1910.09143 [cs math]"},{"key":"e_1_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2021.3051608"},{"key":"e_1_3_2_51_1","doi-asserted-by":"publisher","unstructured":"Ander Zarketa-Astigarraga Alain Martin-Mayor Aimar Maeso Borja De Miguel Manex Martinez-Agirre and Markel Penalba. 2023. A computationally efficient Ga-based optimisation tool for the design of power take-of systems in realistic wave climates: The wells turbine case. DOI: 10.2139\/ssrn.4379648","DOI":"10.2139\/ssrn.4379648"},{"key":"e_1_3_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3067763"}],"container-title":["ACM Transactions on Evolutionary Learning and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653024","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3653024","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:56:55Z","timestamp":1750291015000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653024"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,29]]},"references-count":51,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3653024"],"URL":"https:\/\/doi.org\/10.1145\/3653024","relation":{},"ISSN":["2688-3007"],"issn-type":[{"type":"electronic","value":"2688-3007"}],"subject":[],"published":{"date-parts":[[2024,7,29]]},"assertion":[{"value":"2023-06-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}