{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:08:59Z","timestamp":1750306139917,"version":"3.41.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,9,20]],"date-time":"2017-09-20T00:00:00Z","timestamp":1505865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","award":["SFRH\/BD\/89573\/2012, project UID\/MULTI\/04046\/2013 and project UID\/EEA\/50008\/2013"],"award-info":[{"award-number":["SFRH\/BD\/89573\/2012, project UID\/MULTI\/04046\/2013 and project UID\/EEA\/50008\/2013"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2017,9,30]]},"abstract":"<jats:p>\n            A long-standing goal in artificial intelligence and robotics is synthesising agents that can effectively learn and adapt throughout their lifetime. One open-ended approach to behaviour learning in autonomous robots is\n            <jats:italic>online evolution<\/jats:italic>\n            , which is part of the evolutionary robotics field of research. In online evolution approaches, an evolutionary algorithm is executed on the robots during task execution, which enables continuous optimisation and adaptation of behaviour. Despite the potential for automatic behaviour learning, online evolution has not been widely adopted because it often requires several hours or days to synthesise solutions to a given task. In this respect, research in the field has failed to develop a prevalent algorithm able to effectively synthesise solutions to a large number of different tasks in a timely manner. Rather than focusing on a single algorithm, we argue for more general\n            <jats:italic>mechanisms<\/jats:italic>\n            that can combine the benefits of different algorithms to increase the performance of online evolution of robot controllers. We conduct a comprehensive assessment of a novel approach called\n            <jats:italic>online hyper-evolution<\/jats:italic>\n            (OHE). Robots executing OHE use the different sources of feedback information traditionally associated with controller evaluation to find effective evolutionary algorithms during task execution. First, we study two approaches: OHE-fitness, which uses the fitness score of controllers as the criterion to select promising algorithms over time, and OHE-diversity, which relies on the behavioural diversity of controllers for algorithm selection. We then propose a novel class of techniques called OHE-hybrid, which combine diversity and fitness to search for suitable algorithms. In addition to their effectiveness at selecting suitable algorithms, the different OHE approaches are evaluated for their ability to\n            <jats:italic>construct<\/jats:italic>\n            algorithms by controlling which algorithmic components should be employed for controller generation (e.g., mutation, crossover, among others), an unprecedented approach in evolutionary robotics. Results show that OHE (i) facilitates the evolution of controllers with high performance, (ii) can increase effectiveness at different stages of evolution by combining the benefits of multiple algorithms over time, and (iii) can be effectively applied to construct new algorithms during task execution. Overall, our study shows that OHE is a powerful new paradigm that allows robots to improve their learning process as they operate in the task environment.\n          <\/jats:p>","DOI":"10.1145\/3092815","type":"journal-article","created":{"date-parts":[[2017,9,20]],"date-time":"2017-09-20T12:35:19Z","timestamp":1505910919000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Hyper-Learning Algorithms for Online Evolution of Robot Controllers"],"prefix":"10.1145","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2702-9295","authenticated-orcid":false,"given":"Fernando","family":"Silva","sequence":"first","affiliation":[{"name":"Bio-inspired Computation and Intelligent Machines Lab and BioISI, Faculdade de Ci\u00eancias, Universidade de Lisboa and Instituto de Telecomunica\u00e7\u00f5es, Lisboa, Portugal"}]},{"given":"Lu\u00eds","family":"Correia","sequence":"additional","affiliation":[{"name":"BioISI, Faculdade de Ci\u00eancias, Universidade de Lisboa, Lisboa, Portugal"}]},{"given":"Anders Lyhne","family":"Christensen","sequence":"additional","affiliation":[{"name":"Bio-inspired Computation and Intelligent Machines Lab and Instituto Universit\u00e1rio de Lisboa (ISCTE-IUL) and Instituto de Telecomunica\u00e7\u00f5es, Lisboa, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2017,9,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/11593577_19"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/09540090412331314759"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1080\/13873954.2011.601425"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1057\/jors.2013.71"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2005.08.012"},{"volume-title":"Proceedings of the International Conference on the Practice and Theory of Automated Timetabling. Springer","author":"Cowling P.","key":"e_1_2_1_6_1"},{"volume-title":"Proceedings of the European Conference on the Applications of Evolutionary Computation. Springer","author":"Cuccu G.","key":"e_1_2_1_7_1"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12065-014-0110-x"},{"volume-title":"Proceedings of the International Conference on the Synthesis and Simulation of Living Systems. MIT Press","author":"Duarte M.","key":"e_1_2_1_10_1"},{"volume":"54","volume-title":"Parameter Setting in Evolutionary Algorithms. Studies in Computational Intelligence","author":"Eiben A.","key":"e_1_2_1_11_1"},{"volume-title":"Proceedings of the International Conference on Simulation of Adaptive Behavior. MIT Press","author":"Floreano D.","key":"e_1_2_1_12_1"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/3477.499791"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2006.1597062"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29178-4_6"},{"key":"e_1_2_1_16_1","first-page":"493","article-title":"Messy genetic algorithms: Motivation, analysis, and first results","volume":"3","author":"Goldberg D.","year":"1989","journal-title":"Complex Systems"},{"volume-title":"Proceedings of the Ibero-American Conference on Artificial Intelligence. Springer","author":"Gomes J.","key":"e_1_2_1_17_1"},{"volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation. IEEE Press","author":"Grobler J.","key":"e_1_2_1_18_1"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/75.2.383"},{"volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation. IEEE Press","year":"2086","author":"Kantschik W.","key":"e_1_2_1_20_1"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1830483.1830503"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00025"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/35023115"},{"volume-title":"Proceedings of the Conference on Autonomous Robot Systems and Competitions. IPCB","author":"Mondada F.","key":"e_1_2_1_24_1"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00048"},{"volume-title":"Proceedings of the Portuguese Conference on Artificial Intelligence. Springer","author":"Pillay N.","key":"e_1_2_1_26_1"},{"volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation. IEEE Press","author":"Ross P.","key":"e_1_2_1_27_1"},{"volume-title":"Artificial Intelligence: A Modern Approach","year":"1995","author":"Russell S.","key":"e_1_2_1_28_1"},{"volume-title":"Proceedings of the Annual Convention of the Society for the Study of Artificial Intelligence and the Simulation of Behaviour. Retrieved","year":"2017","author":"Ryser-Welch P.","key":"e_1_2_1_29_1"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-C.1969.222678"},{"volume-title":"Proceedings of the Portuguese Conference on Artificial Intelligence. Springer","author":"Silva F.","key":"e_1_2_1_31_1"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.33333"},{"volume-title":"Proceedings of the European Conference on the Applications of Evolutionary Computation. Springer International Publishing, Switzerland, 165--180","author":"Silva F.","key":"e_1_2_1_33_1"},{"volume-title":"Proceedings of the IEEE International Conference on Self-Adaptive and Self-Organizing Systems. IEEE Computer Society","author":"Silva F.","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.160938"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00172"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00141"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00170-7"},{"volume-title":"Proceedings of the European Conference on Artificial Life. Springer","author":"Wischmann S.","key":"e_1_2_1_39_1"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3092815","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3092815","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:37:27Z","timestamp":1750217847000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3092815"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,9,20]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,9,30]]}},"alternative-id":["10.1145\/3092815"],"URL":"https:\/\/doi.org\/10.1145\/3092815","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"type":"print","value":"1556-4665"},{"type":"electronic","value":"1556-4703"}],"subject":[],"published":{"date-parts":[[2017,9,20]]},"assertion":[{"value":"2017-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-09-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}