{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T04:06:36Z","timestamp":1777521996243,"version":"3.51.4"},"reference-count":25,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2011,2,28]],"date-time":"2011-02-28T00:00:00Z","timestamp":1298851200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2011,4]]},"abstract":"<jats:p>In this article we propose a framework for performing embodied evolution with a limited number of robots, by utilizing time-sharing in subpopulations of virtual agents hosted in each robot. Within this framework, we explore the combination of within-generation learning of basic survival behaviors by reinforcement learning, and evolutionary adaptations over the generations of the basic behavior selection policy, the reward functions, and metaparameters for reinforcement learning. We apply a biologically inspired selection scheme, in which there is no explicit communication of the individuals\u2019 fitness information. The individuals can only reproduce offspring by mating\u2014a pair-wise exchange of genotypes\u2014and the probability that an individual reproduces offspring in its own subpopulation is dependent on the individual\u2019s \u2018\u2018health,\u2019\u2019 that is, energy level, at the mating occasion. We validate the proposed method by comparing it with evolution using standard centralized selection, in simulation, and by transferring the obtained solutions to hardware using two real robots.<\/jats:p>","DOI":"10.1177\/1059712310397633","type":"journal-article","created":{"date-parts":[[2011,3,1]],"date-time":"2011-03-01T16:27:04Z","timestamp":1298996824000},"page":"101-120","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":17,"title":["Darwinian embodied evolution of the learning ability for survival"],"prefix":"10.1177","volume":"19","author":[{"given":"Stefan","family":"Elfwing","sequence":"first","affiliation":[{"name":"Centre for Autonomous Systems, Numerical Analysis and Computer Science, Royal Institute of Technology (KTH), Sweden, Neural Computation Unit, Initial Research Project, Okinawa Institute of Science and Technology, JST, Japan,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eiji","family":"Uchibe","sequence":"additional","affiliation":[{"name":"Neural Computation Unit, Initial Research Project, Okinawa Institute of Science and Technology, JST, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenji","family":"Doya","sequence":"additional","affiliation":[{"name":"Neural Computation Unit, Initial Research Project, Okinawa Institute of Science and Technology, JST, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henrik I","family":"Christensen","sequence":"additional","affiliation":[{"name":"Centre for Autonomous Systems, Numerical Analysis and Computer Science, Royal Institute of Technology (KTH), Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2011,2,28]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1086\/276408"},{"issue":"4","key":"atypb2","first-page":"485","volume":"15","author":"Doya, K.","year":"2002","journal-title":"Neural Networks"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1177\/105971230501300206"},{"key":"atypb4","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation (CEC2005) (Vol. 3)IEEE Press","author":"Elfwing, S."},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2006.890270"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1177\/1059712308092835"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2003.1250664"},{"key":"atypb8","volume-title":"Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB1996)","author":"Floreano, D."},{"key":"atypb9","first-page":"495","volume":"1","author":"Hinton, G.","year":"1987","journal-title":"Complex Systems"},{"key":"atypb10","volume-title":"Proceedings of the International Conference on Machine learning (ICML2003)","author":"Laud, A."},{"key":"atypb11","volume-title":"Proceedings of the International Workshop on Epigenetic Robotics and Robotics (EPIROB2002)","author":"Nehmzow, U."},{"key":"atypb12","volume-title":"Proceedings of the International Conference on Machine learning (ICML1999)","author":"Ng, A.Y."},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.1177\/1059-712302-010001-01"},{"key":"atypb14","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/2889.001.0001","volume-title":"Evolutionary robotics. The biology, intelligence, and technology of self-organizing machines","author":"Nolfi, S.","year":"2000"},{"key":"atypb15","volume-title":"On-line Q-learning using connectionist systems. (Tech. Rep. CUED\/F-INFENG\/TR 166)","author":"Rummery, G.A.","year":"1994"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1038\/nrn729"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1007\/BF00114726"},{"key":"atypb18","unstructured":"Sutton, R.S. ( 1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer & M. E. Hasselmo (Eds.), Advances in neural information processing systems 8 (pp. 1038-1044). Cambridge, MA: MIT Press."},{"key":"atypb19","volume-title":"Reinforcement learning: An introduction","author":"Sutton, R.S.","year":"1998"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1162\/evco.1996.4.3.iv"},{"key":"atypb21","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO2000)","author":"Urzelai, J."},{"key":"atypb22","volume-title":"Proceedings of the International Symposium on Artificial Life and Robotics (AROB2003)","author":"Usui, Y."},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00170-7"},{"key":"atypb24","first-page":"877","volume":"7","author":"Whiteson, S.","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1190"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310397633","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310397633","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:59Z","timestamp":1777392959000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1059712310397633"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,2,28]]},"references-count":25,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2011,4]]}},"alternative-id":["10.1177\/1059712310397633"],"URL":"https:\/\/doi.org\/10.1177\/1059712310397633","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,2,28]]}}}