{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T03:22:20Z","timestamp":1785295340083,"version":"3.55.0"},"reference-count":64,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2010,10,4]],"date-time":"2010-10-04T00:00:00Z","timestamp":1286150400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2010,12]]},"abstract":"<jats:p>Biological brains can adapt and learn from past experience. Yet neuroevolution, that is, automatically creating artificial neural networks (ANNs) through evolutionary algorithms, has sometimes focused on static ANNs that cannot change their weights during their lifetime. A profound problem with evolving adaptive systems is that learning to learn is highly deceptive. Because it is easier at first to improve fitness without evolving the ability to learn, evolution is likely to exploit domain-dependent static (i.e., nonadaptive) heuristics. This article analyzes this inherent deceptiveness in a variety of different dynamic, reward-based learning tasks, and proposes a way to escape the deceptive trap of static policies based on the novelty search algorithm. The main idea in novelty search is to abandon objective-based fitness and instead simply search only for novel behavior, which avoids deception entirely. A series of experiments and an in-depth analysis show how behaviors that could potentially serve as a stepping stone to finding adaptive solutions are discovered by novelty search yet are missed by fitness-based search. The conclusion is that novelty search has the potential to foster the emergence of adaptive behavior in reward-based learning tasks, thereby opening a new direction for research in evolving plastic ANNs.<\/jats:p>","DOI":"10.1177\/1059712310379923","type":"journal-article","created":{"date-parts":[[2010,10,4]],"date-time":"2010-10-04T23:24:27Z","timestamp":1286234667000},"page":"470-491","source":"Crossref","is-referenced-by-count":54,"title":["Evolving plastic neural networks with novelty search"],"prefix":"10.1177","volume":"18","author":[{"given":"Sebastian","family":"Risi","sequence":"first","affiliation":[{"name":"Evolutionary Complexity Research Group, Department of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA,"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charles E","family":"Hughes","sequence":"additional","affiliation":[{"name":"Evolutionary Complexity Research Group, Department of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kenneth O","family":"Stanley","sequence":"additional","affiliation":[{"name":"Evolutionary Complexity Research Group, Department of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2010,10,4]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.102.152001"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1109\/72.265960"},{"key":"atypb3","unstructured":"Barlow, M., Galloway, J. & Abbass, H.A. ( 2002). Mining evolution through visualization. In E. Bilotta et al (Eds.), A Life VIII Workshops (pp. 103-112). UNSW Press."},{"key":"atypb4","volume-title":"Proceedings of the 2001 Congress on Evolutionary Computation","author":"Barnett, L."},{"key":"atypb5","unstructured":"Baxter, J. ( 1992). The evolution of learning algorithms for artificial neural networks. In D. Green, & T. Bossomaier (Eds.), Complex systems (pp. 313-326). Amsterdam, The Netherlands: IOS Press."},{"key":"atypb6","doi-asserted-by":"crossref","unstructured":"Blynel, J. & Floreano, D. ( 2002). Levels of dynamics and adaptive behavior in evolutionary neural controllers. In B. Hallam, D. Floreano, J. Hallam, G. Hayes, & J.A. Meyer (Eds.), From Animals to Animats 7 (pp. 272-281). The MIT Press.","DOI":"10.7551\/mitpress\/3121.003.0045"},{"key":"atypb7","volume-title":"Exploring the T-maze: Evolving learning-like robot behaviors using CTRNNs","author":"Blynel, J.","year":"2003"},{"key":"atypb8","volume-title":"Proceedings of the 6th European Conference on Advances in Artificial Life (ECAL 2001)","author":"Bongard, J.C."},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.01-12-01426.1981"},{"key":"atypb10","doi-asserted-by":"crossref","unstructured":"Chalmers, D.J. ( 1990). The evolution of learning: An experiment in genetic connectionism . In D. S. Touretzky, & S. L. Elman (Eds.), In Proceedings of the 1990 Connectionist Models Summer School, (pp. 81-90). San Mateo, CA: Morgan Kaufmann.","DOI":"10.1016\/B978-1-4832-1448-1.50014-7"},{"key":"atypb11","doi-asserted-by":"crossref","unstructured":"Darwen, P. & Yao, Y. ( 1996). Every niching method has its niche: Fitness sharing and implicit sharing compared. In W. Ebeling , I Rechenberg, H.P. Schwefel , & H.M. Voigt (Eds.), Parallel Problem Solving from Nature (PPSN IV) (pp. 398-407).","DOI":"10.1007\/3-540-61723-X_1004"},{"key":"atypb12","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2004)","author":"De Jong, E.D."},{"key":"atypb13","doi-asserted-by":"crossref","unstructured":"D\u00fcrr, P., Mattiussi, C., Soltoggio, A. & Floreano, D. ( 2008). Evolvability of neuromodulated learning for robots . In A. O. El-Rayis, A. Stoica, E. Tunstel, T. Huntsberger, T. Arslan , & S. Vijayakumar (Eds.), The 2008 ECSIS Symposium on Learning and Adaptive Behavior in Robotic Systems (pp. 41-46). Los Alamitos, CA: IEEE Computer Society.","DOI":"10.1109\/LAB-RS.2008.22"},{"key":"atypb14","volume-title":"Proceedings of The 5th Annual Conference on Evolutionary Programming","author":"Dybowski, R."},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1007\/s12065-007-0002-4"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(00)00032-0"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-6089-0"},{"key":"atypb18","unstructured":"Goldberg, D.E. ( 1987). Simple genetic algorithms and the minimal deceptive problem . In L. D. Davis (Ed.), Genetic algorithms and simulated annealing, research notes in artificial intelligence (pp. 74-88). San Mateo, CA: Morgan Kaufmann."},{"key":"atypb19","volume-title":"Proceedings of the 2nd International Conference on Genetic Algorithms and their Application","author":"Goldberg, D.E."},{"key":"atypb20","volume-title":"Proceedings of the 16th International Joint Conference on Artificial Intelligence","author":"Gomez, F.J."},{"key":"atypb21","volume-title":"Proceedings of the 1st Annual Conference on Genetic Programming","author":"Gruau, F."},{"key":"atypb22","author":"Harvey, I.","year":"1993","journal-title":"School of Cognitive and Computing Sciences"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1007\/BF02481128"},{"key":"atypb24","first-page":"495","volume":"1","author":"Hinton, G.E.","year":"1987","journal-title":"Complex Systems"},{"key":"atypb25","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2006)","author":"Hornby, G.S."},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1162\/1063656054088530"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2005.863127"},{"key":"atypb28","first-page":"1136","volume":"2723","author":"Kim, Y.-H.","year":"2003","journal-title":"LNCS"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1016\/0031-3203(73)90025-3"},{"key":"atypb30","volume-title":"Proceedings of the 11th International Conference on Artificial Life","author":"Lehman, J."},{"key":"atypb31","author":"Lehman, J.","year":"2010","journal-title":"Evolutionary Computation"},{"key":"atypb32","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010)","author":"Lehman, J."},{"key":"atypb33","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010)","author":"Lehman, J."},{"key":"atypb34","author":"Mahfoud, S.W.","year":"1995","journal-title":"Unpublished doctoral dissertation"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1086\/303231"},{"key":"atypb36","volume-title":"4th European Conference on Artificial Life","author":"Mayley, G."},{"key":"atypb37","doi-asserted-by":"crossref","unstructured":"McHale, G. & Husbands, P. ( 2004). GasNets and other evolvable neural networks applied to bipedal locomotion. In S. Schaal, A. J. Ijspeert, A. Billard , S. Vijaykumar, J. Hallam, & J.A. Meyer (Eds.), From Animals to Animats (Vol 8, pp. 163-172). The MIT Press.","DOI":"10.7551\/mitpress\/3122.003.0022"},{"key":"atypb38","volume-title":"Proceedings of the 7th International Conference on Genetic Algorithms","author":"McQuesten, P."},{"key":"atypb39","volume-title":"Proceedings of the 1st European Conference on Artificial Life","author":"Mitchell, M."},{"key":"atypb40","volume-title":"Proceedings of IROS Workshop on Exploring New Horizons in the Evolutionary Design of Robots","author":"Mouret, J."},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1177\/1059-712302-010001-01"},{"key":"atypb42","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008973931182"},{"key":"atypb43","volume-title":"Proceedings of the 2nd European Conference on Artificial Life","author":"Nolfi, S."},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1177\/105971239600500104"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1177\/105971239400300102"},{"key":"atypb46","volume-title":"6th Eurographics Workshop on Animation and Simulation","author":"Panne, M., van de"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996015"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1109\/T-C.1969.222678"},{"issue":"3","key":"atypb49","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/64.393139","volume":"10","author":"Saravanan, N.","year":"1995","journal-title":"IEEE Expert: Intelligent Systems and Their Applications"},{"key":"atypb50","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-18965-4_23"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1080\/09540090600768658"},{"key":"atypb52","unstructured":"Soltoggio, A., Bullinaria, J.A., Mattiussi, C., D\u00fcrr, P. & Floreano, D. ( 2008). Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In S. Bullock, J. Noble, R. Watson, & M. A. Bedau (Eds.), Artificial Life XI (pp. 569-576). Cambridge, MA: MIT Press."},{"key":"atypb53","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2007.4424781"},{"key":"atypb54","volume-title":"rtNEAT C++ software home-page","author":"Stanley, K.O.","year":"2006"},{"key":"atypb55","volume-title":"Proceedings of the 2003 IEEE Congress on Evolutionary Computation (CEC-2003)","author":"Stanley, K.O."},{"key":"atypb56","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2005.856210"},{"key":"atypb57","doi-asserted-by":"publisher","DOI":"10.1162\/106365602320169811"},{"key":"atypb58","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1338"},{"key":"atypb59","volume-title":"Proceedings of the 2001 IEEE International Conference on Evolutionary Computation","author":"Stewart, T."},{"key":"atypb60","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2006)","author":"Taylor, M.E."},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1162\/106365600568095"},{"key":"atypb62","volume-title":"Molecular biology of the gene","author":"Watson, J.D.","year":"1987","edition":"4"},{"key":"atypb63","first-page":"877","volume":"7","author":"Whiteson, S.","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"atypb64","doi-asserted-by":"publisher","DOI":"10.1109\/5.784219"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310379923","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310379923","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:58Z","timestamp":1777392958000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1059712310379923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,10,4]]},"references-count":64,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2010,12]]}},"alternative-id":["10.1177\/1059712310379923"],"URL":"https:\/\/doi.org\/10.1177\/1059712310379923","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2010,10,4]]}}}