{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T10:02:21Z","timestamp":1760608941252},"reference-count":60,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Evolutionary Computation"],"published-print":{"date-parts":[[2016,9]]},"abstract":"<jats:p>The Achilles Heel of stochastic optimization algorithms is getting trapped on local optima. Novelty Search mitigates this problem by encouraging exploration in all interesting directions by replacing the performance objective with a reward for novel behaviors. This reward for novel behaviors has traditionally required a human-crafted, behavioral distance function. While Novelty Search is a major conceptual breakthrough and outperforms traditional stochastic optimization on certain problems, it is not clear how to apply it to challenging, high-dimensional problems where specifying a useful behavioral distance function is difficult. For example, in the space of images, how do you encourage novelty to produce hawks and heroes instead of endless pixel static? Here we propose a new algorithm, the Innovation Engine, that builds on Novelty Search by replacing the human-crafted behavioral distance with a Deep Neural Network (DNN) that can recognize interesting differences between phenotypes. The key insight is that DNNs can recognize similarities and differences between phenotypes at an abstract level, wherein novelty means interesting novelty. For example, a DNN-based novelty search in the image space does not explore in the low-level pixel space, but instead creates a pressure to create new types of images (e.g., churches, mosques, obelisks, etc.). Here, we describe the long-term vision for the Innovation Engine algorithm, which involves many technical challenges that remain to be solved. We then implement a simplified version of the algorithm that enables us to explore some of the algorithm\u2019s key motivations. Our initial results, in the domain of images, suggest that Innovation Engines could ultimately automate the production of endless streams of interesting solutions in any domain: for example, producing intelligent software, robot controllers, optimized physical components, and art.<\/jats:p>","DOI":"10.1162\/evco_a_00189","type":"journal-article","created":{"date-parts":[[2016,7,1]],"date-time":"2016-07-01T14:56:20Z","timestamp":1467384980000},"page":"545-572","source":"Crossref","is-referenced-by-count":29,"title":["Understanding Innovation Engines: Automated Creativity and Improved Stochastic Optimization via Deep Learning"],"prefix":"10.1162","volume":"24","author":[{"given":"A.","family":"Nguyen","sequence":"first","affiliation":[{"name":"University of Wyoming"}]},{"given":"J.","family":"Yosinski","sequence":"additional","affiliation":[{"name":"Cornell University & Geometric Intelligence"}]},{"given":"J.","family":"Clune","sequence":"additional","affiliation":[{"name":"University of Wyoming"}]}],"member":"281","reference":[{"key":"B1","first-page":"629","volume-title":"Artificial Life 13","author":"Auerbach J. E","year":"2012"},{"key":"B2","volume-title":"Foundations of genetic algorithms","author":"Belew R. K.","year":"1995"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1561\/2200000006"},{"key":"B4","first-page":"226","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Bengio Y.","year":"2014"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1145\/2463372.2463404"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2009.4983289"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1145\/2078245.2078246"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1098\/rspb.2012.2863"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2010.2104157"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-20525-5_24"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1038\/nature14422"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1000292"},{"key":"B15","volume-title":"Bio-inspired artificial intelligence: Theories, methods, and technologies","author":"Floreano D.","year":"2008"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/ICEC.1996.542328"},{"key":"B17","first-page":"2:74","author":"Getreuer P.","year":"2012","journal-title":"Image Processing On Line"},{"key":"B18","first-page":"3: 139","author":"Grefenstette J. J.","year":"2014","journal-title":"Foundations of Genetic Algorithms"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/TRA.2003.814502"},{"key":"B21","first-page":"3: 191","author":"Igel C.","year":"1999","journal-title":"Advances in Genetic Programming"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.1999.785505"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.223"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0503610102"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0611630104"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2015.02.001"},{"key":"B29","first-page":"6: 231","author":"Koza J. R.","year":"2005","journal-title":"Genetic Programming and Evolvable Machines"},{"key":"B30","first-page":"1097","volume":"25","author":"Krizhevsky A.","year":"2012","journal-title":"Advances in neural information processing systems"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"B33","doi-asserted-by":"crossref","unstructured":"Lee, S., Yosinski, J., Glette, K., Lipson, H., and Clune, J. (2013).Evolving gaits for physical robots with the HyperNEAT Generative Encoding: The benefits of simulation. A. Esparcia-Alc\u00e1zar (Ed.), pp. 540\u2013549. Berlin Heidelberg: Springer.","DOI":"10.1007\/978-3-642-37192-9_54"},{"key":"B34","first-page":"329","volume-title":"Proceedings of the Eleventh International Conference on Artificial Life","author":"Lehman J.","year":"2008"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00025"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-1770-5_3"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.100.118703"},{"key":"B38","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1145\/2576768.2598222","author":"Li J.","year":"2014","journal-title":"Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation"},{"key":"B39","first-page":"56","volume-title":"Proceedings of the Fourth International Conference on Computational Creativity","author":"Liapis A.","year":"2013"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-18272-3_10"},{"key":"B41","author":"Mouret J.-B.","year":"2015","journal-title":"arXiv preprint arXiv:1504.04909"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2010.5586158"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0601602103"},{"key":"B44","author":"Nguyen A.","year":"2016","journal-title":"arXiv preprint arXiv:1605.09304"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1145\/2739480.2754703"},{"key":"B47","volume-title":"Visualization Workshop, International Conference on Machine Learning","author":"Nguyen A.","year":"2016"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(92)90242-F"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1080\/09540090600768658"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1162\/EVCO_a_00030"},{"key":"B52","first-page":"568","volume":"27","author":"Simonyan K.","year":"2014","journal-title":"Advances in neural information processing systems"},{"key":"B53","first-page":"32: 323","author":"Stallkamp J.","year":"2012","journal-title":"Neural Networks"},{"key":"B54","author":"Stanley K.","year":"2016","journal-title":"In Preparation"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-15524-1"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1162\/106365602320169811"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-007-9028-8"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"B60","first-page":"1","volume-title":"International Conference on Learning Representations","author":"Szegedy C.","year":"2014"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1145\/2001576.2001707"},{"key":"B62","volume-title":"Deep Learning Workshop, International Conference on Machine Learning","author":"Yosinski J.","year":"2015"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_53"}],"container-title":["Evolutionary Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/EVCO_a_00189","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,2]],"date-time":"2022-07-02T14:52:29Z","timestamp":1656773549000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/evco\/article\/24\/3\/545-572\/1020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,9]]},"references-count":60,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,9]]}},"alternative-id":["10.1162\/EVCO_a_00189"],"URL":"https:\/\/doi.org\/10.1162\/evco_a_00189","relation":{},"ISSN":["1063-6560","1530-9304"],"issn-type":[{"value":"1063-6560","type":"print"},{"value":"1530-9304","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,9]]}}}