{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,3,2]],"date-time":"2024-03-02T07:02:01Z","timestamp":1709362921530},"reference-count":68,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2016,8]]},"abstract":"<jats:p> Consider a self-motivated artificial agent who is exploring a complex environment. Part of the complexity is due to the raw high-dimensional sensory input streams, which the agent needs to make sense of. Such inputs can be compactly encoded through a variety of means; one of these is slow feature analysis (SFA). Slow features encode spatiotemporal regularities, which are information-rich explanatory factors (latent variables) underlying the high-dimensional input streams. In our previous work, we have shown how slow features can be learned incrementally, while the agent explores its world, and modularly, such that different sets of features are learned for different parts of the environment (since a single set of regularities does not explain everything). In what order should the agent explore the different parts of the environment? Following Schmidhuber\u2019s theory of artificial curiosity, the agent should always concentrate on the area where it can learn the easiest-to-learn set of features that it has not already learned. We formalize this learning problem and theoretically show that, using our model, called curiosity-driven modular incremental slow feature analysis, the agent on average will learn slow feature representations in order of increasing learning difficulty, under certain mild conditions. We provide experimental results to support the theoretical analysis. <\/jats:p>","DOI":"10.1162\/neco_a_00855","type":"journal-article","created":{"date-parts":[[2016,6,27]],"date-time":"2016-06-27T18:29:53Z","timestamp":1467052193000},"page":"1599-1662","source":"Crossref","is-referenced-by-count":3,"title":["Optimal Curiosity-Driven Modular Incremental Slow Feature Analysis"],"prefix":"10.1162","volume":"28","author":[{"given":"Varun Raj","family":"Kompella","sequence":"first","affiliation":[{"name":"IDSIA, SUPSI, USI, Galleria 2, Manno-Lugano 6928, Switzerland"}]},{"given":"Matthew","family":"Luciw","sequence":"additional","affiliation":[{"name":"IDSIA, SUPSI, USI, Galleria 2, Manno-Lugano 6928, Switzerland"}]},{"given":"Marijn Frederik","family":"Stollenga","sequence":"additional","affiliation":[{"name":"IDSIA, SUPSI, USI, Galleria 2, Manno-Lugano 6928, Switzerland"}]},{"given":"Juergen","family":"Schmidhuber","sequence":"additional","affiliation":[{"name":"IDSIA, SUPSI, USI, Galleria 2, Manno-Lugano 6928, Switzerland"}]}],"member":"281","reference":[{"key":"B1","author":"Abut H.","year":"1990","journal-title":"Vector quantization"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1090\/qam\/42792"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1080\/net.12.3.241.253"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1023\/A:1011388608203"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1016\/0165-1684(94)90029-9"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.2.194"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.0030166"},{"key":"B8","first-page":"197","author":"G\u00e1bor Z.","year":"1998","journal-title":"Proceedings of the Fifteenth International Conference on Machine Learning"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22887-2_4"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1162\/089976699300016755"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128018"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-3064-3_6"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1145\/1015330.1015357"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-1904-8"},{"key":"B15","first-page":"684","author":"Klapper-Rybicka M.","year":"2001","journal-title":"Lecture Notes on Comp. Sci. 2130, Proc. Intl. Conf. on Artificial Neural Networks (ICANN-2001)"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-56927-2"},{"key":"B17","first-page":"1354","author":"Kompella V. R.","year":"2011","journal-title":"Proc. 20th International Joint Conference of Artificial Intelligence"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00344"},{"key":"B19","first-page":"1","author":"Kompella V. R.","year":"2012","journal-title":"Proc. of the Joint Conference on Development and Learning and Epigenetic Robotics"},{"key":"B20","first-page":"622","author":"Kompella V. R.","year":"2011","journal-title":"IEEE-RAS International Conference on Humanoid Robots"},{"key":"B21","first-page":"2705","author":"Kompella V. R.","year":"2014","journal-title":"International Joint Conference on Neural Networks"},{"key":"B22","author":"Kompella V. R.","journal-title":"Artificial Intelligence"},{"key":"B23","first-page":"1015","volume":"22","author":"Konidaris G.","year":"2009","journal-title":"Advances in neural information processing systems"},{"key":"B24","first-page":"1162","volume":"23","author":"Konidaris G.","year":"2010","journal-title":"Advances in neural information processing systems"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1162\/jmlr.2003.4.6.1107"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1038\/44565"},{"key":"B27","first-page":"1096","volume":"22","author":"Lee H.","year":"2009","journal-title":"Advances in neural information processing systems"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000894"},{"key":"B29","first-page":"308","volume-title":"Proc. of the 1993 Connectionist Models Summer School","author":"Lindst\u00e4dt S.","year":"1993"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2013.00009"},{"key":"B31","author":"Luciw M.","year":"2012","journal-title":"Workshop on Deep Hierarchies in Vision"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/BF00114727"},{"issue":"2169","key":"B33","volume":"8","author":"Mahadevan S.","year":"2007","journal-title":"Journal of Machine Learning Research"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.3.312"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2011.2160943"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80089-9"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2012.00006"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.5019\/j.ijcir.2006.63"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2007.07.001"},{"key":"B40","first-page":"2574","author":"Petrik M.","year":"2007","journal-title":"Proceedings of the Joint Conference on Artificial Intelligence"},{"key":"B41","author":"Ring M. B.","year":"1994","journal-title":"Continual learning in reinforcement environments"},{"key":"B42","first-page":"1458","author":"Schmidhuber J.","year":"1991","journal-title":"Proceedings of the International Joint Conference on Neural Networks, Singapore"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.2.234"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.6.863"},{"key":"B45","first-page":"291","volume-title":"Advances in neural information processing systems, 4","author":"Schmidhuber J.","year":"1992"},{"key":"B46","first-page":"1612","author":"Schmidhuber J.","year":"1999","journal-title":"Proceedings of the Congress on Evolutionary Computation"},{"key":"B47","author":"Schmidhuber J.","year":"1999","journal-title":"Adaptive behavior and learning"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1080\/09540090600768658"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1080\/09540090600768658"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2010.2056368"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2010.2056368"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2013.00313"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00214"},{"key":"B54","first-page":"921","volume":"15","author":"Sprekeler H.","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2013.01.022"},{"key":"B56","first-page":"159","volume":"2","author":"Storck J.","year":"1995","journal-title":"Proceedings of the International Conference on Artificial Neural Networks, Paris"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.336"},{"key":"B58","author":"Sutton R. S.","year":"1998","journal-title":"Reinforcement learning: An introduction"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.336"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-010-5232-5"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1016\/S0301-0082(96)00054-8"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2003.1217609"},{"key":"B64","author":"Wiskott L.","year":"2003","journal-title":"Estimating driving forces of nonstationary time series with slow feature analysis"},{"key":"B65","doi-asserted-by":"publisher","DOI":"10.1162\/089976602317318938"},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.1007\/s11063-004-2793-y"},{"key":"B67","author":"Zhang Y.","year":"2001","journal-title":"Convergence analysis of complementary candid incremental principal component analysis"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.157"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00855","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:41:22Z","timestamp":1615585282000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/28\/8\/1599-1662\/8194"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,8]]},"references-count":68,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2016,8]]}},"alternative-id":["10.1162\/NECO_a_00855"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00855","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,8]]}}}