{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T09:57:01Z","timestamp":1777543021488,"version":"3.51.4"},"reference-count":33,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2011,1,13]],"date-time":"2011-01-13T00:00:00Z","timestamp":1294876800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2011,2]]},"abstract":"<jats:p>This article develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowerment measures, for agent\u2014environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control\/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, for example, it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this article is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning.<\/jats:p>","DOI":"10.1177\/1059712310392389","type":"journal-article","created":{"date-parts":[[2011,1,13]],"date-time":"2011-01-13T23:50:44Z","timestamp":1294962644000},"page":"16-39","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":42,"title":["Empowerment for continuous agent\u2014environment systems"],"prefix":"10.1177","volume":"19","author":[{"given":"Tobias","family":"Jung","sequence":"first","affiliation":[{"name":"Department of Computer Science, The University of Texas at Austin, USA,"}]},{"given":"Daniel","family":"Polani","sequence":"additional","affiliation":[{"name":"Adaptive Systems and Algorithms Research Groups, School of Computer Science, University of Hertfordshire, UK"}]},{"given":"Peter","family":"Stone","sequence":"additional","affiliation":[{"name":"Department of Computer Science, The University of Texas at Austin, USA"}]}],"member":"179","published-online":{"date-parts":[[2011,1,13]]},"reference":[{"key":"atypb1","volume-title":"Artificial Life XI: Proceedings of the 11th International Conference on the Simulation and Synthesis of Living Systems","author":"Anthony, T."},{"key":"atypb2","volume-title":"Proceedings of the European Conference on Artificial Life 2009","author":"Anthony, T."},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1140\/epjb\/e2008-00175-0"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1972.1054855"},{"key":"atypb5","first-page":"213","volume":"3","author":"Brafman, R.","year":"2002","journal-title":"Journal of Machine Learning Research"},{"key":"atypb6","unstructured":"Der, R. ( 2000). Self-organized robot behavior from the principle of homeokinesis . In H.M. Gro\u00df, K. Debes & H.J. B\u00f6hme (Eds.), Proceedings of the Workshop SOAVE 2000 (selbstorganisation von adaptivem verhalten) (Vol. 643, pp. 39-46). Ilmenau: VDI Verlag."},{"key":"atypb7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12064-001-0027-7","volume":"120","author":"Der, R.","year":"2001","journal-title":"Theory in Biosciences"},{"key":"atypb8","first-page":"43","volume":"55","author":"Der, R.","year":"1999","journal-title":"Computational intelligence for modelling, control, and automation"},{"key":"atypb9","volume-title":"Proceedings of 15th International Conference on Machine Learning","author":"Dietterich, T.G."},{"key":"atypb10","first-page":"503","volume":"6","author":"Ernst, D.","year":"2005","journal-title":"Journal of Machine Learning Research"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2009.04.005"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jphysparis.2006.10.001"},{"key":"atypb13","first-page":"529","volume":"15","author":"Girard, A.","year":"2003","journal-title":"Proceedings of Advances in Neural Information Processing Systems"},{"key":"atypb14","first-page":"259","volume":"3139","author":"Kaplan, F.","year":"2004","journal-title":"LNAI"},{"key":"atypb15","volume-title":"Advances in Artificial Life, European Conference on Artificial Life (ECAL 2005)","author":"Klyubin, A.S."},{"key":"atypb16","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2005)","author":"Klyubin, A.S."},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0004018"},{"key":"atypb18","first-page":"1107","volume":"4","author":"Lagoudakis, M.G.","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1385\/NI:3:3:243"},{"key":"atypb20","volume-title":"Proceedings of 4th IEEE International Conference on Development and Learning","author":"Lungarella, M."},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.0020144"},{"key":"atypb22","volume-title":"From Animals to Animats 9: 9th International Conference on the Simulation of Adaptive Behavior (SAB 2006)","author":"Prokopenko, M."},{"key":"atypb23","doi-asserted-by":"crossref","unstructured":"Qui\u00f1onero-Candela, J., Rasmussen, C.E. & Williams, C.K.I. (2007). Approximation methods for Gaussian process regression. In L. Bottou , O. Chapelle, D. DeCoste & J. Weston (Eds.), Large scale learning machines (pp. 203-223). Cambridge, MA : MIT Press.","DOI":"10.7551\/mitpress\/7496.003.0011"},{"key":"atypb24","volume-title":"Gaussian processes for machine learning","author":"Rasmussen, C.E.","year":"2006"},{"key":"atypb25","volume-title":"Proceedings of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats","author":"Schmidhuber, J."},{"key":"atypb26","first-page":"1281","volume":"17","author":"Singh, S.","year":"2005","journal-title":"Proceedings of Advances in Neural Information Processing Systems"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1109\/37.341864"},{"key":"atypb28","unstructured":"Sporns, O. & Lungarella, M. (2006). Evolving coordinated behavior by maximizing information structure. In L. M. Rocha, M. Bedau, D. Floreano, R. Goldstone , A. Vespignani & L. Yaeger (Eds.), Proceedings of Artificial Life X (pp. 323-329). MIT Press\/ Bradford Books."},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-27833-7_17"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/85\/28005"},{"key":"atypb31","volume-title":"Reinforcement learning: An introduction","author":"Sutton, R.","year":"1998"},{"key":"atypb32","doi-asserted-by":"crossref","volume-title":"Information theory of decisions and actions","author":"Tishby, N.","DOI":"10.1007\/978-1-4419-1452-1_19"},{"key":"atypb33","doi-asserted-by":"publisher","DOI":"10.1177\/1059712310375314"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310392389","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712310392389","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:58Z","timestamp":1777392958000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1059712310392389"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,1,13]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2011,2]]}},"alternative-id":["10.1177\/1059712310392389"],"URL":"https:\/\/doi.org\/10.1177\/1059712310392389","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,1,13]]}}}