{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T01:39:40Z","timestamp":1792460380022,"version":"4.5.0"},"reference-count":51,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2014,5,21]],"date-time":"2014-05-21T00:00:00Z","timestamp":1400630400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>One aspect of intelligence is the ability to restructure your own environment so that the world you live in becomes more beneficial to you. In this paper we investigate how the information-theoretic measure of agent empowerment can provide a task-independent, intrinsic motivation to restructure the world. We show how changes in embodiment and in the environment change the resulting behaviour of the agent and the artefacts left in the world. For this purpose, we introduce an approximation of the established empowerment formalism based on sparse sampling, which is simpler and significantly faster to compute for deterministic dynamics. Sparse sampling also introduces a degree of randomness into the decision making process, which turns out to beneficial for some cases. We then utilize the measure to generate agent behaviour for different agent embodiments in a Minecraft-inspired three dimensional block world. The paradigmatic results demonstrate that empowerment can be used as a suitable generic intrinsic motivation to not only generate actions in given static environments, as shown in the past, but also to modify existing environmental conditions. In doing so, the emerging strategies to modify an agent\u2019s environment turn out to be meaningful to the specific agent capabilities, i.e., de facto to its embodiment.<\/jats:p>","DOI":"10.3390\/e16052789","type":"journal-article","created":{"date-parts":[[2014,5,21]],"date-time":"2014-05-21T10:38:31Z","timestamp":1400668711000},"page":"2789-2819","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Changing the Environment Based on Empowerment as Intrinsic Motivation"],"prefix":"10.3390","volume":"16","author":[{"given":"Christoph","family":"Salge","sequence":"first","affiliation":[{"name":"Adaptive Systems Research Group, University of Hertfordshire, College Lane,Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cornelius","family":"Glackin","sequence":"additional","affiliation":[{"name":"Adaptive Systems Research Group, University of Hertfordshire, College Lane,Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Polani","sequence":"additional","affiliation":[{"name":"Adaptive Systems Research Group, University of Hertfordshire, College Lane,Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2014,5,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1006\/ceps.1999.1020","article-title":"Intrinsic and extrinsic motivations: Classic definitions and new directions","volume":"25","author":"Ryan","year":"2000","journal-title":"Contemp. Educ. Psychol"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/TEVC.2006.890271","article-title":"Intrinsic motivation systems for autonomous mental development","volume":"11","author":"Oudeyer","year":"2007","journal-title":"IEEE Trans. Evol. Comput"},{"key":"ref_3","unstructured":"Klyubin, A., Polani, D., and Nehaniv, C. (2005, January 2\u20134). Empowerment: A universal agent-centric measure of control, Edinburgh, UK."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1177\/1745691610393528","article-title":"Evolutionary theory and the ultimate\u2013Proximate distinction in the human behavioral sciences","volume":"6","author":"Dickins","year":"2011","journal-title":"Perspect. Psychol. Sci"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Varela, F., Thompson, E., and Rosch, E. (1992). The Embodied Mind: Cognitive Science and Human Experience, The MIT Press.","DOI":"10.7551\/mitpress\/6730.001.0001"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Gallagher, S. (2005). How the Body Shapes the Mind, Cambridge University Press.","DOI":"10.1093\/0199271941.001.0001"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pfeifer, R., Bongard, J., and Grand, S. (2007). How the Body Shapes the Way We Think: A New View of Intelligence, The MIT Press.","DOI":"10.7551\/mitpress\/3585.001.0001"},{"key":"ref_9","unstructured":"Von Uexku\u00a8ll, J. (1909). Umwelt und Innenwelt der Tiere, Springer."},{"key":"ref_10","unstructured":"Gibson James, J. (1979). The Ecological Approach to Visual Perception, Houghton Mifflin."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wright, K.R., and Zegarra, A.V. (2000). Machu Picchu: A Civil Engineering Marvel, ASCE.","DOI":"10.1061\/9780784404447"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6","DOI":"10.3389\/neuro.12.006.2007","article-title":"What is intrinsic motivation? A typology of computational approaches","volume":"1","author":"Oudeyer","year":"2007","journal-title":"Front. Neurorobot"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J. (1991, January 18\u201321). Curious model-building control systems, Singapore.","DOI":"10.1109\/IJCNN.1991.170605"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J. (2010). Formal theory of creativity, fun, and intrinsic motivation (1990\u20132010). IEEE Trans. Autonom. Ment. Dev, 230\u2013247.","DOI":"10.1109\/TAMD.2010.2056368"},{"key":"ref_15","first-page":"629","article-title":"The autotelic principle","volume":"3139","author":"Steels","year":"2004","journal-title":"Embodied Artificial Intelligence"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Csikszentmihalyi, M. (2000). Beyond Boredom and Anxiety, Jossey-Bass.","DOI":"10.1037\/10516-164"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.neunet.2012.02.024","article-title":"Hierarchical curiosity loops and active sensing","volume":"32","author":"Gordon","year":"2012","journal-title":"Neur. Netw"},{"key":"ref_18","first-page":"259","article-title":"Maximizing learning progress: An internal reward system for development","volume":"3139","author":"Kaplan","year":"2004","journal-title":"Embodied Artificial Intelligence"},{"key":"ref_19","unstructured":"Mohammadian, M. (1999). Computational Intelligence for Modelling, Control and Automation, IOS Press."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ay, N., Bertschinger, N., Der, R., G\u00fcttler, F., and Olbrich, E. (2008). Predictive information and explorative behavior of autonomous robots. Eur. Phys. J. B-Cond. Matter Complex Syst, 329\u2013339.","DOI":"10.1140\/epjb\/e2008-00175-0"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Der, R., and Martius, G. (2012). The Playful Machine, Springer.","DOI":"10.1007\/978-3-642-20253-7"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e4018","DOI":"10.1371\/journal.pone.0004018","article-title":"Keep your options open: An information-based driving principle for sensorimotor systems","volume":"3","author":"Klyubin","year":"2008","journal-title":"PLoS One"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Prokopenko, M. (2014). Guided Self-Organization: Inception, Springer.","DOI":"10.1007\/978-3-642-53734-9"},{"key":"ref_24","unstructured":"Oesterreich, R. (1979). Entwicklung eines Konzepts der objectiven Kontrolle und Kontrollkompetenz. Ein handlungstheoretischer Ansatz. [Ph.D. Thesis, Technische Universit\u00e4t Berlin]."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Von Foerster, H. (2003). Understanding Understanding, Springer.","DOI":"10.1007\/b97451"},{"key":"ref_26","unstructured":"Seligman, M.E. (1975). Helplessness: On depression, Development, and Death, WH Freeman\/Times Books\/Henry Holt & Co."},{"key":"ref_27","unstructured":"Trendafilov, D., and Murray-Smith, R. (, January October). Information-Theoretic Characterization of Uncertainty in Manual Control, Manchester, UK."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"168702","DOI":"10.1103\/PhysRevLett.110.168702","article-title":"Causal entropic forces","volume":"110","author":"Freer","year":"2013","journal-title":"Phys. Rev. Lett"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1142\/S0219525908001465","article-title":"Information flows in causal networks","volume":"11","author":"Ay","year":"2008","journal-title":"Adv. Complex Syst"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1002\/j.1538-7305.1948.tb00917.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J"},{"key":"ref_31","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley-Interscience. [99th ed]."},{"key":"ref_32","unstructured":"Pearl, J. (2000). Causality: Models, Reasoning and Inference, Cambridge University Press."},{"key":"ref_33","unstructured":"Bullock, S., Noble, J., Watson, R., and Bedau, M.A. (, January August). On preferred states of agents: How global structure is reflected in local structure. Artificial Life XI, Winchester, UK."},{"key":"ref_34","first-page":"39","article-title":"Empowerment for continuous agent environment systems","volume":"1","author":"Jung","year":"2011","journal-title":"Adapt. Behav"},{"key":"ref_35","unstructured":"Leu, A., Ristic-Durrant, D., Slavnic, S., Glackin, C., Salge, C., Polani, D., Badii, A., Khan, A., and Raval, R. (, January December). CORBYS cognitive control architecture for robotic follower, Kobe, Japan."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1250079","DOI":"10.1142\/S0219525912500798","article-title":"Approximation of empowerment in the continuous domain","volume":"16","author":"Salge","year":"2012","journal-title":"Adv. Complex Syst"},{"key":"ref_37","first-page":"118","article-title":"Empowerment and State-dependent Noise-An Intrinsic Motivation for Avoiding Unpredictable Agents","volume":"12","author":"Salge","year":"2013","journal-title":"Advances in Artificial Life"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1023\/A:1010388907793","article-title":"Computational mechanics: Pattern and prediction, structure and simplicity","volume":"104","author":"Shalizi","year":"2001","journal-title":"J. Statist. Phys"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1109\/TIT.1972.1054855","article-title":"Computation of channel capacity and rate-distortion functions","volume":"18","author":"Blahut","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/TIT.1972.1054753","article-title":"An algorithm for computing the capacity of arbitrary discrete memoryless channels","volume":"18","author":"Arimoto","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_41","unstructured":"Anthony, T., Polani, D., and Nehaniv, C. (, January September). Impoverished empowerment: \u201cMeaningful\u201d action sequence generation through bandwidth limitation. Advances in Artificial Life, Darwin Meets von Neumann, ECAL, Budapest, Hungary."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2013.2295372","article-title":"General self-motivation and strategy identification: Case studies based on Sokoban and Pac-Man","volume":"6","author":"Anthony","year":"2013","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1002\/ett.4460100604","article-title":"Capacity of multi-antenna Gaussian channels","volume":"10","author":"Telatar","year":"1999","journal-title":"Eur. Trans. Telecommun"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A survey of monte carlo tree search methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_45","unstructured":"Persson, M. (2011). Minecraft. Mojang."},{"key":"ref_46","unstructured":"Kim, J.T., and Polani, D. (, January March). Exploring empowerment as a basis for quantifying sustainability, Nashville, USA."},{"key":"ref_47","first-page":"268","article-title":"Environmental Feedback Drives Multiple Behaviors from the Same Neural Circuit","volume":"12","author":"Williams","year":"2013","journal-title":"Advances in Artificial Life"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.3390\/e15051887","article-title":"Quantifying morphological computation","volume":"15","author":"Zahedi","year":"2013","journal-title":"Entropy"},{"key":"ref_49","unstructured":"Capdepuy, P., Polani, D., and Nehaniv, C. (, January April). Maximization of potential information flow as a universal utility for collective behaviour, Honolulu, USA."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/s12064-011-0143-y","article-title":"Perception-action loops of multiple agents: Informational aspects and the impact of coordination","volume":"131","author":"Capdepuy","year":"2012","journal-title":"Theory Biosci"},{"key":"ref_51","unstructured":"Brodbeck, L., Wang, L., and Iida, F. (, January May). Robotic body extension based on hot melt adhesives, St. Paul, USA."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2789\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:11:37Z","timestamp":1760217097000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2789"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,5,21]]},"references-count":51,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2014,5]]}},"alternative-id":["e16052789"],"URL":"https:\/\/doi.org\/10.3390\/e16052789","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,5,21]]}}}