{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T04:16:05Z","timestamp":1776831365583,"version":"3.51.2"},"reference-count":65,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,3,30]],"date-time":"2019-03-30T00:00:00Z","timestamp":1553904000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>How do living organisms decide and act with limited and uncertain information? Here, we discuss two computational approaches to solving these challenging problems: a \u201ccognitive\u201d and a \u201csensorimotor\u201d enrichment of stimuli, respectively. In both approaches, the key notion is that agents can strategically modulate their behavior in informative ways, e.g., to disambiguate amongst alternative hypotheses or to favor the perception of stimuli providing the information necessary to later act appropriately. We discuss how, despite their differences, both approaches appeal to the notion that actions must obey both epistemic (i.e., information-gathering or uncertainty-reducing) and pragmatic (i.e., goal- or reward-maximizing) imperatives and balance them. Our computationally-guided analysis reveals that epistemic behavior is fundamental to understanding several facets of cognitive processing, including perception, decision making, and social interaction.<\/jats:p>","DOI":"10.3390\/e21040350","type":"journal-article","created":{"date-parts":[[2019,4,3]],"date-time":"2019-04-03T03:39:28Z","timestamp":1554262768000},"page":"350","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Making the Environment an Informative Place: A Conceptual Analysis of Epistemic Policies and Sensorimotor Coordination"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6813-8282","authenticated-orcid":false,"given":"Giovanni","family":"Pezzulo","sequence":"first","affiliation":[{"name":"Institute of Cognitive Sciences and Technologies, National Research Council, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefano","family":"Nolfi","sequence":"additional","affiliation":[{"name":"Institute of Cognitive Sciences and Technologies, National Research Council, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,3,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1080\/17588928.2015.1020053","article-title":"Active inference and epistemic value","volume":"6","author":"Friston","year":"2015","journal-title":"Cogn. Neurosci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active inference and learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Carvalho, J.T., and Nolfi, S. (2016). Cognitive Offloading Does Not Prevent but Rather Promotes Cognitive Development. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0160679"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to Predict by the Methods of Temporal Differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1162\/neco.1993.5.4.613","article-title":"Improving generalization for temporal difference learning: The successor representation","volume":"5","author":"Dayan","year":"1993","journal-title":"Neural Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1162\/089976606775093909","article-title":"Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia","volume":"18","author":"Frank","year":"2006","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","article-title":"Planning and acting in partially observable stochastic domains","volume":"101","author":"Kaelbling","year":"1998","journal-title":"Artif. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/j.tics.2012.08.006","article-title":"Planning as inference","volume":"16","author":"Botvinick","year":"2012","journal-title":"Trends Cogn. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s10994-012-5278-7","article-title":"Optimal control as a graphical model inference problem","volume":"87","author":"Kappen","year":"2012","journal-title":"Mach. Learn."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO_a_00912","article-title":"Active Inference: A Process Theory","volume":"29","author":"Friston","year":"2016","journal-title":"Neural Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pezzulo, G., Donnarumma, F., Iodice, P., Maisto, D., and Stoianov, I. (2017). Model-Based Approaches to Active Perception and Control. Entropy, 19.","DOI":"10.3390\/e19060266"},{"key":"ref_19","unstructured":"Roy, N., and Thrun, S. (2000). Coastal Navigation with Mobile Robots. Advances in Neural Information Processing Systems 12, MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1101\/lm.041780.116","article-title":"Active Inference, epistemic value, and vicarious trial and error","volume":"23","author":"Pezzulo","year":"2016","journal-title":"Learn. Mem."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1162\/neco_a_00999","article-title":"Active Inference, Curiosity and Insight","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.pneurobio.2015.09.001","article-title":"Active Inference, homeostatic regulation and adaptive behavioural control","volume":"136","author":"Pezzulo","year":"2015","journal-title":"Prog. Neurobiol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1037\/h0070405","article-title":"The Reflex Arc Concept in Psychology","volume":"3","author":"Dewey","year":"1896","journal-title":"Psychol. Rev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/S0925-2312(01)00598-7","article-title":"Power and the limits of reactive agents","volume":"42","author":"Nolfi","year":"2002","journal-title":"Neurocomputing"},{"key":"ref_25","unstructured":"Cohen, H., and Lefebvre, C. (2005). Chapter 39\u2014Category Formation in Self-Organizing Embodied Agents. Handbook of Categorization in Cognitive Science, Elsevier Science Ltd."},{"key":"ref_26","first-page":"443","article-title":"Behavior and Cognition as a Complex Adaptive System: Insights from Robotic Experiments","volume":"Volume 10","author":"Hooker","year":"2011","journal-title":"Philosophy of Complex Systems"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Siciliano, B., and Khatib, O. (2016). Evolutionary Robotics. Springer Handbook of Robotics, Springer International Publishing. Springer Handbooks.","DOI":"10.1007\/978-3-319-32552-1"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"625","DOI":"10.3758\/BF03196322","article-title":"Six views of embodied cognition","volume":"9","author":"Wilson","year":"2002","journal-title":"Psychon. Bull. Rev."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.concog.2015.05.014","article-title":"Storing information in-the-world: Metacognition and cognitive offloading in a short-term memory task","volume":"36","author":"Risko","year":"2015","journal-title":"Conscious. Cogn."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1080\/17470218.2014.972963","article-title":"Strategic offloading of delayed intentions into the external environment","volume":"68","author":"Gilbert","year":"2015","journal-title":"Q. J. Exp. Psychol. 2006"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.concog.2015.01.006","article-title":"Strategic use of reminders: Influence of both domain-general and task-specific metacognitive confidence, independent of objective memory ability","volume":"33","author":"Gilbert","year":"2015","journal-title":"Conscious. Cogn."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1126\/science.7725104","article-title":"How baseball outfielders determine where to run to catch fly balls","volume":"268","author":"McBeath","year":"1995","journal-title":"Science"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/s41235-018-0098-3","article-title":"Spatial decision dynamics during wayfinding: Intersections prompt the decision-making process","volume":"3","author":"Gardony","year":"2018","journal-title":"Cogn. Res. Princ. Implic."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"de Greeff, J., and Nolfi, S. (2010). Evolution of Implicit and Explicit Communication in Mobile Robots. Evolution of Communication and Language in Embodied Agents, Springer.","DOI":"10.1007\/978-3-642-01250-1_11"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.cortex.2017.01.016","article-title":"Action perception as hypothesis testing","volume":"89","author":"Donnarumma","year":"2017","journal-title":"Cortex"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"151","DOI":"10.3389\/fpsyg.2012.00151","article-title":"Perceptions as hypotheses: Saccades as experiments","volume":"3","author":"Friston","year":"2012","journal-title":"Front. Psychol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1007\/s00426-009-0237-z","article-title":"Thinking as the Control of Imagination: A Conceptual Framework for Goal-Directed Systems","volume":"73","author":"Pezzulo","year":"2009","journal-title":"Psychol. Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s11023-008-9095-5","article-title":"Coordinating with the Future: The Anticipatory Nature of Representation","volume":"18","author":"Pezzulo","year":"2008","journal-title":"Minds Mach."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Clark, A. (2015). Surfing Uncertainty: Prediction, Action, and the Embodied Mind, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780190217013.001.0001"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hohwy, J. (2013). The Predictive Mind, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199682737.001.0001"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s00422-010-0364-z","article-title":"Action and behavior: A free-energy formulation","volume":"102","author":"Friston","year":"2010","journal-title":"Biol. Cybern."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1016\/j.tics.2014.06.011","article-title":"Internally generated sequences in learning and executing goal-directed behavior","volume":"18","author":"Pezzulo","year":"2014","journal-title":"Trends Cogn. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1111\/nyas.13329","article-title":"Internally generated hippocampal sequences as a vantage point to probe future-oriented cognition","volume":"1396","author":"Pezzulo","year":"2017","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Stoianov, I.P., Pennartz, C.M.A., Lansink, C.S., and Pezzulo, G. (2018). Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis. PLoS Comput. Biol., 14.","DOI":"10.1371\/journal.pcbi.1006316"},{"key":"ref_45","unstructured":"O\u2019keefe, J., and Nadel, L. (1978). The Hippocampus as a Cognitive Map, Clarendon Press Oxford."},{"key":"ref_46","unstructured":"Metzinger, T., and Windt, J. (2014). The Cybernetic Bayesian Brain: From Interoceptive Inference to Sensorimotor Contingencies, MIND."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.tics.2018.01.009","article-title":"Hierarchical Active Inference: A Theory of Motivated Control","volume":"22","author":"Pezzulo","year":"2018","journal-title":"Trends Cogn. Sci."},{"key":"ref_48","unstructured":"Newell, A., and Simon, H.A. (1972). Human Problem Solving, Prentice-Hall."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Lepora, N.F., and Pezzulo, G. (2015). Embodied Choice: How action influences perceptual decision making. PLoS Comput. Biol., 11.","DOI":"10.1371\/journal.pcbi.1004110"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.tics.2016.03.013","article-title":"Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition","volume":"20","author":"Pezzulo","year":"2016","journal-title":"Trends Cogn. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychol. Rev."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Acuna, D.E., and Schrater, P. (2010). Structure learning in human sequential decision-making. PLoS Comput. Biol., 6.","DOI":"10.1371\/journal.pcbi.1001003"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Schwartenbeck, P., Passecker, J., Hauser, T., FitzGerald, T.H.B., Kronbichler, M., and Friston, K.J. (2018). Computational mechanisms of curiosity and goal-directed exploration. bioRxiv, 411272.","DOI":"10.1101\/411272"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1177\/1059712303114001","article-title":"The dynamics of active categorical perception in an evolved model agent","volume":"11","author":"Beer","year":"2003","journal-title":"Adapt. Behav."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1109\/TEVC.2010.2046174","article-title":"Active Categorical Perception of Object Shapes in a Simulated Anthropomorphic Robotic Arm","volume":"14","author":"Tuci","year":"2010","journal-title":"Trans. Evol. Comp."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.conb.2014.12.005","article-title":"Predictive motor control of sensory dynamics in auditory active sensing","volume":"31","author":"Morillon","year":"2015","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1207\/s15516709cog1804_1","article-title":"On distinguishing epistemic from pragmatic action","volume":"18","author":"Kirsh","year":"1994","journal-title":"Cogn. Sci."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1126\/science.1145803","article-title":"Self-organization, embodiment, and biologically inspired robotics","volume":"318","author":"Pfeifer","year":"2007","journal-title":"Science"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1007\/s00221-013-3459-7","article-title":"Kinematics fingerprints of leader and follower role-taking during cooperative joint actions","volume":"226","author":"Sacheli","year":"2013","journal-title":"Exp. Brain Res."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"20150644","DOI":"10.1098\/rsif.2015.0644","article-title":"Interactional leader\u2013follower sensorimotor communication strategies during repetitive joint actions","volume":"12","author":"Candidi","year":"2015","journal-title":"J. R. Soc. Interface"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"2945","DOI":"10.1007\/s00221-014-3982-1","article-title":"Strategic communication and behavioral coupling in asymmetric joint action","volume":"232","author":"Vesper","year":"2014","journal-title":"Exp. Brain Res."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1007\/s00221-011-2706-z","article-title":"Making oneself predictable: Reduced temporal variability facilitates joint action coordination","volume":"211","author":"Vesper","year":"2011","journal-title":"Exp. Brain Res."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1007\/s00221-011-2712-1","article-title":"What should I do next? Using shared representations to solve interaction problems","volume":"211","author":"Pezzulo","year":"2011","journal-title":"Exp. Brain Res."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Pezzulo, G., Donnarumma, F., and Dindo, H. (2013). Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0079876"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Pezzulo, G., Donnarumma, F., Dindo, H., D\u2019Ausilio, A., Konvalinka, I., and Castelfranchi, C. (2018). The body talks: Sensorimotor communication and its brain and kinematic signatures. Phys. Life Rev.","DOI":"10.1016\/j.plrev.2018.06.014"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/4\/350\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:41:47Z","timestamp":1760186507000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/4\/350"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,30]]},"references-count":65,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["e21040350"],"URL":"https:\/\/doi.org\/10.3390\/e21040350","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,3,30]]}}}