{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T22:44:50Z","timestamp":1786056290231,"version":"3.56.0"},"reference-count":217,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T00:00:00Z","timestamp":1766102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>A central challenge in artificial intelligence and cognitive science is identifying a unifying principle that governs inference, learning, and action. Active inference proposes such a principle: the minimization of variational free energy. Advocates of active inference argue that the framework subsumes classical models of optimal behavior\u2014including Bayesian decision theory, resource rationality, optimal control, and reinforcement learning\u2014while also instantiating information-theoretic principles such as rate-distortion theory and maximum entropy. However, the literature outlining these conceptual links remains fragmented, limiting integration across fields. This review develops these connections systematically. We show how these major frameworks admit formal correspondences with expected free energy minimization when expressed in variational form, exposing a shared optimization principle that underlies theories of optimal decision-making and information processing. This synthesis is intended both to orient researchers from other fields who are new to active inference and to clarify foundational assumptions for those already working within the framework.<\/jats:p>","DOI":"10.3390\/e28010001","type":"journal-article","created":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T12:59:57Z","timestamp":1766149197000},"page":"1","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Decision, Inference, and Information: Formal Equivalences Under Active Inference"],"prefix":"10.3390","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-9465-5516","authenticated-orcid":false,"given":"Patrick","family":"Sweeney","sequence":"first","affiliation":[{"name":"Centre for Complex Systems, The University of Sydney, Sydney, NSW 2006, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0220-3253","authenticated-orcid":false,"given":"Jaime","family":"Ruiz-Serra","sequence":"additional","affiliation":[{"name":"Centre for Complex Systems, The University of Sydney, Sydney, NSW 2006, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2199-2515","authenticated-orcid":false,"given":"Michael S.","family":"Harr\u00e9","sequence":"additional","affiliation":[{"name":"Centre for Complex Systems, The University of Sydney, Sydney, NSW 2006, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"674","DOI":"10.1162\/neco_a_01357","article-title":"Active Inference: Demystified and Compared","volume":"33","author":"Sajid","year":"2021","journal-title":"Neural Comput."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Parr, T., Pezzulo, G., and Friston, K.J. (2022). Active Inference: The Free Energy Principle in Mind, Brain, and Behavior, The MIT Press.","DOI":"10.7551\/mitpress\/12441.001.0001"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1162\/neco_a_01738","article-title":"Active Inference and Intentional Behavior","volume":"37","author":"Friston","year":"2025","journal-title":"Neural Comput."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gottwald, S., and Braun, D.A. (2020). The Two Kinds of Free Energy and the Bayesian Revolution. PLoS Comput. Biol., 16.","DOI":"10.1371\/journal.pcbi.1008420"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational Inference: A Review for Statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1007\/s00422-019-00805-w","article-title":"Generalised Free Energy and Active Inference","volume":"113","author":"Parr","year":"2019","journal-title":"Biol. Cybern."},{"key":"ref_7","unstructured":"Costa, L.D., Tenka, S., Zhao, D., and Sajid, N. (2024). Active Inference as a Model of Agency. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1007\/s00422-012-0512-8","article-title":"Active Inference and Agency: Optimal Control without Cost Functions","volume":"106","author":"Friston","year":"2012","journal-title":"Biol. Cybern."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tschantz, A., Millidge, B., Seth, A.K., and Buckley, C.L. (2020). Reinforcement Learning through Active Inference. arXiv.","DOI":"10.1109\/IJCNN48605.2020.9207382"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1007\/s00422-018-0785-7","article-title":"Deep Active Inference","volume":"112","year":"2018","journal-title":"Biol. Cybern."},{"key":"ref_11","unstructured":"Lanillos, P., Meo, C., Pezzato, C., Meera, A.A., Baioumy, M., Ohata, W., Tschantz, A., Millidge, B., Wisse, M., and Buckley, C.L. (2021). Active Inference in Robotics and Artificial Agents: Survey and Challenges. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO_a_00912","article-title":"Active Inference: A Process Theory","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102447","DOI":"10.1016\/j.jmp.2020.102447","article-title":"Active Inference on Discrete State-Spaces: A Synthesis","volume":"99","author":"Parr","year":"2020","journal-title":"J. Math. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102632","DOI":"10.1016\/j.jmp.2021.102632","article-title":"A Step-by-Step Tutorial on Active Inference and Its Application to Empirical Data","volume":"107","author":"Smith","year":"2022","journal-title":"J. Math. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"108741","DOI":"10.1016\/j.biopsycho.2023.108741","article-title":"Active Inference as a Theory of Sentient Behavior","volume":"186","author":"Pezzulo","year":"2024","journal-title":"Biol. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Friston, K., Heins, C., Verbelen, T., Da Costa, L., Salvatori, T., Markovic, D., Tschantz, A., Koudahl, M., Buckley, C., and Parr, T. (2025). From Pixels to Planning: Scale-Free Active Inference. Front. Netw. Physiol., 5.","DOI":"10.3389\/fnetp.2025.1521963"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The Free-Energy Principle: A Unified Brain Theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_18","unstructured":"Friston, K. (2019). A Free Energy Principle for a Particular Physics. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The Free Energy Principle for Action and Perception: A Mathematical Review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"20220029","DOI":"10.1098\/rsfs.2022.0029","article-title":"On Bayesian Mechanics: A Physics of and by Beliefs","volume":"13","author":"Ramstead","year":"2023","journal-title":"Interface Focus"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.plrev.2023.08.016","article-title":"Path Integrals, Particular Kinds, and Strange Things","volume":"47","author":"Friston","year":"2023","journal-title":"Phys. Life Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physrep.2023.07.001","article-title":"The Free Energy Principle Made Simpler but Not Too Simple","volume":"1024","author":"Friston","year":"2023","journal-title":"Phys. Rep."},{"key":"ref_23","unstructured":"Jacob, A.P., Wu, D.J., Farina, G., Lerer, A., Hu, H., Bakhtin, A., Andreas, J., and Brown, N. (2022, January 17\u201323). Modeling Strong and Human-Like Gameplay with KL-Regularized Search. Proceedings of the 39th International Conference on Machine Learning, PMLR, Baltimore, MD, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Friston, K., Heins, C., Ueltzh\u00f6ffer, K., Da Costa, L., and Parr, T. (2021). Stochastic Chaos and Markov Blankets. Entropy, 23.","DOI":"10.3390\/e23091220"},{"key":"ref_25","unstructured":"Sakthivadivel, D.A.R. (2022). Weak Markov Blankets in High-Dimensional, Sparsely-Coupled Random Dynamical Systems. arXiv."},{"key":"ref_26","first-page":"1","article-title":"Graphical Models, Exponential Families, and Variational Inference","volume":"1","author":"Wainwright","year":"2008","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1109\/TNN.2004.828762","article-title":"Variational Learning and Bits-Back Coding: An Information-Theoretic View to Bayesian Learning","volume":"15","author":"Honkela","year":"2004","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_28","first-page":"1","article-title":"An Optimization-centric View on Bayes\u2019 Rule: Reviewing and Generalizing Variational Inference","volume":"23","author":"Knoblauch","year":"2022","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1162\/neco_a_00999","article-title":"Active Inference, Curiosity and Insight","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"20170376","DOI":"10.1098\/rsif.2017.0376","article-title":"Uncertainty, Epistemics and Active Inference","volume":"14","author":"Parr","year":"2017","journal-title":"J. R. Soc. Interface"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","article-title":"Planning and Acting in Partially Observable Stochastic Domains","volume":"101","author":"Kaelbling","year":"1998","journal-title":"Artif. Intell."},{"key":"ref_32","unstructured":"Murphy, K.P. (2002). Dynamic Bayesian Networks: Representation, Inference and Learning. [Ph.D. Thesis, University of California]."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/0022-247X(65)90154-X","article-title":"Optimal Control of Markov Processes with Incomplete State Information","volume":"10","year":"1965","journal-title":"J. Math. Anal. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"102921","DOI":"10.1016\/j.jmp.2025.102921","article-title":"A Concise Mathematical Description of Active Inference in Discrete Time","volume":"125","author":"Langer","year":"2025","journal-title":"J. Math. Psychol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1090\/S0002-9904-1952-09620-8","article-title":"Some Aspects of the Sequential Design of Experiments","volume":"58","author":"Robbins","year":"1952","journal-title":"Bull. Am. Math. Soc."},{"key":"ref_36","unstructured":"Russell, S., and Norvig, P. (2020). Artificial Intelligence: A Modern Approach, Pearson."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1080\/17588928.2015.1020053","article-title":"Active Inference and Epistemic Value","volume":"6","author":"Friston","year":"2015","journal-title":"Cogn. Neurosci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active Inference and Learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Limanowski, J., Adams, R.A., Kilner, J., and Parr, T. (2024). The Many Roles of Precision in Action. Entropy, 26.","DOI":"10.31234\/osf.io\/9fqxg"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"P11011","DOI":"10.1088\/1742-5468\/2005\/11\/P11011","article-title":"Path Integrals and Symmetry Breaking for Optimal Control Theory","volume":"2005","author":"Kappen","year":"2005","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_41","unstructured":"Savage, L. (1954). The Foundations of Statistics, Wiley Publications in Statistics."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, Springer. Springer Series in Statistics.","DOI":"10.1007\/978-1-4757-4286-2"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1214\/ss\/1177009939","article-title":"Bayesian Experimental Design: A Review","volume":"10","author":"Chaloner","year":"1995","journal-title":"Stat. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1111\/insr.12107","article-title":"A Review of Modern Computational Algorithms for Bayesian Optimal Design","volume":"84","author":"Ryan","year":"2016","journal-title":"Int. Stat. Rev."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wu, C.M., Schulz, E., and Cogliati Dezza, I. (2022). Active Inference, Bayesian Optimal Design, and Expected Utility. The Drive for Knowledge: The Science of Human Information Seeking, Cambridge University Press.","DOI":"10.1017\/9781009026949"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1162\/neco_a_01574","article-title":"Reward Maximization Through Discrete Active Inference","volume":"35","author":"Sajid","year":"2023","journal-title":"Neural Comput."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"von Neumann, J., and Morgenstern, O. (2007). Theory of Games and Economic Behavior: 60th Anniversary Commemorative Edition, Princeton University Press.","DOI":"10.1515\/9781400829460"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/PL00004100","article-title":"Utility and Entropy","volume":"17","author":"Candeal","year":"2001","journal-title":"Econ. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ortega, P.A., and Braun, D.A. (2010, January 5\u20138). A Conversion between Utility and Information. Proceedings of the 3rd Conference on Artificial General Intelligence (AGI-10), Lugano, Switzerland.","DOI":"10.2991\/agi.2010.10"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1080\/01621459.1971.10482346","article-title":"Elicitation of Personal Probabilities and Expectations","volume":"66","author":"Savage","year":"1971","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly Proper Scoring Rules, Prediction, and Estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s10463-006-0099-8","article-title":"The Geometry of Proper Scoring Rules","volume":"59","author":"Dawid","year":"2007","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1287\/opre.1070.0498","article-title":"Scoring Rules, Generalized Entropy, and Utility Maximization","volume":"56","author":"Jose","year":"2008","journal-title":"Oper. Res."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Amari, S.i. (2016). Information Geometry and Its Applications, Springer.","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.geb.2019.07.012","article-title":"Proper Scoring Rules with General Preferences: A Dual Characterization of Optimal Reports","volume":"117","author":"Chambers","year":"2019","journal-title":"Games Econ. Behav."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Gr\u00fcnwald, P.D. (2007). The Minimum Description Length Principle, The MIT Press.","DOI":"10.7551\/mitpress\/4643.001.0001"},{"key":"ref_57","first-page":"1367","article-title":"Game Theory, Maximum Entropy, Minimum Discrepancy and Robust Bayesian Decision Theory","volume":"32","author":"Dawid","year":"2004","journal-title":"Ann. Stat."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1214\/aoms\/1177728069","article-title":"On a Measure of the Information Provided by an Experiment","volume":"27","author":"Lindley","year":"1956","journal-title":"Ann. Math. Stat."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1214\/aos\/1176344689","article-title":"Expected Information as Expected Utility","volume":"7","author":"Bernardo","year":"1979","journal-title":"Ann. Stat."},{"key":"ref_60","unstructured":"MacKay, D.J.C. (2019). Information Theory, Inference and Learning Algorithms, Cambridge University Press."},{"key":"ref_61","unstructured":"Raiffa, H., and Schlaifer, R. (2000). Applied Statistical Decision Theory, John Wiley & Sons."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1016\/j.neuron.2015.09.010","article-title":"The Psychology and Neuroscience of Curiosity","volume":"88","author":"Kidd","year":"2015","journal-title":"Neuron"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/s41562-019-0793-1","article-title":"How People Decide What They Want to Know","volume":"4","author":"Sharot","year":"2020","journal-title":"Nat. Hum. Behav."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1038\/s41583-018-0078-0","article-title":"Towards a Neuroscience of Active Sampling and Curiosity","volume":"19","author":"Gottlieb","year":"2018","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1038\/nrn3289","article-title":"Knowing How Much You Don\u2019t Know: A Neural Organization of Uncertainty Estimates","volume":"13","author":"Bach","year":"2012","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Oudeyer, P.Y., and Kaplan, F. (2007). What Is Intrinsic Motivation? A Typology of Computational Approaches. Front. Neurorobot., 1.","DOI":"10.3389\/neuro.12.006.2007"},{"key":"ref_67","unstructured":"Chentanez, N., Barto, A., and Singh, S. (2004, January 13\u201318). Intrinsically Motivated Reinforcement Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1162\/neco.1992.4.4.590","article-title":"Information-Based Objective Functions for Active Data Selection","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"ref_69","unstructured":"Cohn, D., Ghahramani, Z., and Jordan, M. (December, January 28). Active Learning with Statistical Models. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TAMD.2010.2056368","article-title":"Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990\u20132010)","volume":"2","author":"Schmidhuber","year":"2010","journal-title":"IEEE Trans. Auton. Ment. Dev."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Kolter, J.Z., and Ng, A.Y. (2009, January 14\u201318). Near-Bayesian Exploration in Polynomial Time. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553441"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J., Th\u00f3risson, K.R., and Looks, M. (2011, January 3\u20136). Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments. Proceedings of the 4th International Conference on Artificial General Intelligence, Mountain View, CA, USA.","DOI":"10.1007\/978-3-642-22887-2"},{"key":"ref_73","unstructured":"Mohamed, S., and Jimenez Rezende, D. (2015, January 7\u201312). Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_74","unstructured":"Houthooft, R., Chen, X., Chen, X., Duan, Y., Schulman, J., De Turck, F., and Abbeel, P. (2016, January 5\u201310). VIME: Variational Information Maximizing Exploration. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_75","unstructured":"Kim, H., Kim, J., Jeong, Y., Levine, S., and Song, H.O. (2019, January 9\u201315). EMI: Exploration with Mutual Information. Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_76","unstructured":"Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. (2018). Diversity Is All You Need: Learning Skills without a Reward Function. arXiv."},{"key":"ref_77","unstructured":"Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., and Pathak, D. (2020, January 13\u201318). Planning to Explore via Self-Supervised World Models. Proceedings of the 37th International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_78","first-page":"13198","article-title":"VariBAD: Variational Bayes-Adaptive Deep RL via Meta-Learning","volume":"22","author":"Zintgraf","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1287\/opre.2017.1663","article-title":"Learning to Optimize via Information-Directed Sampling","volume":"66","author":"Russo","year":"2018","journal-title":"Oper. Res."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1561\/2200000097","article-title":"Reinforcement Learning, Bit by Bit","volume":"16","author":"Lu","year":"2023","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_81","unstructured":"Arumugam, D., and Roy, B.V. (2021, January 18\u201324). Deciding What to Learn: A Rate-Distortion Approach. Proceedings of the 38th International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_82","unstructured":"Sukhija, B., Coros, S., Krause, A., Abbeel, P., and Sferrazza, C. (2024, January 7\u201311). MaxInfoRL: Boosting Exploration in Reinforcement Learning through Information Gain Maximization. Proceedings of the Thirteenth International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1111\/tops.12086","article-title":"Computational Rationality: Linking Mechanism and Behavior through Bounded Utility Maximization","volume":"6","author":"Lewis","year":"2014","journal-title":"Top. Cogn. Sci."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1111\/tops.12142","article-title":"Rational Use of Cognitive Resources: Levels of Analysis between the Computational and the Algorithmic","volume":"7","author":"Griffiths","year":"2015","journal-title":"Top. Cogn. Sci."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1017\/S0140525X1900061X","article-title":"Resource-Rational Analysis: Understanding Human Cognition as the Optimal Use of Limited Computational Resources","volume":"43","author":"Lieder","year":"2019","journal-title":"Behav. Brain Sci."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.cobeha.2021.02.015","article-title":"Resource-Rational Decision Making","volume":"41","author":"Bhui","year":"2021","journal-title":"Curr. Opin. Behav. Sci."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1111\/tops.12562","article-title":"Resource-Rational Models of Human Goal Pursuit","volume":"14","author":"Prystawski","year":"2022","journal-title":"Top. Cogn. Sci."},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Zhu, J.Q., and Griffiths, T.L. (Psychol. Rev., 2025). Computation-Limited Bayesian Updating: A Resource-Rational Analysis of Approximate Bayesian Inference, Psychol. Rev., Epub ahead of printing.","DOI":"10.31234\/osf.io\/6aw3f_v2"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"99","DOI":"10.2307\/1884852","article-title":"A Behavioral Model of Rational Choice","volume":"69","author":"Simon","year":"1955","journal-title":"Q. J. Econ."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1257\/jel.20231592","article-title":"Bounded Rationality in Choice Theory: A Survey","volume":"62","author":"Rozen","year":"2024","journal-title":"J. Econ. Lit."},{"key":"ref_91","first-page":"20120683","article-title":"Thermodynamics as a Theory of Decision-Making with Information-Processing Costs","volume":"469","author":"Ortega","year":"2013","journal-title":"Proc. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_92","unstructured":"Ortega, P.A., Braun, D.A., Dyer, J., Kim, K.E., and Tishby, N. (2015). Information-Theoretic Bounded Rationality. arXiv."},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Gottwald, S., and Braun, D.A. (2019). Bounded Rational Decision-Making from Elementary Computations That Reduce Uncertainty. Entropy, 21.","DOI":"10.3390\/e21040375"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Harr\u00e9, M.S. (2021). Information Theory for Agents in Artificial Intelligence, Psychology, and Economics. Entropy, 23.","DOI":"10.3390\/e23030310"},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Marzen, S. (2025). Resource-Rational Reinforcement Learning and Sensorimotor Causal States, and Resource-Rational Maximiners. arXiv.","DOI":"10.1098\/rsfs.2024.0062"},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function","volume":"79","author":"Shenhav","year":"2013","journal-title":"Neuron"},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1017\/S0140525X12003196","article-title":"An Opportunity Cost Model of Subjective Effort and Task Performance","volume":"36","author":"Kurzban","year":"2013","journal-title":"Behav. Brain Sci."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"395","DOI":"10.3758\/s13415-015-0334-y","article-title":"Cognitive Effort: A Neuroeconomic Approach","volume":"15","author":"Westbrook","year":"2015","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1126\/science.aac6076","article-title":"Computational Rationality: A Converging Paradigm for Intelligence in Brains, Minds, and Machines","volume":"349","author":"Gershman","year":"2015","journal-title":"Science"},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"1286","DOI":"10.1038\/nn.4384","article-title":"Dorsal Anterior Cingulate Cortex and the Value of Control","volume":"19","author":"Shenhav","year":"2016","journal-title":"Nat. Neurosci."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/0004-3702(91)90015-C","article-title":"Principles of Metareasoning","volume":"49","author":"Russell","year":"1991","journal-title":"Artif. Intell."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1613\/jair.133","article-title":"Provably Bounded-Optimal Agents","volume":"2","author":"Russell","year":"1994","journal-title":"J. Artif. Intell. Res."},{"key":"ref_103","unstructured":"Lieder, F., Plunkett, D., Hamrick, J.B., Russell, S.J., Hay, N.J., and Griffiths, T.L. (2014, January 8\u201313). Algorithm Selection by Rational Metareasoning as a Model of Human Strategy Selection. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_104","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1037\/rev0000075","article-title":"Strategy Selection as Rational Metareasoning","volume":"124","author":"Lieder","year":"2017","journal-title":"Psychol. Rev."},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1016\/S0304-3932(03)00029-1","article-title":"Implications of Rational Inattention","volume":"50","author":"Sims","year":"2003","journal-title":"J. Monet. Econ."},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1257\/aer.20130047","article-title":"Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model","volume":"105","author":"McKay","year":"2015","journal-title":"Am. Econ. Rev."},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1257\/jel.20211524","article-title":"Rational Inattention: A Review","volume":"61","author":"Wiederholt","year":"2023","journal-title":"J. Econ. Lit."},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information Theory and Statistical Mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1103\/PhysRev.108.171","article-title":"Information Theory and Statistical Mechanics. II","volume":"108","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_110","unstructured":"Rosenkrantz, R.D. (2012). E. T. Jaynes: Papers on Probability, Statistics and Statistical Physics, Springer Science & Business Media."},{"key":"ref_111","unstructured":"Cover, T.M. (2006). Elements of Information Theory, Wiley-Interscience. [2nd ed.]."},{"key":"ref_112","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/TIT.1980.1056144","article-title":"Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy","volume":"26","author":"Shore","year":"1980","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_113","unstructured":"Kullback, S. (1997). Information Theory and Statistics, Dover Publications."},{"key":"ref_114","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1214\/aop\/1176996454","article-title":"I-Divergence Geometry of Probability Distributions and Minimization Problems","volume":"3","author":"Csiszar","year":"1975","journal-title":"Ann. Probab."},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1109\/21.44019","article-title":"The Generalized Maximum Entropy Principle","volume":"19","author":"Kesavan","year":"1989","journal-title":"IEEE Trans. Syst. Man, Cybern."},{"key":"ref_116","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1146\/annurev.pc.31.100180.003051","article-title":"The Minimum Entropy Production Principle","volume":"31","author":"Jaynes","year":"1980","journal-title":"Annu. Rev. Phys. Chem."},{"key":"ref_117","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1126\/science.201.4358.777","article-title":"Time, Structure, and Fluctuations","volume":"201","author":"Prigogine","year":"1978","journal-title":"Science"},{"key":"ref_118","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1613\/jair.3062","article-title":"A Minimum Relative Entropy Principle for Learning and Acting","volume":"38","author":"Ortega","year":"2010","journal-title":"J. Artif. Intell. Res."},{"key":"ref_119","unstructured":"Hafner, D., Ortega, P.A., Ba, J., Parr, T., Friston, K., and Heess, N. (2022). Action and Perception as Divergence Minimization. arXiv."},{"key":"ref_120","doi-asserted-by":"crossref","unstructured":"Boyd, S.P. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_121","unstructured":"Ziebart, B.D., Maas, A., Bagnell, J.A., and Dey, A.K. (2008, January 13\u201317). Maximum Entropy Inverse Reinforcement Learning. Proceedings of the 23rd National Conference on Artificial Intelligence, Chicago, IL, USA."},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1006\/game.1995.1023","article-title":"Quantal Response Equilibria for Normal Form Games","volume":"10","author":"McKelvey","year":"1995","journal-title":"Games Econ. Behav."},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Goeree, J.K., Holt, C.A., and Palfrey, T.R. (2016). Quantal Response Equilibrium: A Stochastic Theory of Games, Princeton University Press.","DOI":"10.23943\/princeton\/9780691124230.001.0001"},{"key":"ref_124","doi-asserted-by":"crossref","unstructured":"Scharfenaker, E., and Foley, D.K. (2017). Quantal Response Statistical Equilibrium in Economic Interactions: Theory and Estimation. Entropy, 19.","DOI":"10.3390\/e19090444"},{"key":"ref_125","doi-asserted-by":"crossref","first-page":"103990","DOI":"10.1016\/j.jedc.2020.103990","article-title":"Implications of Quantal Response Statistical Equilibrium","volume":"119","author":"Scharfenaker","year":"2020","journal-title":"J. Econ. Dyn. Control."},{"key":"ref_126","doi-asserted-by":"crossref","first-page":"036102","DOI":"10.1103\/PhysRevE.85.036102","article-title":"Hysteresis Effects of Changing the Parameters of Noncooperative Games","volume":"85","author":"Wolpert","year":"2012","journal-title":"Phys. Rev. E"},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1140\/epjb\/e2013-31064-x","article-title":"Simple Nonlinear Systems and Navigating Catastrophes","volume":"86","author":"Atkinson","year":"2013","journal-title":"Eur. Phys. J. B"},{"key":"ref_128","doi-asserted-by":"crossref","unstructured":"Harris, A., McCallum, S., and Harr\u00e9, M.S. (Games Econ. Behav., 2023). On the Smooth Unfolding of Bifurcations in Quantal-Response Equilibria, Games Econ. Behav., in press.","DOI":"10.1016\/j.geb.2023.08.011"},{"key":"ref_129","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1137\/1019036","article-title":"Structural Stability, Catastrophe Theory, and Applied Mathematics","volume":"19","author":"Thom","year":"1977","journal-title":"SIAM Rev."},{"key":"ref_130","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1006\/game.1996.0044","article-title":"Potential Games","volume":"14","author":"Monderer","year":"1996","journal-title":"Games Econ. Behav."},{"key":"ref_131","doi-asserted-by":"crossref","first-page":"103653","DOI":"10.1016\/j.artint.2021.103653","article-title":"Exploration-Exploitation in Multi-Agent Learning: Catastrophe Theory Meets Game Theory","volume":"304","author":"Leonardos","year":"2022","journal-title":"Artif. Intell."},{"key":"ref_132","unstructured":"Fox, R., Mcaleer, S.M., Overman, W., and Panageas, I. (2022, January 28\u201330). Independent Natural Policy Gradient Always Converges in Markov Potential Games. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, PMLR, Virtual."},{"key":"ref_133","doi-asserted-by":"crossref","first-page":"3255","DOI":"10.1016\/j.jedc.2006.09.013","article-title":"The Rise and Fall of Catastrophe Theory Applications in Economics: Was the Baby Thrown out with the Bathwater?","volume":"31","year":"2007","journal-title":"J. Econ. Dyn. Control."},{"key":"ref_134","doi-asserted-by":"crossref","unstructured":"Harr\u00e9, M.S. (2022). Entropy, Economics, and Criticality. Entropy, 24.","DOI":"10.3390\/e24020210"},{"key":"ref_135","doi-asserted-by":"crossref","unstructured":"Evans, B.P., and Prokopenko, M. (2021). A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback. Entropy, 23.","DOI":"10.3390\/e23060669"},{"key":"ref_136","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.conb.2017.08.001","article-title":"Maximum Entropy Models as a Tool for Building Precise Neural Controls","volume":"46","author":"Savin","year":"2017","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_137","doi-asserted-by":"crossref","first-page":"1007","DOI":"10.1038\/nature04701","article-title":"Weak Pairwise Correlations Imply Strongly Correlated Network States in a Neural Population","volume":"440","author":"Schneidman","year":"2006","journal-title":"Nature"},{"key":"ref_138","doi-asserted-by":"crossref","first-page":"P03011","DOI":"10.1088\/1742-5468\/2013\/03\/P03011","article-title":"The Simplest Maximum Entropy Model for Collective Behavior in a Neural Network","volume":"2013","author":"Tkacik","year":"2013","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_139","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_140","first-page":"325","article-title":"Coding Theorems for a Discrete Source with a Fidelity Criterion","volume":"Volume 7","author":"Shannon","year":"1993","journal-title":"Claude E. Shannon: Collected Papers; Institute of Radio Engineers, International Convention Record"},{"key":"ref_141","unstructured":"Berger, T. (1971). Rate Distortion Theory: A Mathematical Basis for Data Compression, Prentice-Hall."},{"key":"ref_142","doi-asserted-by":"crossref","first-page":"28005","DOI":"10.1209\/0295-5075\/85\/28005","article-title":"Information-Theoretic Approach to Interactive Learning","volume":"85","author":"Still","year":"2009","journal-title":"Europhys. Lett."},{"key":"ref_143","doi-asserted-by":"crossref","unstructured":"Cutsuridis, V., Hussain, A., and Taylor, J.G. (2011). Information Theory of Decisions and Actions. Perception-Action Cycle: Models, Architectures, and Hardware, Springer.","DOI":"10.1007\/978-1-4419-1452-1"},{"key":"ref_144","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1109\/TIT.1972.1054855","article-title":"Computation of Channel Capacity and Rate-Distortion Functions","volume":"18","author":"Blahut","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_145","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/TIT.1972.1054753","article-title":"An Algorithm for Computing the Capacity of Arbitrary Discrete Memoryless Channels","volume":"18","author":"Arimoto","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_146","unstructured":"Poole, B., Ozair, S., Oord, A.V.D., Alemi, A., and Tucker, G. (2019, January 9\u201315). On Variational Bounds of Mutual Information. Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_147","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/j.neucom.2019.05.083","article-title":"Caching Mechanisms for Habit Formation in Active Inference","volume":"359","author":"Maisto","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_148","unstructured":"Varona, M.d.L., Buckley, C.L., and Millidge, B. (2024). Exploring Action-Centric Representations Through the Lens of Rate-Distortion Theory. arXiv."},{"key":"ref_149","unstructured":"Tishby, N., Pereira, F.C., and Bialek, W. (2000). The Information Bottleneck Method. arXiv."},{"key":"ref_150","doi-asserted-by":"crossref","first-page":"041925","DOI":"10.1103\/PhysRevE.79.041925","article-title":"Past-Future Information Bottleneck in Dynamical Systems","volume":"79","author":"Creutzig","year":"2009","journal-title":"Phys. Rev. E"},{"key":"ref_151","unstructured":"Alemi, A.A. (2019, January 8). Variational Predictive Information Bottleneck. Proceedings of the 2nd Symposium on Advances in Approximate Bayesian Inference, PMLR, Vancouver, BC, Canada."},{"key":"ref_152","unstructured":"Stengel, R.F. (1994). Optimal Control and Estimation, Dover Publications."},{"key":"ref_153","unstructured":"Bertsekas, D., and Shreve, S.E. (1996). Stochastic Optimal Control: The Discrete-Time Case, Athena Scientific."},{"key":"ref_154","unstructured":"Levine, S. (2018). Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review. arXiv."},{"key":"ref_155","first-page":"12","article-title":"Active Inference or Control as Inference? A Unifying View","volume":"Volume 1326","author":"Verbelen","year":"2020","journal-title":"Proceedings of the 2020 International Workshop on Active Inference"},{"key":"ref_156","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1073\/pnas.45.8.1288","article-title":"A Mathematical Theory of Adaptive Control Processes","volume":"45","author":"Bellman","year":"1959","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_157","unstructured":"Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons."},{"key":"ref_158","doi-asserted-by":"crossref","unstructured":"Bellman, R. (1953). An Introduction to the Theory of Dynamic Programming, RAND Corporation.","DOI":"10.1073\/pnas.39.10.1077"},{"key":"ref_159","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1214\/aoms\/1177700285","article-title":"Discounted Dynamic Programming","volume":"36","author":"Blackwell","year":"1965","journal-title":"Ann. Math. Stat."},{"key":"ref_160","first-page":"279","article-title":"Q-Learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_161","unstructured":"Attias, H. (2003, January 3\u20136). Planning by Probabilistic Inference. Proceedings of the International Workshop on Artificial Intelligence and Statistics, PMLR, Key West, FL, USA."},{"key":"ref_162","unstructured":"Todorov, E. (2006, January 4\u20137). Linearly-Solvable Markov Decision Problems. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_163","doi-asserted-by":"crossref","unstructured":"Todorov, E. (2008, January 9\u201311). General Duality between Optimal Control and Estimation. Proceedings of the 2008 47th IEEE Conference on Decision and Control, Cancun, Mexico.","DOI":"10.1109\/CDC.2008.4739438"},{"key":"ref_164","doi-asserted-by":"crossref","unstructured":"Toussaint, M. (2009, January 14\u201318). Robot Trajectory Optimization Using Approximate Inference. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553508"},{"key":"ref_165","doi-asserted-by":"crossref","first-page":"3352","DOI":"10.3390\/e17053352","article-title":"Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations","volume":"17","author":"Theodorou","year":"2015","journal-title":"Entropy"},{"key":"ref_166","unstructured":"Song, Z., Parr, R., and Carin, L. (2019, January 9\u201315). Revisiting the Softmax Bellman Operator: New Benefits and New Perspective. Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_167","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A New Approach to Linear Filtering and Prediction Problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_168","unstructured":"Basar, T. (2001). Contributions to the Theory of Optimal Control. Control Theory: Twenty-Five Seminal Papers (1932\u20131981), IEEE."},{"key":"ref_169","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. Adaptive Computation and Machine Learning Series."},{"key":"ref_170","unstructured":"Kochenderfer, M.J., Wheeler, T.A., and Wray, K.H. (2022). Algorithms for Decision Making, MIT Press."},{"key":"ref_171","unstructured":"Ziebart, B.D. (2010). Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy. [Ph.D. Thesis, Carnegie Mellon University]."},{"key":"ref_172","doi-asserted-by":"crossref","first-page":"1447","DOI":"10.1111\/j.1468-0262.2006.00716.x","article-title":"Ambiguity Aversion, Robustness, and the Variational Representation of Preferences","volume":"74","author":"Maccheroni","year":"2006","journal-title":"Econometrica"},{"key":"ref_173","unstructured":"Hansen, L.P., and Sargent, T.J. (2011). Robustness, Princeton University Press."},{"key":"ref_174","unstructured":"Eysenbach, B., and Levine, S. (2021, January 3\u20137). Maximum Entropy RL (Provably) Solves Some Robust RL Problems. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_175","doi-asserted-by":"crossref","first-page":"6368","DOI":"10.1038\/s41467-024-49711-1","article-title":"Complex Behavior from Intrinsic Motivation to Occupy Future Action-State Path Space","volume":"15","author":"Grytskyy","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_176","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A Neural Substrate of Prediction and Reward","volume":"275","author":"Schultz","year":"1997","journal-title":"Science"},{"key":"ref_177","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.conb.2008.08.003","article-title":"Reinforcement Learning: The Good, The Bad and The Ugly","volume":"18","author":"Dayan","year":"2008","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_178","doi-asserted-by":"crossref","first-page":"15647","DOI":"10.1073\/pnas.1014269108","article-title":"Understanding Dopamine and Reinforcement Learning: The Dopamine Reward Prediction Error Hypothesis","volume":"108","author":"Glimcher","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_179","unstructured":"Peters, J., M\u00fclling, K., and Alt\u00fcn, Y. (2010, January 11\u201315). Relative Entropy Policy Search. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, Atlanta, GA, USA."},{"key":"ref_180","first-page":"12163","article-title":"Leverage the Average: An Analysis of KL Regularization in Reinforcement Learning","volume":"Volume 33","author":"Vieillard","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_181","unstructured":"Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. (2017, January 6\u201311). Reinforcement Learning with Deep Energy-Based Policies. Proceedings of the 34th International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_182","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the 35th International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_183","unstructured":"Schulman, J., Chen, X., and Abbeel, P. (2018). Equivalence Between Policy Gradients and Soft Q-Learning. arXiv."},{"key":"ref_184","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/S0167-6377(02)00231-6","article-title":"Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization","volume":"31","author":"Beck","year":"2003","journal-title":"Oper. Res. Lett."},{"key":"ref_185","unstructured":"Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability, Springer Science & Business Media."},{"key":"ref_186","doi-asserted-by":"crossref","first-page":"1076","DOI":"10.3390\/e13061076","article-title":"A Philosophical Treatise of Universal Induction","volume":"13","author":"Rathmanner","year":"2011","journal-title":"Entropy"},{"key":"ref_187","unstructured":"Ross, S., Chaib-Draa, B., and Pineau, J. (2007, January 3\u20136). Bayes-Adaptive POMDPs. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_188","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1561\/2200000049","article-title":"Bayesian Reinforcement Learning: A Survey","volume":"8","author":"Ghavamzadeh","year":"2015","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_189","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1162\/opmi_a_00132","article-title":"Bayesian Reinforcement Learning with Limited Cognitive Load","volume":"8","author":"Arumugam","year":"2024","journal-title":"Open Mind"},{"key":"ref_190","unstructured":"Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. (2019, January 9\u201315). Learning Latent Dynamics for Planning from Pixels. Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_191","doi-asserted-by":"crossref","unstructured":"Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. (2024). Mastering Diverse Domains through World Models. arXiv.","DOI":"10.1038\/s41586-025-08744-2"},{"key":"ref_192","unstructured":"Janner, M., Fu, J., Zhang, M., and Levine, S. (2019, January 8\u201314). When to Trust Your Model: Model-Based Policy Optimization. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_193","unstructured":"Ha, D., and Schmidhuber, J. (2018, January 2\u20138). Recurrent World Models Facilitate Policy Evolution. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_194","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.neunet.2022.03.037","article-title":"Deep Learning, Reinforcement Learning, and World Models","volume":"152","author":"Matsuo","year":"2022","journal-title":"Neural Netw."},{"key":"ref_195","unstructured":"Richens, J., Everitt, T., and Abel, D. (2025, January 13\u201319). General Agents Need World Models. Proceedings of the Forty-Second International Conference on Machine Learning, Vancouver, BC, Canada."},{"key":"ref_196","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000086","article-title":"Model-Based Reinforcement Learning: A Survey","volume":"16","author":"Moerland","year":"2023","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_197","unstructured":"Ha, D., and Schmidhuber, J. (2018). World Models. arXiv."},{"key":"ref_198","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"ref_199","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1214\/09-EJS485","article-title":"Dynamics of Bayesian Updating with Dependent Data and Misspecified Models","volume":"3","author":"Shalizi","year":"2009","journal-title":"Electron. J. Stat."},{"key":"ref_200","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1063\/1.1530990","article-title":"Regularities Unseen, Randomness Observed: Levels of Entropy Convergence","volume":"13","author":"Crutchfield","year":"2003","journal-title":"Chaos"},{"key":"ref_201","unstructured":"Hoffman, M.D., and Johnson, M.J. (2016, January 9). ELBO Surgery: Yet Another Way to Carve up the Variational Evidence Lower Bound. Proceedings of the NeurIPS Workshop on Advances in Approximate Bayesian Inference, Barcelona, Spain."},{"key":"ref_202","doi-asserted-by":"crossref","unstructured":"Friston, K., Parr, T., Heins, C., Da Costa, L., Salvatori, T., Tschantz, A., Koudahl, M., Van de Maele, T., Buckley, C., and Verbelen, T. (2025). Gradient-Free De Novo Learning. Entropy, 27.","DOI":"10.3390\/e27090992"},{"key":"ref_203","doi-asserted-by":"crossref","first-page":"108891","DOI":"10.1016\/j.biopsycho.2024.108891","article-title":"Supervised Structure Learning","volume":"193","author":"Friston","year":"2024","journal-title":"Biol. Psychol."},{"key":"ref_204","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1162\/neco_a_01351","article-title":"Sophisticated Inference","volume":"33","author":"Friston","year":"2021","journal-title":"Neural Comput."},{"key":"ref_205","doi-asserted-by":"crossref","unstructured":"Kaufmann, R., Gupta, P., and Taylor, J. (2021). An Active Inference Model of Collective Intelligence. Entropy, 23.","DOI":"10.3390\/e23070830"},{"key":"ref_206","doi-asserted-by":"crossref","unstructured":"Albarracin, M., Pitliya, R.J., St. Clere Smithe, T., Friedman, D.A., Friston, K., and Ramstead, M.J.D. (2024). Shared Protentions in Multi-Agent Active Inference. Entropy, 26.","DOI":"10.3390\/e26040303"},{"key":"ref_207","unstructured":"Hyland, D., Gaven\u010diak, T., Costa, L.D., Heins, C., Kovarik, V., Gutierrez, J., Wooldridge, M.J., and Kulveit, J. (2024, January 26). Free-Energy Equilibria: Toward a Theory of Interactions Between Boundedly-Rational Agents. Proceedings of the ICML 2024 Workshop on Models of Human Feedback for AI Alignment, Vienna, Austria."},{"key":"ref_208","unstructured":"Vorobeychik, Y., Das, S., and Now\u00e9, A. (2025, January 19\u201323). Factorised Active Inference for Strategic Multi-Agent Interactions. Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), Detroit, MI, USA."},{"key":"ref_209","doi-asserted-by":"crossref","first-page":"124315","DOI":"10.1016\/j.eswa.2024.124315","article-title":"On Efficient Computation in Active Inference","volume":"253","author":"Paul","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_210","doi-asserted-by":"crossref","first-page":"129319","DOI":"10.1016\/j.neucom.2024.129319","article-title":"Active Inference Tree Search in Large POMDPs","volume":"623","author":"Maisto","year":"2025","journal-title":"Neurocomputing"},{"key":"ref_211","first-page":"11662","article-title":"Deep Active Inference Agents Using Monte-Carlo Methods","volume":"Volume 33","author":"Fountas","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_212","doi-asserted-by":"crossref","first-page":"2132","DOI":"10.1162\/neco_a_01529","article-title":"Branching Time Active Inference with Bayesian Filtering","volume":"34","author":"Champion","year":"2022","journal-title":"Neural Comput."},{"key":"ref_213","first-page":"2479","article-title":"Multimodal and Multifactor Branching Time Active Inference","volume":"36","author":"Champion","year":"2024","journal-title":"Neural Comput."},{"key":"ref_214","unstructured":"Mazzaglia, P., Verbelen, T., and Dhoedt, B. (2021, January 6\u201314). Contrastive Active Inference. Proceedings of the Advances in Neural Information Processing Systems 2021, Virtual."},{"key":"ref_215","unstructured":"Nuijten, W.W.L., Lukashchuk, M., van de Laar, T., and de Vries, B. (2025). A Message Passing Realization of Expected Free Energy Minimization. arXiv."},{"key":"ref_216","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1287\/moor.12.3.441","article-title":"The Complexity of Markov Decision Processes","volume":"12","author":"Papadimitriou","year":"1987","journal-title":"Math. Oper. Res."},{"key":"ref_217","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/S0004-3702(02)00378-8","article-title":"On the Undecidability of Probabilistic Planning and Related Stochastic Optimization Problems","volume":"147","author":"Madani","year":"2003","journal-title":"Artif. Intell."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/1\/1\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T05:10:56Z","timestamp":1766725856000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/1\/1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,19]]},"references-count":217,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["e28010001"],"URL":"https:\/\/doi.org\/10.3390\/e28010001","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,19]]}}}