{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T08:26:36Z","timestamp":1772526396855,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T00:00:00Z","timestamp":1637712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Dutch Research Council","doi-asserted-by":"publisher","award":["NWO perspective program P16-25"],"award-info":[{"award-number":["NWO perspective program P16-25"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.<\/jats:p>","DOI":"10.3390\/e23121565","type":"journal-article","created":{"date-parts":[[2021,11,29]],"date-time":"2021-11-29T05:23:02Z","timestamp":1638163382000},"page":"1565","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["On Epistemics in Expected Free Energy for Linear Gaussian State Space Models"],"prefix":"10.3390","volume":"23","author":[{"given":"Magnus T.","family":"Koudahl","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0547-4817","authenticated-orcid":false,"given":"Wouter M.","family":"Kouw","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bert","family":"de Vries","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"},{"name":"GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sajid, N., Costa, L.D., Parr, T., and Friston, K. (2021). Active inference, Bayesian optimal design, and expected utility. arXiv.","DOI":"10.1017\/9781009026949.007"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active inference and learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Baltieri, M., and Buckley, C. (2019). PID Control as a Process of Active Inference with Linear Generative Models. Entropy, 21.","DOI":"10.20944\/preprints201902.0246.v1"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"937860","DOI":"10.1155\/2012\/937860","article-title":"Free Energy, Value, and Attractors","volume":"2012","author":"Friston","year":"2012","journal-title":"Comput. Math. Methods Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: A mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"van de Laar, T.W., and de Vries, B. (2019). Simulating Active Inference Processes by Message Passing. Front. Robot. AI, 6.","DOI":"10.3389\/frobt.2019.00020"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Baltieri, M., and Buckley, C.L. (2017). An active inference implementation of phototaxis. arXiv.","DOI":"10.7551\/ecal_a_011"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1080\/17588928.2015.1020053","article-title":"Active inference and epistemic value","volume":"6","author":"Friston","year":"2015","journal-title":"Cogn. Neurosci."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sajid, N., Ball, P.J., and Friston, K.J. (2020). Active inference: Demystified and compared. arXiv.","DOI":"10.1162\/neco_a_01357"},{"key":"ref_10","unstructured":"Ghavamzadeh, M., Mannor, S., Pineau, J., and Tamar, A. (2016). Bayesian Reinforcement Learning: A Survey. arXiv."},{"key":"ref_11","first-page":"809","article-title":"Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness","volume":"3","author":"Cullen","year":"2018","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1007\/s00422-019-00805-w","article-title":"Generalised free energy and active inference","volume":"113","author":"Parr","year":"2019","journal-title":"Biol. Cybern."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Friston, K., Da Costa, L., Hafner, D., Hesp, C., and Parr, T. (2020). Sophisticated Inference. arXiv.","DOI":"10.1162\/neco_a_01351"},{"key":"ref_14","unstructured":"Fountas, Z., Sajid, N., Mediano, P.A.M., and Friston, K. (2020). Deep active inference agents using Monte-Carlo methods. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tschantz, A., Seth, A.K., and Buckley, C.L. (2020). Learning action-oriented models through active inference. PLoS Comput. Biol., 16.","DOI":"10.1371\/journal.pcbi.1007805"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tschantz, A., Millidge, B., Seth, A.K., and Buckley, C.L. (2020). Reinforcement Learning through Active Inference. arXiv.","DOI":"10.1109\/IJCNN48605.2020.9207382"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Millidge, B. (2019). Deep Active Inference as Variational Policy Gradients. arXiv.","DOI":"10.1016\/j.jmp.2020.102348"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tschantz, A., Baltieri, M., Seth, A.K., and Buckley, C.L. (2019). Scaling active inference. arXiv.","DOI":"10.1109\/IJCNN48605.2020.9207382"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1007\/s00422-018-0785-7","article-title":"Deep Active Inference","volume":"112","year":"2018","journal-title":"Biol. Cybern."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1109\/18.910573","article-title":"Codes on graphs: Normal realizations","volume":"47","author":"Forney","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1109\/JPROC.2007.896497","article-title":"The Factor Graph Approach to Model-Based Signal Processing","volume":"95","author":"Loeliger","year":"2007","journal-title":"Proc. IEEE"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MSP.2004.1267047","article-title":"An introduction to factor graphs","volume":"21","author":"Loeliger","year":"2004","journal-title":"Signal Process. Mag. IEEE"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sarkka, S. (2013). Bayesian Filtering and Smoothing, Cambridge University Press.","DOI":"10.1017\/CBO9781139344203"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Schwartenbeck, P., FitzGerald, T., Dolan, R.J., and Friston, K. (2013). Exploration, novelty, surprise, and free energy minimization. Front. Psychol., 4.","DOI":"10.3389\/fpsyg.2013.00710"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., and Friston, K. (2020). Active inference on discrete state-spaces: A synthesis. arXiv.","DOI":"10.1016\/j.jmp.2020.102447"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"\u015een\u00f6z, \u0130., van de Laar, T., Bagaev, D., and de Vries, B. (2021). Variational Message Passing and Local Constraint Manipulation in Factor Graphs. Entropy, 23.","DOI":"10.3390\/e23070807"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational Inference: A Review for Statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Schwartenbeck, P., Passecker, J., Hauser, T.U., FitzGerald, T.H.B., Kronbichler, M., and Friston, K. (2018). Computational mechanisms of curiosity and goal-directed exploration. Neuroscience.","DOI":"10.1101\/411272"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Millidge, B., Tschantz, A., and Buckley, C.L. (2020). Whence the Expected Free Energy?. arXiv.","DOI":"10.1162\/neco_a_01354"},{"key":"ref_30","unstructured":"Hafner, D., Ortega, P.A., Ba, J., Parr, T., Friston, K., and Heess, N. (2020). Action and Perception as Divergence Minimization. arXiv."},{"key":"ref_31","unstructured":"Buisson-Fenet, M., Solowjow, F., and Trimpe, S. (2020, January 11\u201312). Actively learning gaussian process dynamics. Proceedings of the 2nd Conference on Learning for Dynamics and Control, Online."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Bai, S., Wang, J., Chen, F., and Englot, B. (2016, January 9\u201314). Information-theoretic exploration with Bayesian optimization. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759289"},{"key":"ref_33","unstructured":"Berseth, G., Geng, D., Devin, C., Finn, C., Jayaraman, D., and Levine, S. (2019). SMiRL: Surprise Minimizing RL in Dynamic Environments. arXiv."},{"key":"ref_34","unstructured":"Friston, K. (2019). A free energy principle for a particular physics. arXiv."},{"key":"ref_35","unstructured":"Solopchuk, O. (2021). Information Theoretic Approach to Decision Making in Continuous Domains. [Ph.D. Thesis, UCL-Universit\u00e9 Catholique de Louvain]."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s10994-012-5278-7","article-title":"Optimal control as a graphical model inference problem","volume":"87","author":"Kappen","year":"2012","journal-title":"Mach. Learn."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2530","DOI":"10.1162\/neco_a_01108","article-title":"Active Inference, Belief Propagation, and the Bethe Approximation","volume":"30","author":"Schwoebel","year":"2018","journal-title":"Neural Comput."},{"key":"ref_38","unstructured":"Millidge, B., Tschantz, A., Seth, A., and Buckley, C. (2021). Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1565\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:35:25Z","timestamp":1760168125000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1565"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,24]]},"references-count":38,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["e23121565"],"URL":"https:\/\/doi.org\/10.3390\/e23121565","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,24]]}}}