{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T05:24:08Z","timestamp":1784697848810,"version":"3.55.0"},"reference-count":127,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T00:00:00Z","timestamp":1618185600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001866","name":"Fonds National de la Recherche Luxembourg","doi-asserted-by":"publisher","award":["13568875"],"award-info":[{"award-number":["13568875"]}],"id":[{"id":"10.13039\/501100001866","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010269","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["088130\/Z\/09\/Z"],"award-info":[{"award-number":["088130\/Z\/09\/Z"]}],"id":[{"id":"10.13039\/100010269","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000833","name":"Rosetrees Trust","doi-asserted-by":"publisher","award":["173346"],"award-info":[{"award-number":["173346"]}],"id":[{"id":"10.13039\/501100000833","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Active inference is a normative framework for explaining behaviour under the free energy principle\u2014a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on (variational) free energy\u2014a measure of the fit between an internal (generative) model and sensory observations. The free energy gradient is a prediction error\u2014plausibly encoded in the average membrane potentials of neuronal populations. Conversely, the expected probability of a state can be expressed in terms of neuronal firing rates. We show that this is consistent with current models of neuronal dynamics and establish face validity by synthesising plausible electrophysiological responses. We then show that these neuronal dynamics approximate natural gradient descent, a well-known optimisation algorithm from information geometry that follows the steepest descent of the objective in information space. We compare the information length of belief updating in both schemes, a measure of the distance travelled in information space that has a direct interpretation in terms of metabolic cost. We show that neural dynamics under active inference are metabolically efficient and suggest that neural representations in biological agents may evolve by approximating steepest descent in information space towards the point of optimal inference.<\/jats:p>","DOI":"10.3390\/e23040454","type":"journal-article","created":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T11:05:06Z","timestamp":1618225506000},"page":"454","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0126-4588","authenticated-orcid":false,"given":"Lancelot","family":"Da Costa","sequence":"first","affiliation":[{"name":"Department of Mathematics, Imperial College London, London SW7 2AZ, UK"},{"name":"Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5108-5743","authenticated-orcid":false,"given":"Thomas","family":"Parr","sequence":"additional","affiliation":[{"name":"Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Biswa","family":"Sengupta","sequence":"additional","affiliation":[{"name":"Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK"},{"name":"Core Machine Learning Group, Zebra AI, London WC2H 8TJ, UK"},{"name":"Department of Bioengineering, Imperial College London, London SW7 2AZ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7984-8909","authenticated-orcid":false,"given":"Karl","family":"Friston","sequence":"additional","affiliation":[{"name":"Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.jmp.2015.11.003","article-title":"A tutorial on the free-energy framework for modelling perception and learning","volume":"76","author":"Bogacz","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.tics.2009.04.005","article-title":"The free-energy principle: A rough guide to the brain?","volume":"13","author":"Friston","year":"2009","journal-title":"Trends Cogn. Sci."},{"key":"ref_3","first-page":"70","article-title":"A free energy principle for the brain","volume":"100","author":"Friston","year":"2006","journal-title":"J. Physiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"20130475","DOI":"10.1098\/rsif.2013.0475","article-title":"Life as we know it","volume":"10","author":"Friston","year":"2013","journal-title":"J. R. Soc. Interface"},{"key":"ref_5","unstructured":"Friston, K. (2020, February 29). A Free Energy Principle for a Particular Physics. arXiv:190610184 [q-bio]. Available online: http:\/\/arxiv.org\/abs\/1906.10184."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"20190159","DOI":"10.1098\/rsta.2019.0159","article-title":"Markov blankets, information geometry and stochastic thermodynamics","volume":"378","author":"Parr","year":"2020","journal-title":"Philos. Trans. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"102447","DOI":"10.1016\/j.jmp.2020.102447","article-title":"Active inference on discrete state-spaces: A synthesis","volume":"99","author":"Parr","year":"2020","journal-title":"J. Math. Psychol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1098\/rstb.2005.1622","article-title":"A theory of cortical responses","volume":"360","author":"Friston","year":"2005","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_9","unstructured":"Beal, M.J. (2003). Variational Algorithms for Approximate Bayesian Inference. [Ph.D. Thesis, University of London]."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Jordan, M.I. (1998). An Introduction to Variational Methods for Graphical Models. Learning in Graphical Models, Springer.","DOI":"10.1007\/978-94-011-5014-9"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000001","article-title":"Graphical Models, Exponential Families, and Variational Inference","volume":"1","author":"Wainwright","year":"2007","journal-title":"FNT Mach. Learn."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: A mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Colombo, M., and Wright, C. (2018). First principles in the life sciences: The free-energy principle, organicism, and mechanism. Synthese.","DOI":"10.1007\/s11229-018-01932-w"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s10994-012-5278-7","article-title":"Optimal control as a graphical model inference problem","volume":"87","author":"Kappen","year":"2012","journal-title":"Mach. Learn."},{"key":"ref_16","unstructured":"Da Costa, L., Sajid, N., Parr, T., Friston, K., and Smith, R. (2021, January 31). The Relationship between Dynamic Programming and Active Inference: The Discrete, Finite-Horizon Case. arXiv:200908111 [cs, math, q-bio]. Available online: http:\/\/arxiv.org\/abs\/2009.08111."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Millidge, B., Tschantz, A., Seth, A.K., and Buckley, C.L. (2020, June 28). On the Relationship between Active Inference and Control as Inference. arXiv:200612964 [cs, stat]. Available online: http:\/\/arxiv.org\/abs\/2006.12964.","DOI":"10.1007\/978-3-030-64919-7_1"},{"key":"ref_18","unstructured":"Watson, J., Imohiosen, A., and Peters, J. (2021, January 27). Active Inference or Control as Inference? A Unifying View. arXiv:201000262 [cs, stat]. Available online: http:\/\/arxiv.org\/abs\/2010.00262."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.conb.2017.08.010","article-title":"With or without you: Predictive coding and Bayesian inference in the brain","volume":"46","author":"Aitchison","year":"2017","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1016\/j.tins.2004.10.007","article-title":"The Bayesian brain: The role of uncertainty in neural coding and computation","volume":"27","author":"Knill","year":"2004","journal-title":"Trends Neurosci."},{"key":"ref_21","unstructured":"Lake, B.M., Ullman, T.D., Tenenbaum, J.B., and Gershman, S.J. (2019, August 11). Building Machines That Learn and Think Like People. arXiv:160400289 [cs, stat]. Available online: http:\/\/arxiv.org\/abs\/1604.00289."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nat. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1016\/j.neuron.2012.10.038","article-title":"Canonical Microcircuits for Predictive Coding","volume":"76","author":"Bastos","year":"2012","journal-title":"Neuron"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1211","DOI":"10.1098\/rstb.2008.0300","article-title":"Predictive coding under the free-energy principle","volume":"364","author":"Friston","year":"2009","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s00422-018-0753-2","article-title":"Planning and navigation as active inference","volume":"112","author":"Kaplan","year":"2018","journal-title":"Biol. Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"478","DOI":"10.3389\/fpsyg.2012.00478","article-title":"An Active Inference view of cognitive control","volume":"3","author":"Pezzulo","year":"2012","journal-title":"Front. Psychol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2530","DOI":"10.1162\/neco_a_01108","article-title":"Active Inference, Belief Propagation, and the Bethe Approximation","volume":"30","author":"Kiebel","year":"2018","journal-title":"Neural Comput."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Matsumoto, T., and Tani, J. (2020). Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network. Entropy, 22.","DOI":"10.3390\/e22050564"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"\u00c7atal, O., Verbelen, T., Nauta, J., Boom, C.D., and Dhoedt, B. (2020, January 4\u20138). Learning Perception and Planning With Deep Active Inference. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054364"},{"key":"ref_30","unstructured":"\u00c7atal, O., Wauthier, S., Verbelen, T., De Boom, C., and Dhoedt, B. (2020, May 22). Deep Active Inference for Autonomous Robot Navigation. arXiv:200303220. Available online: http:\/\/arxiv.org\/abs\/2003.03220."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sancaktar, C., van Gerven, M., and Lanillos, P. (2020, September 18). End-to-End Pixel-Based Deep Active Inference for Body Perception and Action. arXiv:200105847 [cs, q-bio]. 2020 [cited 18 Sep 2020]. Available online: http:\/\/arxiv.org\/abs\/2001.05847.","DOI":"10.1109\/ICDL-EpiRob48136.2020.9278105"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Tschantz, A., Seth, A.K., and Buckley, C.L. (2020). Learning action-oriented models through active inference. PLoS Comput. Biol., 16.","DOI":"10.1371\/journal.pcbi.1007805"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"45","DOI":"10.7554\/eLife.41703","article-title":"Computational mechanisms of curiosity and goal-directed exploration","volume":"8","author":"Schwartenbeck","year":"2019","journal-title":"eLife"},{"key":"ref_34","unstructured":"Tschantz, A., Millidge, B., Seth, A.K., and Buckley, C.L. (2021, April 09). Reinforcement Learning through Active Inference. ICLR. Available online: http:\/\/arxiv.org\/abs\/2002.12636."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Markovi\u0107, D., Goschke, T., and Kiebel, S.J. (2020). Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales. Cogn. Affect. Behav. Neurosci.","DOI":"10.1101\/847566"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1162\/neco_a_01351","article-title":"Sophisticated Inference","volume":"33","author":"Friston","year":"2021","journal-title":"Neural Comput."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lanillos, P.G. (2018, January 1\u20135). Adaptive Robot Body Learning and Estimation Through Predictive Coding. Proceedings of the2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593684"},{"key":"ref_38","unstructured":"Lanillos, P., Pages, J., and Cheng, G. (2020, September 18). Robot Self\/Other Distinction: Active Inference Meets Neural Networks Learning in a Mirror. arXiv:200405473 [cs]. Available online: http:\/\/arxiv.org\/abs\/2004.05473."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1162\/neco_a_00999","article-title":"Active Inference, Curiosity and Insight","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Smith, R., Schwartenbeck, P., Parr, T., and Friston, K.J. (2020). An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case. Front. Comput. Neurosci., 14.","DOI":"10.3389\/fncom.2020.00041"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Wauthier, S.T., \u00c7atal, O., Verbelen, T., and Dhoedt, B. (2020). Sleep: Model Reduction. Deep Active Inference, Springer.","DOI":"10.1007\/978-3-030-64919-7_9"},{"key":"ref_42","first-page":"809","article-title":"Active Inference in OpenAI Gym: A Paradigm for Computational Investigations into Psychiatric Illness","volume":"3","author":"Cullen","year":"2018","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"ref_43","unstructured":"Fountas, Z., Sajid, N., Mediano, P.A.M., and Friston, K. (2020, July 16). Deep Active Inference Agents Using Monte-Carlo Methods. arXiv:200604176 [cs, q-bio, stat]. Available online: http:\/\/arxiv.org\/abs\/2006.04176."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1007\/s00422-018-0785-7","article-title":"Deep active inference","volume":"112","year":"2018","journal-title":"Biol. Cybern."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Markovi\u0107, D., Reiter, A.M.F., and Kiebel, S.J. (2019). Predicting change: Approximate inference under explicit representation of temporal structure in changing environments. PLoS Comput. Biol., 15.","DOI":"10.1371\/journal.pcbi.1006707"},{"key":"ref_46","unstructured":"Markovic, D., Stojic, H., Schwoebel, S., and Kiebel, S.J. (2021, January 26). An Empirical Evaluation of Active Inference in Multi-Armed Bandits. arXiv:210108699 [cs]. Available online: http:\/\/arxiv.org\/abs\/2101.08699."},{"key":"ref_47","unstructured":"Sajid, N., Ball, P.J., and Friston, K.J. (2020, April 30). Active Inference: Demystified and Compared. arXiv:190910863 [cs, q-bio]. Available online: http:\/\/arxiv.org\/abs\/1909.10863."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.neuropsychologia.2018.01.041","article-title":"Active inference and the anatomy of oculomotion","volume":"111","author":"Parr","year":"2018","journal-title":"Neuropsychologia"},{"key":"ref_49","unstructured":"Parr, T. (2019). The Computational Neurology of Active Vision. [Ph.D. Thesis, University College London]."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"56","DOI":"10.3389\/fncom.2016.00056","article-title":"Scene Construction, Visual Foraging, and Active Inference","volume":"10","author":"Mirza","year":"2016","journal-title":"Front. Comput. Neurosci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"20170376","DOI":"10.1098\/rsif.2017.0376","article-title":"Uncertainty, epistemics and active inference","volume":"14","author":"Parr","year":"2017","journal-title":"J. R. Soc. Interface"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Adams, R.A., Stephan, K.E., Brown, H.R., Frith, C.D., and Friston, K.J. (2013). The Computational Anatomy of Psychosis. Front. Psychiatry, 4.","DOI":"10.3389\/fpsyt.2013.00047"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Smith, R., Kirlic, N., Stewart, J.L., Touthang, J., Kuplicki, R., Khalsa, S.S., Feinstein, J., Paulus, M.P., and Aupperle, R.L. (2020). Greater decision uncertainty characterizes a transdiagnostic patient sample during approach-avoidance conflict: A computational modeling approach. PsyArXiv.","DOI":"10.31234\/osf.io\/t2dhn"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Smith, R., Schwartenbeck, P., Stewart, J.L., Kuplicki, R., Ekhtiari, H., Investigators, T., and Martin, P. (2020). Imprecise Action Selection in Substance Use Disorder: Evidence for Active Learning Impairments When Solving the Explore-Exploit Dilemma. PsyArXiv.","DOI":"10.31234\/osf.io\/a794k"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Smith, R., Kuplicki, R., Feinstein, J., Forthman, K.L., Stewart, J.L., Paulus, M.P., Tulsa 1000 Investigators, and Khalsa, S.S. (2020). A Bayesian computational model reveals a failure to adapt interoceptive precision estimates across depression, anxiety, eating, and substance use disorders. PLoS Comput. Biol., 16.","DOI":"10.1101\/2020.06.03.20121343"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"102348","DOI":"10.1016\/j.jmp.2020.102348","article-title":"Deep active inference as variational policy gradients","volume":"96","author":"Millidge","year":"2020","journal-title":"J. Math. Psychol."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO_a_00912","article-title":"Active Inference: A Process Theory","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"90","DOI":"10.3389\/fncom.2018.00090","article-title":"The Anatomy of Inference: Generative Models and Brain Structure","volume":"12","author":"Parr","year":"2018","journal-title":"Front. Comput. Neurosci."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1093\/cercor\/bhu159","article-title":"The Dopaminergic Midbrain Encodes the Expected Certainty about Desired Outcomes","volume":"25","author":"Schwartenbeck","year":"2015","journal-title":"Cereb. Cortex"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Fitzgerald, T.H.B., Dolan, R.J., and Friston, K. (2015). Dopamine, reward learning, and active inference. Front. Comput. Neurosci., 9.","DOI":"10.3389\/fncom.2015.00136"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"710","DOI":"10.3389\/fpsyg.2013.00710","article-title":"Exploration, novelty, surprise, and free energy minimization","volume":"4","author":"Schwartenbeck","year":"2013","journal-title":"Front. Psychol."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"16575","DOI":"10.1038\/srep16575","article-title":"Evidence for surprise minimization over value maximization in choice behavior","volume":"5","author":"Schwartenbeck","year":"2015","journal-title":"Sci. Rep."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Sengupta, B., Stemmler, M.B., and Friston, K.J. (2013). Information and Efficiency in the Nervous System\u2014A Synthesis. PLoS Comput. Biol., 9.","DOI":"10.1371\/journal.pcbi.1003157"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1162\/neco.1996.8.3.531","article-title":"Energy Efficient Neural Codes","volume":"8","author":"Levy","year":"1996","journal-title":"Neural Comput."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"3351","DOI":"10.1523\/JNEUROSCI.16-10-03351.1996","article-title":"Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory","volume":"16","author":"Dan","year":"1996","journal-title":"J. Neurosci."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1038\/nn831","article-title":"Efficient coding of natural sounds","volume":"5","author":"Lewicki","year":"2002","journal-title":"Nat. Neurosci."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"4723","DOI":"10.1073\/pnas.0506806103","article-title":"Wiring optimization can relate neuronal structure and function","volume":"103","author":"Chen","year":"2006","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Raj, A., and Chen, Y.-H. (2011). The Wiring Economy Principle: Connectivity Determines Anatomy in the Human Brain. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0014832"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/net.12.3.241.253","article-title":"Redundancy reduction revisited","volume":"12","author":"Barlow","year":"2001","journal-title":"Comput. Neural Syst."},{"key":"ref_70","unstructured":"Barlow, H.B. (2021, April 09). Possible Principles Underlying the Transformations of Sensory Messages; The MIT Press: 1961. Available online: https:\/\/www.universitypressscholarship.com\/view\/10.7551\/mitpress\/9780262518420.001.0001\/upso-9780262518420-chapter-13."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Binder, M.D., Hirokawa, N., and Windhorst, U. (2009). Efficient Coding Hypothesis. Encyclopedia of Neuroscience, Springer.","DOI":"10.1007\/978-3-540-29678-2"},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1038\/nn.4243","article-title":"Efficient codes and balanced networks","volume":"19","author":"Machens","year":"2016","journal-title":"Nat. Neurosci."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"16344","DOI":"10.1073\/pnas.0807744105","article-title":"Efficient coding in heterogeneous neuronal populations","volume":"105","author":"Chelaru","year":"2008","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Kostal, L., L\u00e1nsk\u00fd, P., and Rospars, J.-P. (2008). Efficient Olfactory Coding in the Pheromone Receptor Neuron of a Moth. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000053"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1088\/0954-898X_7_2_014","article-title":"Natural image statistics and efficient coding","volume":"7","author":"Olshausen","year":"1996","journal-title":"Net. Comput. Neural Syst."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1038\/nn0402-292","article-title":"A new window on sound","volume":"5","author":"Olshausen","year":"2002","journal-title":"Nat. Neurosci."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1146\/annurev.neuro.24.1.1193","article-title":"Natural Image Statistics and Neural Representation","volume":"24","author":"Simoncelli","year":"2001","journal-title":"Annu. Rev. Neurosci."},{"key":"ref_78","unstructured":"Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., and Weinberger, K.Q. (2011). Efficient coding of natural images with a population of noisy Linear-Nonlinear neurons. Advances in Neural Information Processing Systems 24, Curran Associates, Inc."},{"key":"ref_79","first-page":"607","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nat. Cell Biol."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1511\/2000.3.238","article-title":"Vision and the Coding of Natural Images: The human brain may hold the secrets to the best image-compression algorithms","volume":"88","author":"Olshausen","year":"2000","journal-title":"Am. Sci."},{"key":"ref_81","first-page":"501","article-title":"Notes on Landauer\u2019s principle, reversible computation, and Maxwell\u2019s Demon","volume":"34","author":"Bennett","year":"2003","journal-title":"Stud. Hist. Philos. Sci. Part B: Stud. Hist. Philos. Mod. Phys."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1147\/rd.53.0183","article-title":"Irreversibility and Heat Generation in the Computing Process","volume":"5","author":"Landauer","year":"1961","journal-title":"IBM J. Res. Dev."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"030605","DOI":"10.1103\/PhysRevLett.121.030605","article-title":"Stochastic Thermodynamic Interpretation of Information Geometry","volume":"121","author":"Ito","year":"2018","journal-title":"Phys. Rev. Lett."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"100602","DOI":"10.1103\/PhysRevLett.99.100602","article-title":"Measuring Thermodynamic Length","volume":"99","author":"Crooks","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","article-title":"Natural Gradient Works Efficiently in Learning","volume":"10","author":"Amari","year":"1998","journal-title":"Neural Comput."},{"key":"ref_86","first-page":"55","article-title":"A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue","volume":"13","author":"Wilson","year":"1973","journal-title":"Biol. Cybern."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1111\/j.1469-7793.1999.0791s.x","article-title":"Cell-attached measurements of the firing threshold of rat hippocampal neurones","volume":"517","author":"Fricker","year":"1999","journal-title":"J. Physiol."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.neuroimage.2008.04.239","article-title":"Population dynamics: Variance and the sigmoid activation function","volume":"42","author":"Marreiros","year":"2008","journal-title":"NeuroImage"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1016\/j.neuroimage.2008.10.008","article-title":"Population dynamics under the Laplace assumption","volume":"44","author":"Marreiros","year":"2009","journal-title":"NeuroImage"},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1016\/S1053-8119(03)00202-7","article-title":"Dynamic causal modelling","volume":"19","author":"Friston","year":"2003","journal-title":"NeuroImage"},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Deco, G., Jirsa, V.K., Robinson, P.A., Breakspear, M., and Friston, K. (2008). The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000092"},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"57","DOI":"10.3389\/fncom.2013.00057","article-title":"Neural masses and fields in dynamic causal modeling","volume":"7","author":"Moran","year":"2013","journal-title":"Front. Comput. Neurosci."},{"key":"ref_93","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_94","unstructured":"Stone, J.V. (2019). Artificial Intelligence Engines: A Tutorial Introduction to the Mathematics of Deep Learning, Sebtel Press."},{"key":"ref_95","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1111\/j.1467-9574.1988.tb01238.x","article-title":"Polytomous logistic regression","volume":"42","author":"Engel","year":"1988","journal-title":"Stat. Neerlandica"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Huang, F.-L., Hsieh, C.-J., Chang, K.-W., and Lin, C.-J. (2009). Iterative scaling and coordinate descent methods for maximum entropy. ACL-IJCNLP 2009 Conf. Short Pap.","DOI":"10.3115\/1667583.1667671"},{"key":"ref_97","unstructured":"Rodr\u00edguez, G. (2021, April 09). Lecture Notes on Generalized Linear Models. Available online: https:\/\/data.princeton.edu\/wws509\/notes\/."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/0022-247X(65)90154-X","article-title":"Optimal control of Markov processes with incomplete state information","volume":"10","year":"1965","journal-title":"J. Math. Anal. Appl."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On Information and Sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann Math Statist."},{"key":"ref_100","doi-asserted-by":"crossref","unstructured":"Lovric, M. (2011). Kullback-Leibler Divergence. International Encyclopedia of Statistical Science, Springer.","DOI":"10.1007\/978-3-642-04898-2"},{"key":"ref_101","unstructured":"Rezende, D.J. (2021, April 09). Short Notes on Divergence Measures. Available online: https:\/\/danilorezende.com\/wp-content\/uploads\/2018\/07\/divergences.pdf."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"1643","DOI":"10.1038\/nn.4650","article-title":"The hippocampus as a predictive map","volume":"20","author":"Stachenfeld","year":"2017","journal-title":"Nat. Neurosci."},{"key":"ref_103","first-page":"801","article-title":"Microstructure of a spatial map in the entorhinal cortex","volume":"436","author":"Hafting","year":"2005","journal-title":"Nat. Cell Biol."},{"key":"ref_104","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1007\/BF00243212","article-title":"Head-direction cells in the rat posterior cortex","volume":"16","author":"Chen","year":"1994","journal-title":"Exp. Brain Res."},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1523\/JNEUROSCI.10-02-00420.1990","article-title":"Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis","volume":"10","author":"Taube","year":"1990","journal-title":"J. Neurosci."},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1113\/jphysiol.2004.068668","article-title":"Coding of position by simultaneously recorded sensory neurones in the cat dorsal root ganglion","volume":"560","author":"Stein","year":"2004","journal-title":"J. Physiol."},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"016002","DOI":"10.1088\/1741-2560\/8\/1\/016002","article-title":"State-space decoding of primary afferent neuron firing rates","volume":"8","author":"Wagenaar","year":"2011","journal-title":"J. Neural Eng."},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1109\/TNSRE.2006.875575","article-title":"Decoding Sensory Feedback From Firing Rates of Afferent Ensembles Recorded in Cat Dorsal Root Ganglia in Normal Locomotion","volume":"14","author":"Weber","year":"2006","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1113\/jphysiol.1959.sp006308","article-title":"Receptive fields of single neurones in the cat\u2019s striate cortex","volume":"148","author":"Hubel","year":"1959","journal-title":"J. Physiol."},{"key":"ref_110","doi-asserted-by":"crossref","unstructured":"Amari, S. (2016). Information Geometry and Its Applications, Springer.","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_111","doi-asserted-by":"crossref","unstructured":"Ay, N., Jost, J., L\u00ea, H.V., and Schwachh\u00f6fer, L. (2017). Information Geometry, Springer.","DOI":"10.1007\/978-3-319-56478-4"},{"key":"ref_112","unstructured":"Nielsen, F. (2019, August 11). An Elementary Introduction to Information Geometry. arXiv:180808271 [cs, math, stat]. Available online: http:\/\/arxiv.org\/abs\/1808.08271."},{"key":"ref_113","unstructured":"Cencov, N.N. (1982). Statistical Decision Rules and Optimal Inference, American Mathematical Society."},{"key":"ref_114","unstructured":"Liang, T., Poggio, T., Rakhlin, A., and Stokes, J. (2019, August 11). Fisher-Rao Metric, Geometry, and Complexity of Neural Networks. arXiv:171101530 [cs, stat]. Available online: http:\/\/arxiv.org\/abs\/1711.01530."},{"key":"ref_115","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley.","DOI":"10.1002\/047174882X"},{"key":"ref_116","unstructured":"Amari, S., and Douglas, S.C. (1998, January 12\u201315). Why natural gradient?. Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP \u201998 (Cat No98CH36181), Seattle, WA, USA."},{"key":"ref_117","first-page":"5941","article-title":"Exact natural gradient in deep linear networks and its application to the nonlinear case","volume":"31","author":"Bernacchia","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_118","unstructured":"Zonghai, S., and Buhai, S. (2021, April 09). The Projection Adaptive Natural Gradient Online Algorithm for SVM. Available online: https:\/\/www.infona.pl\/\/resource\/bwmeta1.element.ieee-art-000005573523."},{"key":"ref_119","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/97.796292","article-title":"Natural gradient algorithm for blind separation of overdetermined mixture with additive noise","volume":"6","author":"Zhang","year":"1999","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_120","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1002\/oca.874","article-title":"Natural gradient-projection algorithm for distribution control","volume":"30","author":"Zhang","year":"2008","journal-title":"Optim. Control. Appl. Methods"},{"key":"ref_121","unstructured":"Duan, T., Anand, A., Ding, D.Y., Thai, K.K., Basu, S., Ng, A., and Schuler, A. (2021, April 09). NGBoost: Natural Gradient Boosting for Probabilistic Prediction. Available online: http:\/\/proceedings.mlr.press\/v119\/duan20a.html."},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MSP.2004.1267047","article-title":"An Introduction to factor graphs","volume":"21","author":"Loeliger","year":"2004","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Mirza, M.B., Adams, R.A., Mathys, C., and Friston, K.J. (2018). Human visual exploration reduces uncertainty about the sensed world. PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0190429"},{"key":"ref_124","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1038\/s41598-018-38246-3","article-title":"Neuronal message passing using Mean-field, Bethe, and Marginal approximations","volume":"9","author":"Parr","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_125","doi-asserted-by":"crossref","first-page":"2282","DOI":"10.1109\/TIT.2005.850085","article-title":"Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms","volume":"51","author":"Yedidia","year":"2005","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_126","doi-asserted-by":"crossref","unstructured":"Dauwels, J. (2007, January 24\u201329). On Variational Message Passing on Factor Graphs. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557602"},{"key":"ref_127","first-page":"661","article-title":"Variational Message Passing","volume":"34","author":"Winn","year":"2005","journal-title":"J. Mach. Learn. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/454\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:59:31Z","timestamp":1760363971000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/454"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,12]]},"references-count":127,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["e23040454"],"URL":"https:\/\/doi.org\/10.3390\/e23040454","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,12]]}}}