{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,19]],"date-time":"2026-07-19T03:28:12Z","timestamp":1784431692844,"version":"3.55.0"},"reference-count":45,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,5]],"date-time":"2021-10-05T00:00:00Z","timestamp":1633392000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The free energy principle from neuroscience has recently gained traction as one of the most prominent brain theories that can emulate the brain\u2019s perception and action in a bio-inspired manner. This renders the theory with the potential to hold the key for general artificial intelligence. Leveraging this potential, this paper aims to bridge the gap between neuroscience and robotics by reformulating an FEP-based inference scheme\u2014Dynamic Expectation Maximization\u2014into an algorithm that can perform simultaneous state, input, parameter, and noise hyperparameter estimation of any stable linear state space system subjected to colored noises. The resulting estimator was proved to be of the form of an augmented coupled linear estimator. Using this mathematical formulation, we proved that the estimation steps have theoretical guarantees of convergence. The algorithm was rigorously tested in simulation on a wide variety of linear systems with colored noises. The paper concludes by demonstrating the superior performance of DEM for parameter estimation under colored noise in simulation, when compared to the state-of-the-art estimators like Sub Space method, Prediction Error Minimization (PEM), and Expectation Maximization (EM) algorithm. These results contribute to the applicability of DEM as a robust learning algorithm for safe robotic applications.<\/jats:p>","DOI":"10.3390\/e23101306","type":"journal-article","created":{"date-parts":[[2021,10,7]],"date-time":"2021-10-07T09:24:04Z","timestamp":1633598644000},"page":"1306","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Dynamic Expectation Maximization Algorithm for Estimation of Linear Systems with Colored Noise"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2310-0579","authenticated-orcid":false,"given":"Ajith","family":"Anil Meera","sequence":"first","affiliation":[{"name":"Department of Cognitive Robotics, Faculty of Mechanical, Maritime and Materials Engineering, Delft Institute of Technology, 2628 CN Delft, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martijn","family":"Wisse","sequence":"additional","affiliation":[{"name":"Department of Cognitive Robotics, Faculty of Mechanical, Maritime and Materials Engineering, Delft Institute of Technology, 2628 CN Delft, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Anil Meera, A., and Wisse, M. (2020, January 1\u20133). Free Energy Principle Based State and Input Observer Design for Linear Systems with Colored Noise. Proceedings of the 2020 American Control Conference (ACC), Denver, CO, USA.","DOI":"10.23919\/ACC45564.2020.9147581"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bos, F., Anil Meera, A., Benders, D., and Wisse, M. (2021). Free Energy Principle for State and Input Estimation of a Quadcopter Flying in Wind. arXiv.","DOI":"10.1109\/ICRA46639.2022.9812415"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2973","DOI":"10.1109\/LRA.2020.2974451","article-title":"A novel adaptive controller for robot manipulators based on active inference","volume":"5","author":"Pezzato","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_5","unstructured":"Pezzato, C., Hernandez, C., and Wisse, M. (2020). Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Baioumy, M., Duckworth, P., Lacerda, B., and Hawes, N. (2020). Active inference for integrated state-estimation, control, and learning. arXiv.","DOI":"10.1109\/ICRA48506.2021.9562009"},{"key":"ref_7","unstructured":"Oliver, G., Lanillos, P., and Cheng, G. (2019). Active inference body perception and action for humanoid robots. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.neunet.2021.05.010","article-title":"Robot navigation as hierarchical active inference","volume":"142","author":"Verbelen","year":"2021","journal-title":"Neural Netw."},{"key":"ref_9","unstructured":"Lennart, L. (1999). System Identification: Theory for the User, PTR Prentice Hall."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, L.Q., Cichocki, A., and Amari, S. (2000). Kalman filter and state-space approach to blind deconvolution. Neural Networks for Signal Processing X, Proceedings of the 2000 IEEE Signal Processing Society Workshop, Sydney, Australia, 11\u201313 December 2000, IEEE.","DOI":"10.1109\/NNSP.2000.889435"},{"key":"ref_11","unstructured":"Han, R., Bohn, C., and Bauer, G. (2021). Blind identification of state-space models in physical coordinates. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1310","DOI":"10.1109\/5.622507","article-title":"Blind system identification","volume":"85","author":"Qiu","year":"1997","journal-title":"Proc. IEEE"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1810","DOI":"10.1016\/j.mcm.2010.12.059","article-title":"Unbiased identification of a class of multi-input single-output systems with correlated disturbances using bias compensation methods","volume":"53","author":"Zhang","year":"2011","journal-title":"Math. Comput. Model."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/j.automatica.2010.01.007","article-title":"Least squares based iterative identification for a class of multirate systems","volume":"46","author":"Liu","year":"2010","journal-title":"Automatica"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1109\/78.678479","article-title":"On a least-squares-based algorithm for identification of stochastic linear systems","volume":"46","author":"Zheng","year":"1998","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1709","DOI":"10.1016\/j.apm.2010.10.003","article-title":"Bias compensation methods for stochastic systems with colored noise","volume":"35","author":"Zhang","year":"2011","journal-title":"Appl. Math. Model."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1002\/acs.3101","article-title":"Combined estimation of the parameters and states for a multivariable state-space system in presence of colored noise","volume":"34","author":"Cui","year":"2020","journal-title":"Int. J. Adapt. Control. Signal Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/j.neuroimage.2008.02.054","article-title":"DEM: A variational treatment of dynamic systems","volume":"41","author":"Friston","year":"2008","journal-title":"Neuroimage"},{"key":"ref_19","unstructured":"Van de Laar, T., \u00d6z\u00e7elikkale, A., and Wymeersch, H. (2019). Application of the free energy principle to estimation and control. arXiv."},{"key":"ref_20","unstructured":"Anil Meera, A., and Wisse, M. (2021). A Brain Inspired Learning Algorithm for the Perception of a Quadrotor in Wind. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: A mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hohwy, J. (2013). The Predictive Mind, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199682737.001.0001"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.tics.2009.04.005","article-title":"The free-energy principle: A rough guide to the brain?","volume":"13","author":"Friston","year":"2009","journal-title":"Trends Cogn. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s00422-010-0364-z","article-title":"Action and behavior: A free-energy formulation","volume":"102","author":"Friston","year":"2010","journal-title":"Biol. Cybern."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1093\/brain\/awq010","article-title":"The default-mode, ego-functions and free-energy: A neurobiological account of Freudian ideas","volume":"133","author":"Friston","year":"2010","journal-title":"Brain"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Friston, K.J., Daunizeau, J., and Kiebel, S.J. (2009). Reinforcement learning or active inference?. PLoS ONE, 4.","DOI":"10.1371\/journal.pone.0006421"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Friston, K. (2008). Hierarchical models in the brain. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000211"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Baltieri, M., and Buckley, C.L. (2019). PID control as a process of active inference with linear generative models. Entropy, 21.","DOI":"10.20944\/preprints201902.0246.v1"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e90","DOI":"10.1017\/S0140525X19001213","article-title":"Thinking through other minds: A variational approach to cognition and culture","volume":"43","author":"Constant","year":"2020","journal-title":"Behav. Brain Sci."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kaufmann, R., Gupta, P., and Taylor, J. (2021). An active inference model of collective intelligence. arXiv.","DOI":"10.3390\/e23070830"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-018-04494-0","article-title":"Innovation and cumulative culture through tweaks and leaps in online programming contests","volume":"9","author":"Miu","year":"2018","journal-title":"Nat. Commun."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1211","DOI":"10.1098\/rstb.2008.0300","article-title":"Predictive coding under the free-energy principle","volume":"364","author":"Friston","year":"2009","journal-title":"Philos. Trans. R. Soc. Biol. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-018-38246-3","article-title":"Neuronal message passing using Mean-field, Bethe, and Marginal approximations","volume":"9","author":"Parr","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"20","DOI":"10.3389\/frobt.2019.00020","article-title":"Simulating active inference processes by message passing","volume":"6","year":"2019","journal-title":"Front. Robot. AI"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active inference and learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1016\/j.neuroimage.2008.03.017","article-title":"Variational filtering","volume":"41","author":"Friston","year":"2008","journal-title":"NeuroImage"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Friston, K., Stephan, K., Li, B., and Daunizeau, J. (2010). Generalised filtering. Math. Probl. Eng., 2010.","DOI":"10.1155\/2010\/621670"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1016\/j.neuroimage.2006.08.035","article-title":"Variational free energy and the Laplace approximation","volume":"34","author":"Friston","year":"2007","journal-title":"Neuroimage"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Balaji, B., and Friston, K. (2011, January 5). Bayesian state estimation using generalized coordinates. Proceedings of the Signal Processing, Sensor Fusion, and Target Recognition XX, Orlando, FL, USA.","DOI":"10.1117\/12.883513"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Anil Meera, A., and Wisse, M. (2021). On the convergence of DEM\u2019s linear parameter estimator. International Workshop on Active Inference, Springer. accepted.","DOI":"10.1007\/978-3-030-93736-2_49"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/j.neuroimage.2011.01.085","article-title":"Generalised filtering and stochastic DCM for fMRI","volume":"58","author":"Li","year":"2011","journal-title":"Neuroimage"},{"key":"ref_43","first-page":"222","article-title":"A numerically efficient implementation of the expectation maximization algorithm for state space models","volume":"241","author":"Mader","year":"2014","journal-title":"Appl. Math. Comput."},{"key":"ref_44","unstructured":"Friston, K. (2019). A free energy principle for a particular physics. arXiv."},{"key":"ref_45","first-page":"3","article-title":"Using the EM algorithm to estimate the state space model for OMAX","volume":"1000","author":"Cara","year":"2014","journal-title":"Practice"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1306\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:09:31Z","timestamp":1760166571000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1306"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,5]]},"references-count":45,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["e23101306"],"URL":"https:\/\/doi.org\/10.3390\/e23101306","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,5]]}}}