{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T10:30:08Z","timestamp":1760524208661,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T00:00:00Z","timestamp":1622160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["program ZERO with project number P15-06"],"award-info":[{"award-number":["program ZERO with project number P15-06"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Time-varying autoregressive (TVAR) models are widely used for modeling of non-stationary signals. Unfortunately, online joint adaptation of both states and parameters in these models remains a challenge. In this paper, we represent the TVAR model by a factor graph and solve the inference problem by automated message passing-based inference for states and parameters. We derive structured variational update rules for a composite \u201cAR node\u201d with probabilistic observations that can be used as a plug-in module in hierarchical models, for example, to model the time-varying behavior of the hyper-parameters of a time-varying AR model. Our method includes tracking of variational free energy (FE) as a Bayesian measure of TVAR model performance. The proposed methods are verified on a synthetic data set and validated on real-world data from temperature modeling and speech enhancement tasks.<\/jats:p>","DOI":"10.3390\/e23060683","type":"journal-article","created":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T11:33:20Z","timestamp":1622201600000},"page":"683","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Message Passing-Based Inference for Time-Varying Autoregressive Models"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0515-0465","authenticated-orcid":false,"given":"Albert","family":"Podusenko","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0547-4817","authenticated-orcid":false,"given":"Wouter M.","family":"Kouw","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bert","family":"de Vries","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"},{"name":"GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/BF02532251","article-title":"Fitting autoregressive models for prediction","volume":"21","author":"Akaike","year":"1969","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/0165-1684(87)90002-8","article-title":"Results on AR-modelling of nonstationary signals","volume":"12","author":"Charbonnier","year":"1987","journal-title":"Signal Process."},{"key":"ref_3","unstructured":"Tahir, S.M., Shaameri, A.Z., and Salleh, S.H.S. (2001, January 13\u201316). Time-varying autoregressive modeling approach for speech segmentation. Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467), Kuala Lumpur, Malaysia."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1109\/TASL.2010.2073704","article-title":"Time-Varying Autoregressions in Speech: Detection Theory and Applications","volume":"19","author":"Rudoy","year":"2011","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chu, Y.J., Chan, S.C., Zhang, Z.G., and Tsui, K.M. (2012, January 5\u20138). A new regularized TVAR-based algorithm for recursive detection of nonstationarity and its application to speech signals. Proceedings of the 2012 IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, MI, USA.","DOI":"10.1109\/SSP.2012.6319704"},{"key":"ref_6","unstructured":"Paulik, M.J., Mohankrishnan, N., and Nikiforuk, M. (1994, January 3\u20135). A time varying vector autoregressive model for signature verification. Proceedings of the 1994 37th Midwest Symposium on Circuits and Systems, Lafayette, LA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3257","DOI":"10.1109\/TBME.2019.2903012","article-title":"A Novel Framework for Estimating Time-Varying Multivariate Autoregressive Models and Application to Cardiovascular Responses to Acute Exercise","volume":"66","author":"Kostoglou","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1023\/A:1008475713345","article-title":"Analysis of Acoustic Signatures from Moving Vehicles Using Time-Varying Autoregressive Models","volume":"10","author":"Eom","year":"1999","journal-title":"Multidimens. Syst. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1109\/TSP.2006.888064","article-title":"Time-Varying Autoregressive (TVAR) Models for Multiple Radar Observations","volume":"55","author":"Abramovich","year":"2007","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1109\/TBME.2010.2089686","article-title":"Local Polynomial Modeling of Time-Varying Autoregressive Models With Application to Time\u2013Frequency Analysis of Event-Related EEG","volume":"58","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_11","unstructured":"Wang, H., Bai, L., Xu, J., and Fei, W. (2015, January 15\u201317). EEG recognition through Time-varying Vector Autoregressive Model. Proceedings of the 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Zhangjiajie, China."},{"key":"ref_12","unstructured":"Sharman, K., and Friedlander, B. (1984, January 19\u201321). Time-varying autoregressive modeling of a class of nonstationary signals. Proceedings of the ICASSP\u201984\u2014IEEE International Conference on Acoustics, Speech, and Signal Processing, San Diego, CA, USA."},{"key":"ref_13","unstructured":"Reddy, G.R.S., and Rao, R. (2014, January 10\u201312). Non stationary signal prediction using TVAR model. Proceedings of the 2014 International Conference on Communication and Signal Processing, Bangkok, Thailand."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1109\/LSP.2018.2880086","article-title":"A Time-Varying Autoregressive Model for Characterizing Nonstationary Processes","volume":"26","author":"Souza","year":"2019","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_15","unstructured":"Zheng, Y., and Lin, Z. (2000, January 5\u20139). Time-varying autoregressive system identification using wavelets. Proceedings of the 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing (Cat. No.00CH37100), Istanbul, Turkey."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Moon, T.K., and Gunther, J.H. (2020). Estimation of Autoregressive Parameters from Noisy Observations Using Iterated Covariance Updates. Entropy, 22.","DOI":"10.3390\/e22050572"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1049\/ip-vis:19971305","article-title":"Bayesian approach to parameter estimation and interpolation of time-varying autoregressive processes using the Gibbs sampler","volume":"144","author":"Rajan","year":"1997","journal-title":"IEE Proc. Vis. Image Signal Process."},{"key":"ref_18","unstructured":"Prado, R., Huerta, G., and West, M. (2000). Bayesian tIme-Varying Autoregressions: Theory, Methods and Applications, University of Sao Paolo."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/j.jjie.2011.07.004","article-title":"Bayesian analysis of time-varying parameter vector autoregressive model for the Japanese economy and monetary policy","volume":"25","author":"Nakajima","year":"2011","journal-title":"J. Jpn. Int. Econ."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Barber, D. (2012). Bayesian Reasoning and Machine Learning, Cambridge University Press.","DOI":"10.1017\/CBO9780511804779"},{"key":"ref_21","unstructured":"Zhong, X., Song, S., and Pei, C. (2005, January 21\u201324). ime-varying Parameters Estimation based on Kalman Particle Filter with Forgetting Factors. Proceedings of the EUROCON 2005\u2014The International Conference on \u201cComputer as a Tool\u201d, Belgrade, Serbia."},{"key":"ref_22","first-page":"661","article-title":"Variational Message Passing","volume":"6","author":"Winn","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational Inference: A Review for Statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_24","unstructured":"Korl, S. (2005). A Factor Graph Approach to Signal Modelling, System Identification and Filtering. [Ph.D. Thesis, Swiss Federal Institute of Technology]."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1049\/ip-vis:20020149","article-title":"Bayesian multivariate autoregressive models with structured priors","volume":"149","author":"Penny","year":"2002","journal-title":"IEE Proc. Vis. Image Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1109\/JPROC.2007.896497","article-title":"The Factor Graph Approach to Model-Based Signal Processing","volume":"95","author":"Loeliger","year":"2007","journal-title":"Proc. IEEE"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Dauwels, J., Korl, S., and Loeliger, H.A. (2005). Expectation maximization as message passing. Int. Symp. Inf. Theory, 583\u2013586.","DOI":"10.1109\/ISIT.2005.1523402"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.ijar.2018.11.002","article-title":"A factor graph approach to automated design of Bayesian signal processing algorithms","volume":"104","author":"Cox","year":"2019","journal-title":"Int. J. Approx. Reason."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"De Vries, B., and Friston, K.J. (2017). A Factor Graph Description of Deep Temporal Active Inference. Front. Comput. Neurosci., 11.","DOI":"10.3389\/fncom.2017.00095"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Beck, J. (2010). Bayesian system identification based on probability logic. Struct. Control. Health Monit.","DOI":"10.1002\/stc.424"},{"key":"ref_31","unstructured":"Zhang, D., Song, X., Wang, W., Fettweis, G., and Gao, X. (2019). Unifying Message Passing Algorithms Under the Framework of Constrained Bethe Free Energy Minimization. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Dauwels, J. (2007, January 24\u201329). On Variational Message Passing on Factor Graphs. Proceedings of the IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557602"},{"key":"ref_33","unstructured":"Zhang, D., Wang, W., Fettweis, G., and Gao, X. (2017). Unifying Message Passing Algorithms Under the Framework of Constrained Bethe Free Energy Minimization. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1662","DOI":"10.1109\/LSP.2016.2614539","article-title":"Exact Distribution for the Product of Two Correlated Gaussian Random Variables","volume":"23","author":"Cui","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1109\/82.718814","article-title":"Subband Kalman filtering for speech enhancement","volume":"45","author":"Wu","year":"1998","journal-title":"IEEE Trans. Circuits Syst. Analog. Digit. Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1016\/j.specom.2011.02.001","article-title":"Modulation-domain Kalman filtering for single-channel speech enhancement","volume":"53","author":"So","year":"2011","journal-title":"Speech Commun."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nossier, S.A., Wall, J., Moniri, M., Glackin, C., and Cannings, N. (2021). An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement. Electronics, 10.","DOI":"10.3390\/electronics10010017"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1016\/j.specom.2006.12.006","article-title":"Subjective comparison and evaluation of speech enhancement algorithms","volume":"49","author":"Hu","year":"2007","journal-title":"Speech Commun."},{"key":"ref_39","unstructured":"Paliwal, K., and Basu, A. (1987, January 6\u20139). A speech enhancement method based on Kalman filtering. Proceedings of the ICASSP\u201987\u2014IEEE International Conference on Acoustics, Speech, and Signal Processing, Dallas, TX, USA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"You, C.H., Rahardja, S., and Koh, S.N. (2007, January 15\u201320). Autoregressive Parameter Estimation for Kalman Filtering Speech Enhancement. Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing\u2014ICASSP\u201907, Honolulu, HI, USA.","DOI":"10.1109\/ICASSP.2007.367219"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1109\/TASSP.1983.1164152","article-title":"Time-dependent ARMA modeling of nonstationary signals","volume":"31","author":"Grenier","year":"1983","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_42","unstructured":"Kamary, K., Mengersen, K., Robert, C.P., and Rousseau, J. (2014). Testing hypotheses via a mixture estimation model. arXiv."},{"key":"ref_43","unstructured":"Friston, K., Parr, T., and Zeidman, P. (2019). Bayesian model reduction. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Podusenko, A., Kouw, W.M., and de Vries, B. (2020, January 21\u201326). Online variational message passing in hierarchical autoregressive models. Proceedings of the 2020 IEEE International Symposium on Information Theory, Los Angeles, CA, USA.","DOI":"10.1109\/ISIT44484.2020.9174134"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/6\/683\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:10:07Z","timestamp":1760163007000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/6\/683"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,28]]},"references-count":44,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["e23060683"],"URL":"https:\/\/doi.org\/10.3390\/e23060683","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,5,28]]}}}