{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T02:01:10Z","timestamp":1792548070029,"version":"4.5.0"},"reference-count":54,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100006445","name":"NSF","doi-asserted-by":"publisher","award":["DMS-2210717"],"award-info":[{"award-number":["DMS-2210717"]}],"id":[{"id":"10.13039\/100006445","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>This article considers Bayesian model selection via mean-field (MF) variational approximation. Towards this goal, we study the non-asymptotic properties of MF inference that allows latent variables and model misspecification. Concretely, we show a Bernstein\u2013von Mises (BvM) theorem for the variational distribution from MF under possible model misspecification, which implies the distributional convergence of MF variational approximation to a normal distribution centring at the maximal likelihood estimator. Motivated by the BvM theorem, we propose a model selection criterion using the evidence lower bound (ELBO), and demonstrate that the model selected by ELBO tends to asymptotically agree with the one selected by the commonly used Bayesian information criterion (BIC) as the sample size tends to infinity. Compared to BIC, ELBO tends to incur smaller approximation error to the log-marginal likelihood (a.k.a. model evidence) due to a better dimension dependence and full incorporation of the prior information. Moreover, we show the geometric convergence of the coordinate ascent variational inference algorithm, which provides a practical guidance on how many iterations one typically needs to run when approximating the ELBO. These findings demonstrate that variational inference is capable of providing a computationally efficient alternative to conventional approaches in tasks beyond obtaining point estimates.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkad164","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T20:17:06Z","timestamp":1712693826000},"page":"742-770","source":"Crossref","is-referenced-by-count":17,"title":["Bayesian model selection via mean-field variational approximation"],"prefix":"10.1093","volume":"86","author":[{"given":"Yangfan","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Illinois Urbana-Champaign , Champaign , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6080-0470","authenticated-orcid":false,"given":"Yun","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Illinois Urbana-Champaign , Champaign , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,4,9]]},"reference":[{"issue":"3","key":"2024071202434233600_qkad164-B1","doi-asserted-by":"crossref","first-page":"1475","DOI":"10.1214\/19-AOS1855","article-title":"Concentration of tempered posteriors and of their variational approximations","volume":"48","author":"Alquier","year":"2020","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B2","first-page":"8374","article-title":"On the properties of variational approximations of Gibbs posteriors","volume":"17","author":"Alquier","year":"2016","journal-title":"The Journal of Machine Learning Research"},{"key":"2024071202434233600_qkad164-B3","author":"Bhattacharya","year":"2020"},{"issue":"1","key":"2024071202434233600_qkad164-B4","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1214\/18-AOS1712","article-title":"Bayesian fractional posteriors","volume":"47","author":"Bhattacharya","year":"2019","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2024071202434233600_qkad164-B5","doi-asserted-by":"crossref","first-page":"1922","DOI":"10.1214\/13-AOS1124","article-title":"Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels","volume":"41","author":"Bickel","year":"2013","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B6","volume-title":"Pattern recognition and machine learning","author":"Bishop","year":"2006"},{"issue":"518","key":"2024071202434233600_qkad164-B7","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"Journal of the American Statistical Association"},{"key":"2024071202434233600_qkad164-B8","first-page":"221","article-title":"Optimal rate of convergence for finite mixture models","author":"Chen","year":"1995","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B9","author":"Ch\u00e9rief-Abdellatif","year":"2019"},{"key":"2024071202434233600_qkad164-B10","doi-asserted-by":"crossref","first-page":"2995","DOI":"10.1214\/18-EJS1475","article-title":"Consistency of variational Bayes inference for estimation and model selection in mixtures","volume":"12","author":"Ch\u00e9rief-Abdellatif","year":"2018","journal-title":"Electronic Journal of Statistics"},{"issue":"1","key":"2024071202434233600_qkad164-B11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"issue":"2","key":"2024071202434233600_qkad164-B12","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1111\/rssb.12187","article-title":"A Bayesian information criterion for singular models","volume":"79","author":"Drton","year":"2017","journal-title":"Journal of the Royal Statistical Society. Series B (Statistical Methodology)"},{"issue":"6","key":"2024071202434233600_qkad164-B13","doi-asserted-by":"crossref","first-page":"3161","DOI":"10.1214\/19-AOS1924","article-title":"Singularity, misspecification and the convergence rate of EM","volume":"48","author":"Dwivedi","year":"2020","journal-title":"The Annals of Statistics"},{"issue":"410","key":"2024071202434233600_qkad164-B14","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1080\/01621459.1990.10476213","article-title":"Sampling-based approaches to calculating marginal densities","volume":"85","author":"Gelfand","year":"1990","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2024071202434233600_qkad164-B15","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1214\/aos\/1016218228","article-title":"Convergence rates of posterior distributions","volume":"28","author":"Ghosal","year":"2000","journal-title":"Annals of Statistics"},{"key":"2024071202434233600_qkad164-B16","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1214\/009053606000001172","article-title":"Convergence rates of posterior distributions for noniid observations","volume":"35","author":"Ghosal","year":"2007","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B17","first-page":"369","article-title":"Theory of Gaussian variational approximation for a Poisson mixed model","author":"Hall","year":"2011","journal-title":"Statistica Sinica"},{"issue":"5","key":"2024071202434233600_qkad164-B18","doi-asserted-by":"crossref","first-page":"2502","DOI":"10.1214\/11-AOS908","article-title":"Asymptotic normality and valid inference for Gaussian variational approximation","volume":"39","author":"Hall","year":"2011","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B19","volume-title":"Monte Carlo methods","author":"Hammersley","year":"2013"},{"key":"2024071202434233600_qkad164-B20","author":"Han","year":"2019"},{"key":"2024071202434233600_qkad164-B21","author":"Hastings","year":"1970"},{"issue":"6","key":"2024071202434233600_qkad164-B22","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","article-title":"A new look at the statistical model identification","volume":"19","author":"Hirotugu","year":"1974","journal-title":"IEEE Transactions on Automatic Control"},{"issue":"4","key":"2024071202434233600_qkad164-B23","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1137\/18M122947X","article-title":"Singularity structures and impacts on parameter estimation in finite mixtures of distributions","volume":"1","author":"Ho","year":"2019","journal-title":"SIAM Journal on Mathematics of Data Science"},{"key":"2024071202434233600_qkad164-B24","author":"Jain","year":"2018"},{"issue":"2","key":"2024071202434233600_qkad164-B25","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1007665907178","article-title":"An introduction to variational methods for graphical models","volume":"37","author":"Jordan","year":"1999","journal-title":"Machine Learning"},{"issue":"none","key":"2024071202434233600_qkad164-B26","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1214\/12-EJS675","article-title":"The Bernstein-Von-Mises theorem under misspecification","volume":"6","author":"Kleijn","year":"2012","journal-title":"Electronic Journal of Statistics"},{"key":"2024071202434233600_qkad164-B27","article-title":"Fast convergence of belief propagation to global optima: Beyond correlation decay","volume":"32","author":"Koehler","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024071202434233600_qkad164-B28","article-title":"Mean field for the stochastic blockmodel: Optimization landscape and convergence issues","volume":"31","author":"Mukherjee","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024071202434233600_qkad164-B29","author":"Ohn","year":"2021"},{"issue":"1","key":"2024071202434233600_qkad164-B30","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1198\/jcgs.2011.09118","article-title":"Gaussian variational approximate inference for generalized linear mixed models","volume":"21","author":"Ormerod","year":"2012","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2024071202434233600_qkad164-B31","author":"Pati","year":"2018"},{"issue":"11","key":"2024071202434233600_qkad164-B32","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.3390\/e22111263","article-title":"Dynamics of coordinate ascent variational inference: A case study in 2D Ising models","volume":"22","author":"Plummer","year":"2020","journal-title":"Entropy"},{"key":"2024071202434233600_qkad164-B33","first-page":"2328","article-title":"Variational particle approximations","volume":"18","author":"Saeedi","year":"2017","journal-title":"The Journal of Machine Learning Research"},{"issue":"22","key":"2024071202434233600_qkad164-B34","first-page":"1","article-title":"When random initializations help: A study of variational inference for community detection","volume":"22","author":"Sarkar","year":"2021","journal-title":"Journal of Machine Learning Research"},{"issue":"2","key":"2024071202434233600_qkad164-B35","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the dimension of a model","volume":"6","author":"Schwarz","year":"1978","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2024071202434233600_qkad164-B36","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1214\/aos\/1009210686","article-title":"Rates of convergence of posterior distributions","volume":"29","author":"Shen","year":"2001","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2024071202434233600_qkad164-B37","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MSP.2004.1311138","article-title":"Model-order selection: A review of information criterion rules","volume":"21","author":"Stoica","year":"2004","journal-title":"IEEE Signal Processing Magazine"},{"key":"2024071202434233600_qkad164-B38","author":"Titsias","year":"2014"},{"issue":"3","key":"2024071202434233600_qkad164-B39","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1214\/06-BA121","article-title":"Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model","volume":"1","author":"Titterington","year":"2006","journal-title":"Bayesian Analysis"},{"key":"2024071202434233600_qkad164-B40","doi-asserted-by":"crossref","DOI":"10.1137\/1.9780898719574","volume-title":"Numerical linear algebra","author":"Trefethen","year":"1997"},{"key":"2024071202434233600_qkad164-B41","volume-title":"Asymptotic statistics","author":"Van der Vaart","year":"2000"},{"key":"2024071202434233600_qkad164-B42","author":"Wang","year":"2005"},{"key":"2024071202434233600_qkad164-B43","first-page":"13357","article-title":"Variational Bayes under model misspecification","volume":"32","author":"Wang","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"527","key":"2024071202434233600_qkad164-B44","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1080\/01621459.2018.1473776","article-title":"Frequentist consistency of variational Bayes","volume":"114","author":"Wang","year":"2019","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"2024071202434233600_qkad164-B45","first-page":"1","article-title":"Particle-based energetic variational inference","volume":"31","author":"Wang","year":"2021","journal-title":"Statistics and Computing"},{"key":"2024071202434233600_qkad164-B46","article-title":"Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory","volume":"11","author":"Watanabe","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"2024071202434233600_qkad164-B47","author":"Westling","year":"2015"},{"issue":"1","key":"2024071202434233600_qkad164-B48","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10107-015-0892-3","article-title":"Coordinate descent algorithms","volume":"151","author":"Wright","year":"2015","journal-title":"Mathematical Programming"},{"issue":"4","key":"2024071202434233600_qkad164-B49","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1093\/biomet\/92.4.937","article-title":"Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation","volume":"92","author":"Yang","year":"2005","journal-title":"Biometrika"},{"issue":"2","key":"2024071202434233600_qkad164-B50","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1214\/19-AOS1827","article-title":"\u03b1-variational inference with statistical guarantees","volume":"48","author":"Yang","year":"2020","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B51","doi-asserted-by":"crossref","first-page":"2497","DOI":"10.1214\/15-AOS1417","article-title":"On the computational complexity of high-dimensional Bayesian variable selection","volume":"44","author":"Yang","year":"2016","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B52","author":"Yin","year":"2020"},{"key":"2024071202434233600_qkad164-B53","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1214\/19-AOS1898","article-title":"Theoretical and computational guarantees of mean field variational inference for community detection","volume":"48","author":"Zhang","year":"2020","journal-title":"The Annals of Statistics"},{"key":"2024071202434233600_qkad164-B54","doi-asserted-by":"crossref","first-page":"2180","DOI":"10.1214\/19-AOS1883","article-title":"Convergence rates of variational posterior distributions","volume":"48","author":"Zhang","year":"2020","journal-title":"The Annals of Statistics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/86\/3\/742\/58504465\/qkad164.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/86\/3\/742\/58504465\/qkad164.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T02:45:12Z","timestamp":1720752312000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/86\/3\/742\/7642825"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,9]]},"references-count":54,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,4,9]]},"published-print":{"date-parts":[[2024,7,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkad164","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,7]]},"published":{"date-parts":[[2024,4,9]]}}}