{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T02:01:09Z","timestamp":1792548069653,"version":"4.5.0"},"reference-count":46,"publisher":"Institute of Mathematical Statistics","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2024,8,1]]},"abstract":"The main computational challenge in Bayesian inference is to compute integrals against a high-dimensional posterior distribution. In the past decades, variational inference (VI) has emerged as a tractable approximation to these integrals, and a viable alternative to the more established paradigm of Markov chain Monte Carlo. However, little is known about the approximation accuracy of VI. In this work, we bound the TV error and the mean and covariance approximation error of Gaussian VI in terms of dimension and sample size. Our error analysis relies on a Hermite series expansion of the log posterior whose first terms are precisely cancelled out by the first order optimality conditions associated to the Gaussian VI optimization problem.<\/jats:p>","DOI":"10.1214\/24-aos2393","type":"journal-article","created":{"date-parts":[[2024,10,4]],"date-time":"2024-10-04T01:48:23Z","timestamp":1728006503000},"page":"1384-1409","source":"Crossref","is-referenced-by-count":15,"title":["On the approximation accuracy of Gaussian variational inference"],"prefix":"10.1214","volume":"52","author":[{"given":"Anya","family":"Katsevich","sequence":"first","affiliation":[{"name":"Department of Mathematics, Massachusetts Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philippe","family":"Rigollet","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Massachusetts Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"6","doi-asserted-by":"publisher","unstructured":"Bishop, C. M.<\/small> (2006). Pattern Recognition and Machine Learning<\/i>. Information Science and Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-45528-0"},{"key":"30","doi-asserted-by":"publisher","unstructured":"Kleijn, B. J. K.<\/small> and van der Vaart, A. W.<\/small> (2012). The Bernstein\u2013Von-Mises theorem under misspecification. Electron. J. Stat.<\/i> 6<\/b> 354\u2013381.","DOI":"10.1214\/12-EJS675"},{"key":"32","doi-asserted-by":"publisher","unstructured":"Lang, S.<\/small> (1993). Real and Functional Analysis<\/i>, 3rd ed. Graduate Texts in Mathematics<\/i> 142<\/b>. Springer, New York.","DOI":"10.1007\/978-1-4612-0897-6"},{"key":"14","doi-asserted-by":"publisher","unstructured":"Dehaene, G.<\/small> and Barthelm\u00e9, S.<\/small> (2018). Expectation propagation in the large data limit. J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> 80<\/b> 199\u2013217.","DOI":"10.1111\/rssb.12241"},{"key":"44","doi-asserted-by":"publisher","unstructured":"Zhang, A. Y.<\/small> and Zhou, H. H.<\/small> (2020). Theoretical and computational guarantees of mean field variational inference for community detection. Ann. Statist.<\/i> 48<\/b> 2575\u20132598.","DOI":"10.1214\/19-AOS1898"},{"key":"11","doi-asserted-by":"publisher","unstructured":"Cowles, M. K.<\/small> and Carlin, B. P.<\/small> (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review. J. Amer. Statist. Assoc.<\/i> 91<\/b> 883\u2013904.","DOI":"10.2307\/2291683"},{"key":"46","doi-asserted-by":"publisher","unstructured":"Zhang, X.<\/small>, Ling, C.<\/small> and Qi, L.<\/small> (2012). The best rank-1 approximation of a symmetric tensor and related spherical optimization problems. SIAM J. Matrix Anal. Appl.<\/i> 33<\/b> 806\u2013821.","DOI":"10.1137\/110835335"},{"key":"40","doi-asserted-by":"crossref","unstructured":"TIERNEY, L.<\/small> and KADANE, J. B.<\/small> (1986). Accurate approximations for posterior moments and marginal densities. J. Amer. Statist. Assoc.<\/i> 81<\/b> 82\u201386.","DOI":"10.1080\/01621459.1986.10478240"},{"key":"3","doi-asserted-by":"publisher","unstructured":"ALQUIER, P.<\/small> and RIDGWAY, J.<\/small> (2020). Concentration of tempered posteriors and of their variational approximations. Ann. Statist.<\/i> 48<\/b> 1475\u20131497.","DOI":"10.1214\/19-AOS1855"},{"key":"45","doi-asserted-by":"publisher","unstructured":"ZHANG, F.<\/small> and GAO, C.<\/small> (2020). Convergence rates of variational posterior distributions. Ann. Statist.<\/i> 48<\/b> 2180\u20132207.","DOI":"10.1214\/19-AOS1883"},{"key":"22","doi-asserted-by":"publisher","unstructured":"HALL, P.<\/small>, PHAM, T.<\/small>, WAND, M. P.<\/small> and WANG, S. S. J.<\/small> (2011). Asymptotic normality and valid inference for Gaussian variational approximation. Ann. Statist.<\/i> 39<\/b> 2502\u20132532.","DOI":"10.1214\/11-AOS908"},{"key":"41","doi-asserted-by":"publisher","unstructured":"van der Vaart, A. W.<\/small> (1998). Asymptotic Statistics<\/i>. Cambridge Series in Statistical and Probabilistic Mathematics<\/i> 3<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511802256"},{"key":"5","doi-asserted-by":"publisher","unstructured":"Ben Arous, G.<\/small>, Dembo, A.<\/small> and Guionnet, A.<\/small> (2001). Aging of spherical spin glasses. Probab. Theory Related Fields<\/i> 120<\/b> 1\u201367.","DOI":"10.1007\/PL00008774"},{"key":"1","doi-asserted-by":"publisher","unstructured":"Adamczak, R.<\/small>, Litvak, A. E.<\/small>, Pajor, A.<\/small> and Tomczak-Jaegermann, N.<\/small> (2010). Quantitative estimates of the convergence of the empirical covariance matrix in log-concave ensembles. J. Amer. Math. Soc.<\/i> 23<\/b> 535\u2013561.","DOI":"10.1090\/S0894-0347-09-00650-X"},{"key":"7","doi-asserted-by":"publisher","unstructured":"Blei, D. M.<\/small>, Kucukelbir, A.<\/small> and McAuliffe, J. D.<\/small> (2017). Variational inference: A review for statisticians. J. Amer. Statist. Assoc.<\/i> 112<\/b> 859\u2013877.","DOI":"10.1080\/01621459.2017.1285773"},{"key":"42","doi-asserted-by":"publisher","unstructured":"Wang, Y.<\/small> and Blei, D. M.<\/small> (2019). Frequentist consistency of variational Bayes. J. Amer. Statist. Assoc.<\/i> 114<\/b> 1147\u20131161.","DOI":"10.1080\/01621459.2018.1473776"},{"key":"10","doi-asserted-by":"publisher","unstructured":"CH\u00c9RIEF-ABDELLATIF, B.-E.<\/small> and ALQUIER, P.<\/small> (2018). Consistency of variational Bayes inference for estimation and model selection in mixtures. Electron. J. Stat.<\/i> 12<\/b> 2995\u20133035.","DOI":"10.1214\/18-EJS1475"},{"key":"2","doi-asserted-by":"publisher","unstructured":"ALBANESE, L.<\/small> and ALESSANDRELLI, A.<\/small> (2022). On Gaussian spin glass with P-wise interactions. J. Math. Phys.<\/i> 63<\/b> Paper No. 043302, 28.","DOI":"10.1063\/5.0079776"},{"key":"4","doi-asserted-by":"publisher","unstructured":"BARRA, A.<\/small>, GENOVESE, G.<\/small>, GUERRA, F.<\/small> and TANTARI, D.<\/small> (2014). About a solvable mean field model of a Gaussian spin glass. J. Phys. A<\/i> 47<\/b> 155002, 16.","DOI":"10.1088\/1751-8113\/47\/15\/155002"},{"key":"8","unstructured":"BURT, D. R.<\/small>, RASMUSSEN, C. E.<\/small> and VAN DER WILK, M.<\/small> (2020). Convergence of sparse variational inference in Gaussian processes regression. J. Mach. Learn. Res.<\/i> 21<\/b> Paper No. 131, 63."},{"key":"9","unstructured":"CHALLIS, E.<\/small> and BARBER, D.<\/small> (2013). Gaussian Kullback\u2013Leibler approximate inference. J. Mach. Learn. Res.<\/i> 14<\/b> 2239\u20132286."},{"key":"12","unstructured":"DAUDEL, K.<\/small> and DOUC, R.<\/small> (2021). Mixture weights optimisation for alpha-divergence variational inference. Adv. Neural Inf. Process. Syst.<\/i> 34<\/b> 4397\u20134408."},{"key":"13","doi-asserted-by":"publisher","unstructured":"DAUDEL, K.<\/small>, DOUC, R.<\/small> and PORTIER, F.<\/small> (2021). Infinite-dimensional gradient-based descent for alpha-divergence minimisation. Ann. Statist.<\/i> 49<\/b> 2250\u20132270.","DOI":"10.1214\/20-aos2035"},{"key":"15","unstructured":"DEHAENE, G. P.<\/small> (2019). A deterministic and computable Bernstein\u2013von Mises theorem. arXiv preprint. Available at arXiv:1904.02505<\/a>."},{"key":"16","unstructured":"DEHAENE, G. P.<\/small> and BARTHELM\u00c9, S.<\/small> (2015). Bounding errors of expectation-propagation. Adv. Neural Inf. Process. Syst.<\/i> 28<\/b>."},{"key":"17","unstructured":"DIAO, M.<\/small>, BALASUBRAMANIAN, K.<\/small>, CHEWI, S.<\/small> and SALIM, A.<\/small> (2023). Forward-backward Gaussian variational inference via JKO in the Bures\u2013Wasserstein space. arXiv preprint. Available at arXiv:2304.05398<\/a>."},{"key":"18","unstructured":"DURANTE, D.<\/small>, POZZA, F.<\/small> and SZABO, B.<\/small> (2023). Skewed Bernstein\u2013von Mises theorem and skew-modal approximations. arXiv preprint. Available at arXiv:2301.03038<\/a>."},{"key":"19","doi-asserted-by":"publisher","unstructured":"FASANO, A.<\/small>, DURANTE, D.<\/small> and ZANELLA, G.<\/small> (2022). Scalable and accurate variational Bayes for high-dimensional binary regression models. Biometrika<\/i> 109<\/b> 901\u2013919.","DOI":"10.1093\/biomet\/asac026"},{"key":"20","doi-asserted-by":"publisher","unstructured":"GHOSAL, S.<\/small> (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal.<\/i> 74<\/b> 49\u201368.","DOI":"10.1006\/jmva.1999.1874"},{"key":"21","unstructured":"GHOSH, J. K.<\/small>and RAMAMOORTHI, R. V.<\/small> (2003). Bayesian Nonparametrics<\/i>. Springer Series in Statistics<\/i>. Springer, New York."},{"key":"23","unstructured":"HAN, W.<\/small> and YANG, Y.<\/small> (2019). Statistical inference in mean-field variational Bayes. arXiv preprint. Available at arXiv:1911.01525<\/a>."},{"key":"25","doi-asserted-by":"publisher","unstructured":"JOHNSON, R. A.<\/small> (1970). Asymptotic expansions associated with posterior distributions. Ann. Math. Stat.<\/i> 41<\/b> 851\u2013864.","DOI":"10.1214\/aoms\/1177696963"},{"key":"26","unstructured":"KASPRZAK, M. J.<\/small>, GIORDANO, R.<\/small> and BRODERICK, T.<\/small> (2022). How good is your Gaussian approximation of the posterior? Finite-sample computable error bounds for a variety of useful divergences. arXiv preprint. Available at arXiv:2209.14992<\/a>."},{"key":"27","unstructured":"KATSEVICH, A.<\/small> (2023). The Laplace approximation accuracy in high dimensions: A refined analysis and new skew adjustment. arXiv preprint. Available at arXiv:2306.07262<\/a>."},{"key":"28","unstructured":"KATSEVICH, A.<\/small> (2023). Improved dimension dependence in the Bernstein von Mises Theorem via a new Laplace approximation bound. arXiv preprint. Available at arXiv:2308.06899<\/a>."},{"key":"29","doi-asserted-by":"crossref","unstructured":"KATSEVICH, A.<\/small> and RIGOLLET, P.<\/small> (2024). Supplement to \u201cOn the approximation accuracy of Gaussian variational inference.\u201d https:\/\/doi.org\/10.1214\/24-AOS2393SUPP<\/a>","DOI":"10.1214\/24-AOS2393SUPP"},{"key":"31","unstructured":"LAMBERT, M.<\/small>, CHEWI, S.<\/small>, BACH, F.<\/small>, BONNABEL, S.<\/small> and RIGOLLET, P.<\/small> (2022). Variational inference via Wasserstein gradient flows. arXiv preprint. Available at arXiv:2205.15902<\/a>."},{"key":"33","unstructured":"LEBEDEV, N. N.<\/small> (1972). Special Functions and Their Applications<\/i>. Dover, New York. Revised edition, translated from the Russian and edited by Richard A. Silverman, Unabridged and corrected republication."},{"key":"34","unstructured":"LU, Y.<\/small> (2017). On the Bernstein\u2013von Mises theorem for high dimensional nonlinear Bayesian inverse problems. arXiv preprint. Available at arXiv:1706.00289<\/a>."},{"key":"35","unstructured":"MILLER, J. W.<\/small> (2021). Asymptotic normality, concentration, and coverage of generalized posteriors. J. Mach. Learn. Res.<\/i> 22<\/b> Paper No. 168, 53."},{"key":"36","unstructured":"MUKHERJEE, S.<\/small> and SEN, S.<\/small> (2022). Variational inference in high-dimensionalAccurate approximations for posterior linear regression. J. Mach. Learn. Res.<\/i> 23<\/b> Paper No. [304], 56."},{"key":"37","unstructured":"SPOKOINY, V.<\/small> (2013). Bernstein\u2013von Mises Theorem for growing parameter dimension. arXiv preprint. Available at arXiv:1302.3430<\/a>."},{"key":"38","doi-asserted-by":"publisher","unstructured":"SPOKOINY, V.<\/small> (2023). Dimension free nonasymptotic bounds on the accuracy of high-dimensional Laplace approximation. SIAM\/ASA J. Uncertain. Quantificat.<\/i> 11<\/b> 1044\u20131068.","DOI":"10.1137\/22M1495688"},{"key":"39","unstructured":"SUR, P.<\/small> (2019). A modern maximum-likelihood theory for high-dimensional logistic regression. Ph.D. thesis, Stanford Univ."},{"key":"43","doi-asserted-by":"publisher","unstructured":"WENG, R. C.<\/small> (2010). A Bayesian Edgeworth expansion by Stein\u2019s identity. Bayesian Anal.<\/i> 5<\/b> 741\u2013763.","DOI":"10.1214\/10-BA526"},{"key":"24","doi-asserted-by":"publisher","unstructured":"HELIN, T.<\/small> and KRETSCHMANN, R.<\/small> (2022). Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems. Numer. Math.<\/i> 150<\/b> 521\u2013549.","DOI":"10.1007\/s00211-021-01266-9"}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T17:34:56Z","timestamp":1791394496000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-52\/issue-4\/On-the-approximation-accuracy-of-Gaussian-variational-inference\/10.1214\/24-AOS2393.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,1]]},"references-count":46,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,8,1]]}},"URL":"https:\/\/doi.org\/10.1214\/24-aos2393","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,1]]}}}