{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T15:24:14Z","timestamp":1787498654879,"version":"build-2736575974"},"reference-count":52,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,11,29]],"date-time":"2024-11-29T00:00:00Z","timestamp":1732838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,16]]},"abstract":"<jats:title>Summary<\/jats:title>\n                  <jats:p>While generalized linear mixed models are a fundamental tool in applied statistics, many specifications, such as those involving categorical factors with many levels or interaction terms, can be computationally challenging to estimate due to the need to compute or approximate high-dimensional integrals. Variational inference is a popular way to perform such computations, especially in the Bayesian context. However, naive use of such methods can provide unreliable uncertainty quantification. We show that this is indeed the case for mixed models, proving that standard mean-field variational inference dramatically underestimates posterior uncertainty in high dimensions. We then show how appropriately relaxing the mean-field assumption leads to methods whose uncertainty quantification does not deteriorate in high dimensions, and whose total computational cost scales linearly with the number of parameters and observations. Our theoretical and numerical results focus on mixed models with Gaussian or binomial likelihoods, and rely on connections to random graph theory to obtain sharp high-dimensional asymptotic analysis. We also provide generic results, which are of independent interest, relating the accuracy of variational inference to the convergence rate of the corresponding coordinate ascent algorithm that is used to find it. Our proposed methodology is implemented in the R package vglmer. Numerical results with simulated and real data examples illustrate the favourable computation cost versus accuracy trade-off of our approach compared to various alternatives.<\/jats:p>","DOI":"10.1093\/biomet\/asae067","type":"journal-article","created":{"date-parts":[[2024,11,27]],"date-time":"2024-11-27T23:26:10Z","timestamp":1732749970000},"source":"Crossref","is-referenced-by-count":7,"title":["Partially factorized variational inference for high-dimensional mixed models"],"prefix":"10.1093","volume":"112","author":[{"given":"M","family":"Goplerud","sequence":"first","affiliation":[{"name":"Department of Government, University of Texas at Austin , 158 West 21st Street, Stop A1800 , Austin, Texas 78712,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4452-6154","authenticated-orcid":false,"given":"O","family":"Papaspiliopoulos","sequence":"additional","affiliation":[{"name":"Department of Decision Sciences, Bocconi University , via Roentgen 1 , 20136 Milan,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G","family":"Zanella","sequence":"additional","affiliation":[{"name":"Department of Decision Sciences, Bocconi University , via Roentgen 1 , 20136 Milan,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,11,29]]},"reference":[{"key":"2025062420154700200_asae067-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting linear mixed-effects models using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Statist. Software"},{"key":"2025062420154700200_asae067-B2","author":"Bhattacharya","year":"2023"},{"key":"2025062420154700200_asae067-B3","volume-title":"Pattern Recognition and Machine Learning","author":"Bishop","year":"2006"},{"key":"2025062420154700200_asae067-B4","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: a review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Statist. Assoc"},{"key":"2025062420154700200_asae067-B5","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1017\/S0963548321000249","article-title":"Spectral gap in random bipartite biregular graphs and applications","volume":"31","author":"Brito","year":"2022","journal-title":"Combinat. Prob. Comp"},{"key":"2025062420154700200_asae067-B6","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1017\/S0003055418000011","article-title":"Bias in perceptions of public opinion among political elites","volume":"112","author":"Broockman","year":"2018","journal-title":"Am. Polit. Sci. Rev"},{"key":"2025062420154700200_asae067-B7","author":"Castiglione","year":"2022"},{"key":"2025062420154700200_asae067-B8","volume-title":"Analysis of Longitudinal Data","author":"Diggle","year":"2001"},{"key":"2025062420154700200_asae067-B9","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1214\/19-STS712","article-title":"Conditionally conjugate mean-field variational Bayes for logistic models","volume":"34","author":"Durante","year":"2019","journal-title":"Statist. Sci."},{"key":"2025062420154700200_asae067-B10","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1198\/jasa.2011.tm10301","article-title":"Variational Bayesian inference for parametric and nonparametric regression with missing data","volume":"106","author":"Faes","year":"2011","journal-title":"J. Am. Statist. Assoc"},{"key":"2025062420154700200_asae067-B11","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1093\/biomet\/asac026","article-title":"Scalable and accurate variational Bayes for high-dimensional binary regression models","volume":"109","author":"Fasano","year":"2022","journal-title":"Biometrika"},{"key":"2025062420154700200_asae067-B12","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1080\/10618600.2022.2134875","article-title":"Variational Bayes in state space models: inferential and predictive accuracy","volume":"32","author":"Frazier","year":"2023","journal-title":"J. Comp. Graph. Statist"},{"key":"2025062420154700200_asae067-B13","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1214\/17-EJS1236","article-title":"Efficient moment calculations for variance components in large unbalanced crossed random effects models","volume":"11","author":"Gao","year":"2017","journal-title":"Electron. J. Statist."},{"key":"2025062420154700200_asae067-B14","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1111\/ajps.12004","article-title":"Deep interactions with MRP: election turnout and voting patterns among small electoral subgroups","volume":"57","author":"Ghitza","year":"2013","journal-title":"Am. Polit. Sci. Rev"},{"key":"2025062420154700200_asae067-B15","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1214\/21-BA1266","article-title":"Fast and accurate estimation of non-nested binomial hierarchical models using variational inference","volume":"17","author":"Goplerud","year":"2022","journal-title":"Bayesian Anal."},{"key":"2025062420154700200_asae067-B16","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1017\/S0003055423000035","article-title":"Re-evaluating machine learning for MRP given the comparable performance of (deep) hierarchical models","volume":"118","author":"Goplerud","year":"2024","journal-title":"Am. Polit. Sci. Rev"},{"key":"2025062420154700200_asae067-B17","first-page":"361","volume-title":"Proc. 18th Int. Conf. Artif. Intel. Statist","author":"Hoffman","year":"2015"},{"key":"2025062420154700200_asae067-B18","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/978-1-4613-0147-9_5","volume-title":"Spatial Statistics: Methodological Aspects and Applications (Lecture Notes Statist. 159), Ed","author":"Huang","year":"2001"},{"key":"2025062420154700200_asae067-B19","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1023\/A:1008932416310","article-title":"Bayesian parameter estimation via variational methods","volume":"10","author":"Jaakkola","year":"2000","journal-title":"Statist. Comp"},{"key":"2025062420154700200_asae067-B20","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1007\/s11336-017-9555-z","article-title":"A variational maximization\u2013maximization algorithm for generalized linear mixed models with crossed random effects","volume":"82","author":"Jeon","year":"2017","journal-title":"Psychometrika"},{"key":"2025062420154700200_asae067-B21","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1086\/681261","article-title":"Polarizing the electoral connection: partisan representation in supreme court confirmation politics","volume":"77","author":"Kastellec","year":"2015","journal-title":"J. Polit"},{"key":"2025062420154700200_asae067-B22","first-page":"1701","volume-title":"Proc. 24th Int. Conf. Neural Info. Proces","author":"Knowles","year":"2011"},{"key":"2025062420154700200_asae067-B23","first-page":"1","article-title":"Automatic differentiation variational inference","volume":"18","author":"Kucukelbir","year":"2017","journal-title":"J. Mach. Learn. Res"},{"key":"2025062420154700200_asae067-B24","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1111\/j.1540-5907.2008.00360.x","article-title":"How should we estimate public opinion in the States?","volume":"53","author":"Lax","year":"2009","journal-title":"Am. Polit. Sci. Rev"},{"key":"2025062420154700200_asae067-B25","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1017\/S0003055419000315","article-title":"The party or the purse? Unequal representation in the US Senate","volume":"113","author":"Lax","year":"2019","journal-title":"Am. Polit. Sci. Rev"},{"key":"2025062420154700200_asae067-B26","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.jeconom.2021.05.002","article-title":"Fast and accurate variational inference for models with many latent variables","volume":"230","author":"Loaiza-Maya","year":"2022","journal-title":"J. Economet"},{"key":"2025062420154700200_asae067-B27","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1080\/10618600.2022.2096622","article-title":"Streamlined variational inference for linear mixed models with crossed random effects","volume":"32","author":"Menictas","year":"2023","journal-title":"J. Comp. Graph. Statist"},{"key":"2025062420154700200_asae067-B28","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1177\/1471082X20930894","article-title":"Streamlined variational inference for higher level group-specific curve models","volume":"21","author":"Menictas","year":"2021","journal-title":"Statist. Mod."},{"key":"2025062420154700200_asae067-B29","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1198\/jcgs.2011.09118","article-title":"Gaussian variational approximate inference for generalized linear mixed models","volume":"21","author":"Ormerod","year":"2012","journal-title":"J. Comp. Graph. Statist"},{"key":"2025062420154700200_asae067-B30","author":"Pandolfi","year":"2024"},{"key":"2025062420154700200_asae067-B31","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1214\/088342307000000014","article-title":"A general framework for the parametrization of hierarchical models","volume":"22","author":"Papaspiliopoulos","year":"2007","journal-title":"Statist. Sci."},{"key":"2025062420154700200_asae067-B32","first-page":"25","article-title":"Scalable inference for crossed random effects models","volume":"107","author":"Papaspiliopoulos","year":"2020","journal-title":"Biometrika"},{"key":"2025062420154700200_asae067-B33","doi-asserted-by":"crossref","first-page":"3575","DOI":"10.1214\/23-EJS2172","article-title":"Scalable Bayesian computation for crossed and nested hierarchical models","volume":"17","author":"Papaspiliopoulos","year":"2023","journal-title":"Electron. J. Statist."},{"key":"2025062420154700200_asae067-B34","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1093\/pan\/mph024","article-title":"Bayesian multilevel estimation with poststratification: state-level estimates from national polls","volume":"12","author":"Park","year":"2004","journal-title":"Polit. Anal."},{"key":"2025062420154700200_asae067-B35","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/rssb.12165","article-title":"Fast moment-based estimation for hierarchical models","volume":"79","author":"Perry","year":"2016","journal-title":"J. R. Statist. Soc. B"},{"key":"2025062420154700200_asae067-B36","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1080\/01621459.2013.829001","article-title":"Bayesian inference for logistic models using P\u00f3lya\u2013gamma latent variables","volume":"108","author":"Polson","year":"2013","journal-title":"J. Am. Statist. Assoc"},{"key":"2025062420154700200_asae067-B37","author":"R Development Core Team","year":"2025"},{"key":"2025062420154700200_asae067-B38","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1214\/13-BA858","article-title":"Fixed-form variational posterior approximation through stochastic linear regression","volume":"8","author":"Salimans","year":"2013","journal-title":"Bayesian Anal."},{"key":"2025062420154700200_asae067-B39","first-page":"486","volume-title":"Proc. 8th Int. Conf. Neural Info. Proces. Syst","author":"Saul","year":"1995"},{"key":"2025062420154700200_asae067-B40","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1111\/rssb.12399","article-title":"Use of model reparametrization to improve variational Bayes","volume":"83","author":"Tan","year":"2021","journal-title":"J. R. Statist. Soc. B"},{"key":"2025062420154700200_asae067-B41","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1214\/13-STS418","article-title":"Variational inference for generalized linear mixed models using partially noncentered parametrizations","volume":"28","author":"Tan","year":"2013","journal-title":"Statist. Sci."},{"key":"2025062420154700200_asae067-B42","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s11222-017-9729-7","article-title":"Gaussian variational approximation with sparse precision matrices","volume":"28","author":"Tan","year":"2018","journal-title":"Statist. Comp"},{"key":"2025062420154700200_asae067-B43","first-page":"1353","volume-title":"Proc. 19th Int. Conf. Neural Info. Proces. Syst., Ed","author":"Teh","year":"2006"},{"key":"2025062420154700200_asae067-B44","author":"Titsias","year":"2014"},{"key":"2025062420154700200_asae067-B45","doi-asserted-by":"crossref","DOI":"10.1017\/9781108627771","volume-title":"High-Dimensional Statistics: A Non-Asymptotic Viewpoint","author":"Wainwright","year":"2019"},{"key":"2025062420154700200_asae067-B46","first-page":"1351","article-title":"Fully simplified multivariate normal updates in non-conjugate variational message passing","volume":"15","author":"Wand","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"2025062420154700200_asae067-B47","doi-asserted-by":"crossref","DOI":"10.1201\/b14876","volume-title":"Kernel Smoothing","author":"Wand","year":"1994"},{"key":"2025062420154700200_asae067-B48","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1017\/S0022381611001204","article-title":"How should we measure district-level public opinion on individual issues?","volume":"74","author":"Warshaw","year":"2012","journal-title":"J. Polit"},{"key":"2025062420154700200_asae067-B49","doi-asserted-by":"crossref","DOI":"10.1201\/9781315370279","volume-title":"Generalized Additive Models","author":"Wood","year":"2017","edition":"2nd ed"},{"key":"2025062420154700200_asae067-B50","author":"Xu","year":"2023"},{"key":"2025062420154700200_asae067-B51","first-page":"1913","volume-title":"Proc. 21st Int. Conf. Neural Info. Proces. Syst","author":"Zhang","year":"2008"},{"key":"2025062420154700200_asae067-B52","first-page":"1","article-title":"Pathfinder: Parallel quasi-Newton variational inference","volume":"23","author":"Zhang","year":"2022","journal-title":"J. Mach. Learn. Res"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asae067\/60923732\/asae067.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/112\/2\/asae067\/60923732\/asae067.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/112\/2\/asae067\/60923732\/asae067.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T20:16:00Z","timestamp":1750796160000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/doi\/10.1093\/biomet\/asae067\/7914011"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,29]]},"references-count":52,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,3,16]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asae067","relation":{},"ISSN":["1464-3510"],"issn-type":[{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025]]},"published":{"date-parts":[[2024,11,29]]},"article-number":"asae067"}}