{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T23:01:45Z","timestamp":1772751705460,"version":"3.50.1"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2020,11,18]],"date-time":"2020-11-18T00:00:00Z","timestamp":1605657600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We propose using model reparametrization to improve variational Bayes inference for hierarchical models whose variables can be classified as global (shared across observations) or local (observation-specific). Posterior dependence between local and global variables is minimized by applying an invertible affine transformation on the local variables. The functional form of this transformation is deduced by approximating the posterior distribution of each local variable conditional on the global variables by a Gaussian density via a second order Taylor expansion. Variational Bayes inference for the reparametrized model is then obtained using stochastic approximation. Our approach can be readily extended to large datasets via a divide and recombine strategy. Using generalized linear mixed models, we demonstrate that reparametrized variational Bayes (RVB) provides improvements in both accuracy and convergence rate compared to state of the art Gaussian variational approximation methods.<\/jats:p>","DOI":"10.1111\/rssb.12399","type":"journal-article","created":{"date-parts":[[2020,11,19]],"date-time":"2020-11-19T07:29:19Z","timestamp":1605770959000},"page":"30-57","source":"Crossref","is-referenced-by-count":16,"title":["Use of Model Reparametrization to Improve Variational Bayes"],"prefix":"10.1093","volume":"83","author":[{"given":"Linda S. L.","family":"Tan","sequence":"first","affiliation":[{"name":"National University of Singapore \u00a0 Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,11,18]]},"reference":[{"key":"2023022408370283200_","first-page":"21","volume-title":"Proceedings of the 15th conference on uncertainty in artificial intelligence","author":"Attias","year":"1999"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1080\/01621459.1993.10594284","article-title":"Approximate inference in generalized linear mixed models","volume":"88","author":"Breslow","year":"1993","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408370283200_","first-page":"1727","volume-title":"Streaming variational bayes","author":"Broderick","year":"2013"},{"key":"2023022408370283200_","first-page":"1","article-title":"Stan: A probabilistic programming language","volume":"76","author":"Carpenter","year":"2017","journal-title":"Journal of Statistical Software, Articles"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1198\/106186006X100470","article-title":"Robust Markov chain Monte Carlo methods for spatial generalized linear mixed models","volume":"15","author":"Christensen","year":"2006","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1890\/07-0744.1","article-title":"Accounting for uncertainty in ecological analysis: the strengths and limitations of hierarchical statistical modeling","volume":"19","author":"Cressie","year":"2009","journal-title":"Ecological Applications"},{"key":"2023022408370283200_","first-page":"34","article-title":"Beta-binomial anova for proportions","volume":"27","author":"Crowder","year":"1978","journal-title":"Journal of the Royal Statistical Society. Series C (Applied Statistics)"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0190-9622(98)70486-4","article-title":"Twelve weeks of continuous oral therapy for toenail onychomycosis caused by dermatophytes: A double-blind comparative trial of terbinafine 250 mg\/day versus itraconazole 200 mg\/day","volume":"38","author":"De Backer","year":"1998","journal-title":"Journal of the American Academy of Dermatology"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/biomet\/80.1.27","article-title":"Bias reduction of maximum likelihood estimates","volume":"80","author":"Firth","year":"1993","journal-title":"Biometrika"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1093\/biomet\/82.3.479","article-title":"Efficient parametrisations for normal linear mixed models","volume":"82","author":"Gelfand","year":"1995","journal-title":"Biometrika"},{"key":"2023022408370283200_","volume-title":"Efficient parametrisations for generalized linear mixed models","author":"Gelfand","year":"1996"},{"key":"2023022408370283200_","volume-title":"Data analysis using regression and multilevel\/hierarchical models. Analytical methods for social research","author":"Gelman","year":"2006"},{"key":"2023022408370283200_","first-page":"663","volume-title":"Nonparametric variational inference","author":"Gershman","year":"2012"},{"key":"2023022408370283200_","first-page":"361","volume-title":"Stochastic structured variational inference","author":"Hoffman","year":"2015"},{"key":"2023022408370283200_","first-page":"1303","article-title":"Stochastic variational inference","volume":"14","author":"Hoffman","year":"2013","journal-title":"Journal of Machine Learning Research"},{"issue":"7","key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1001\/jama.280.7.605","article-title":"Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women","volume":"280","author":"Hulley","year":"1998","journal-title":"JAMA"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","article-title":"Adaptive mixtures of local experts","volume":"3","author":"Jacobs","year":"1991","journal-title":"Neural Computing"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1214\/06-BA117B","article-title":"A default conjugate prior for variance components in generalized linear mixed models (comment on article by browne and draper","volume":"1","author":"Kass","year":"2006","journal-title":"Bayesian Analysis"},{"key":"2023022408370283200_","volume-title":"Auto-encoding variational Bayes","author":"Kingma","year":"2014"},{"key":"2023022408370283200_","volume-title":"Automatic differentiation variational inference","author":"Kucukelbir","year":"2016"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/sim.6737","article-title":"Variational methods for fitting complex Bayesian mixed effects models to health data","volume":"35","author":"Lee","year":"2016","journal-title":"Statistics in Medicine"},{"key":"2023022408370283200_","volume-title":"Methods and applications of longitudinal data analysis","author":"Liu","year":"2016"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"3049","DOI":"10.1002\/sim.3680","article-title":"The bugs project: Evolution, critique and future directions","volume":"28","author":"Lunn","year":"2009","journal-title":"Statistics in Medicine"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1214\/11-AOAS522","article-title":"Bayesian hierarchical rule modeling for predicting medical conditions","volume":"6","author":"McCormick","year":"2012","journal-title":"The Annals of Applied Statistics"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1198\/tast.2010.09058","article-title":"Explaining variational approximations","volume":"64","author":"Ormerod","year":"2010","journal-title":"The American Statistician"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1198\/jcgs.2011.09118","article-title":"Gaussian variational approximate inference for generalized linear mixed models","volume":"21","author":"Ormerod","year":"2012","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022408370283200_","volume-title":"Variational Bayesian inference with stochastic search","author":"Paisley","year":"2012"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1093\/oso\/9780198526155.003.0017","volume-title":"Bayesian statistics 7","author":"Papaspiliopoulos","year":"2003"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1214\/088342307000000014","article-title":"A general framework for the parametrization of hierarchical models","volume":"22","author":"Papaspiliopoulos","year":"2007","journal-title":"Statistical Science"},{"key":"2023022408370283200_","first-page":"1530","volume-title":"Variational inference with normalizing flows","author":"Rezende","year":"2015"},{"key":"2023022408370283200_","first-page":"1278","volume-title":"Stochastic backpropagation and approximate inference in deep generative models","author":"Rezende","year":"2014"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A stochastic approximation method","volume":"22","author":"Robbins","year":"1951","journal-title":"The Annals of Mathematical Statistics"},{"key":"2023022408370283200_","volume-title":"Sticking the landing: Simple, lower-variance gradient estimators for variational inference","author":"Roeder","year":"2017"},{"key":"2023022408370283200_","first-page":"1","article-title":"Semiparametric mean field variational bayes: General principles and numerical issues","volume":"17","author":"Rohde","year":"2016","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1111\/j.1467-9868.2008.00700.x","article-title":"Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations","volume":"71","author":"Rue","year":"2009","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1214\/13-BA858","article-title":"Fixed-form variational posterior approximation through stochastic linear regression","volume":"8","author":"Salimans","year":"2013","journal-title":"Bayesian Analysis"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","DOI":"10.1002\/0471722138","volume-title":"Introduction to stochastic search and optimization: Estimation, simulation and control","author":"Spall","year":"2003"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1214\/13-STS418","article-title":"Variational inference for generalized linear mixed models using partially non-centered parametrizations","volume":"28","author":"Tan","year":"2013","journal-title":"Statistical Science"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s11222-017-9729-7","article-title":"Gaussian variational approximation with sparse precision matrices","volume":"28","author":"Tan","year":"2018","journal-title":"Statistics and Computing"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"657","DOI":"10.2307\/2532086","article-title":"Some covariance models for longitudinal count data with overdispersion","volume":"46","author":"Thall","year":"1990","journal-title":"Biometrics"},{"key":"2023022408370283200_","volume-title":"Learning model reparametrizations: Implicit variational inference by fitting mcmc distributions","author":"Titsias","year":"2017"},{"key":"2023022408370283200_","first-page":"1971","volume-title":"Doubly stochastic variational Bayes for non-conjugate inference","author":"Titsias","year":"2014"},{"issue":"2","key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1080\/10618600.2015.1012293","article-title":"Parallel variational bayes for large datasets with an application to generalized linear mixed models","volume":"25","author":"Tran","year":"2016","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022408370283200_","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1198\/jcgs.2011.203main","article-title":"To center or not to center: That is not the question\u2014An ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC efficiency","volume":"20","author":"Yu","year":"2011","journal-title":"Journal of Computational and Graphical Statistics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12399","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12399","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/83\/1\/30\/49324099\/jrsssb_83_1_30.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/83\/1\/30\/49324099\/jrsssb_83_1_30.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,17]],"date-time":"2024-08-17T17:23:28Z","timestamp":1723915408000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/83\/1\/30\/7050743"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,18]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,11,18]]},"published-print":{"date-parts":[[2021,2,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12399","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,2]]},"published":{"date-parts":[[2020,11,18]]}}}