{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T23:02:42Z","timestamp":1787612562776,"version":"build-2736575974"},"reference-count":42,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T00:00:00Z","timestamp":1651708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,11,29]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>Modern methods for Bayesian regression beyond the Gaussian response setting are often computationally impractical or inaccurate in high dimensions. In fact, as discussed in recent literature, bypassing such a trade-off is still an open problem even in routine binary regression models, and there is limited theory on the quality of variational approximations in high-dimensional settings. To address this gap, we study the approximation accuracy of routinely used mean-field variational Bayes solutions in high-dimensional probit regression with Gaussian priors, obtaining novel and practically relevant results on the pathological behaviour of such strategies in uncertainty quantification, point estimation and prediction. Motivated by these results, we further develop a new partially factorized variational approximation for the posterior distribution of the probit coefficients that leverages a representation with global and local variables but, unlike for classical mean-field assumptions, it avoids a fully factorized approximation, and instead assumes a factorization only for the local variables. We prove that the resulting approximation belongs to a tractable class of unified skew-normal densities that crucially incorporates skewness and, unlike for state-of-the-art mean-field solutions, converges to the exact posterior density as $p \\rightarrow \\infty$. To solve the variational optimization problem, we derive a tractable coordinate ascent variational inference algorithm that easily scales to $p$ in the tens of thousands, and provably requires a number of iterations converging to $1$ as $p \\rightarrow \\infty$. Such findings are also illustrated in extensive empirical studies where our novel solution is shown to improve the approximation accuracy of mean-field variational Bayes for any $n$ and $p$, with the magnitude of these gains being remarkable in those high-dimensional $p&amp;gt;n$ settings where state-of-the-art methods are computationally impractical.<\/jats:p>","DOI":"10.1093\/biomet\/asac026","type":"journal-article","created":{"date-parts":[[2022,5,2]],"date-time":"2022-05-02T19:11:45Z","timestamp":1651518705000},"page":"901-919","source":"Crossref","is-referenced-by-count":16,"title":["Scalable and accurate variational Bayes for high-dimensional binary regression models"],"prefix":"10.1093","volume":"109","author":[{"given":"Augusto","family":"Fasano","sequence":"first","affiliation":[{"name":"Bocconi University Department of Decision Sciences, , Via R\u00f6ntgen 1, 20136 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniele","family":"Durante","sequence":"additional","affiliation":[{"name":"Bocconi University Department of Decision Sciences, , Via R\u00f6ntgen 1, 20136 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giacomo","family":"Zanella","sequence":"additional","affiliation":[{"name":"Bocconi University Department of Decision Sciences, , Via R\u00f6ntgen 1, 20136 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,5,5]]},"reference":[{"key":"2022112918000261700_B1","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1080\/01621459.1993.10476321","article-title":"Bayesian analysis of binary and polychotomous response data","volume":"88","author":"Albert,","year":"1993","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B2","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1111\/j.1467-9469.2006.00503.x","article-title":"On the unification of families of skew-normal distributions","volume":"33","author":"Arellano-Valle,","year":"2006","journal-title":"Scand. J. Statist."},{"key":"2022112918000261700_B3","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1016\/j.csda.2010.06.005","article-title":"A note on mean-field variational approximations in Bayesian probit models","volume":"55","author":"Armagan,","year":"2011","journal-title":"Comp. Statist. Data Anal."},{"key":"2022112918000261700_B4","volume-title":"The Skew-Normal and Related Families.","author":"Azzalini,","year":"2014"},{"key":"2022112918000261700_B5","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: a review for statisticians","volume":"112","author":"Blei,","year":"2017","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B6","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1111\/rssb.12162","article-title":"The normal law under linear restrictions: simulation and estimation via minimax tilting","volume":"79","author":"Botev,","year":"2017","journal-title":"J. R. Statist. Soc. B"},{"key":"2022112918000261700_B7","doi-asserted-by":"crossref","DOI":"10.1080\/10618600.2022.2036614","article-title":"Scalable computation of predictive probabilities in probit models with Gaussian process priors","author":"Cao,","year":"2022","journal-title":"J. Comp. Graph. Statist."},{"key":"2022112918000261700_B8","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1007\/s11222-018-9825-3","article-title":"Hierarchical-block conditioning approximations for high-dimensional multivariate normal probabilities","volume":"29","author":"Cao,","year":"2019","journal-title":"Statist. Comp."},{"key":"2022112918000261700_B9","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1214\/09-AOAS285","article-title":"BART: Bayesian additive regression trees","volume":"4","author":"Chipman,","year":"2010","journal-title":"Ann. Appl. Statist."},{"key":"2022112918000261700_B10","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1214\/16-STS581","article-title":"Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation","volume":"32","author":"Chopin,","year":"2017","journal-title":"Statist. Sci."},{"key":"2022112918000261700_B11","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1016\/j.csda.2006.10.028","article-title":"Mean-field variational approximate Bayesian inference for latent variable models","volume":"52","author":"Consonni,","year":"2007","journal-title":"Comp. Statist. Data Anal."},{"key":"2022112918000261700_B12","doi-asserted-by":"crossref","first-page":"e18850","DOI":"10.1371\/journal.pone.0018850","article-title":"Multiplexed immunoassay panel identifies novel CSF biomarkers for Alzheimer\u2019s disease diagnosis and prognosis","volume":"6","author":"Craig-Schapiro,","year":"2011","journal-title":"PloS one"},{"key":"2022112918000261700_B13","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1093\/biomet\/asz034","article-title":"Conjugate Bayes for probit regression via unified skew-normal distributions","volume":"106","author":"Durante,","year":"2019","journal-title":"Biometrika"},{"key":"2022112918000261700_B14","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1214\/19-STS712","article-title":"Conditionally conjugate mean-field variational Bayes for logistic models","volume":"34","author":"Durante,","year":"2019","journal-title":"Statist. Sci."},{"key":"2022112918000261700_B15","first-page":"1","article-title":"A class of conjugate priors for multinomial probit models which includes the multivariate normal one","volume":"23","author":"Fasano,","year":"2022","journal-title":"J. Mach. Learn. Res."},{"key":"2022112918000261700_B16","doi-asserted-by":"crossref","first-page":"3509","DOI":"10.1016\/j.csda.2006.10.006","article-title":"Auxiliary mixture sampling with applications to logistic models","volume":"51","author":"Fr\u00fchwirth-Schnatter,","year":"2007","journal-title":"Comp. Statist. Data Anal."},{"key":"2022112918000261700_B17","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1214\/19-BA1185","article-title":"Intuitive joint priors for variance parameters","volume":"15","author":"Fuglstad,","year":"2020","journal-title":"Bayesian Anal."},{"key":"2022112918000261700_B18","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1214\/08-AOAS191","article-title":"A weakly informative default prior distribution for logistic and other regression models","volume":"2","author":"Gelman,","year":"2008","journal-title":"Ann. Appl. Statist."},{"key":"2022112918000261700_B19","doi-asserted-by":"crossref","first-page":"1790","DOI":"10.1162\/neco.2006.18.8.1790","article-title":"Variational Bayesian multinomial probit regression with Gaussian process priors","volume":"18","author":"Girolami,","year":"2006","journal-title":"Neural Comp."},{"key":"2022112918000261700_B20","doi-asserted-by":"crossref","first-page":"223","DOI":"10.2307\/3318737","article-title":"An adaptive Metropolis algorithm","volume":"7","author":"Haario,","year":"2001","journal-title":"Bernoulli"},{"key":"2022112918000261700_B21","first-page":"1593","article-title":"The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo","volume":"15","author":"Hoffman,","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"2022112918000261700_B22","first-page":"145","article-title":"Bayesian auxiliary variable models for binary and multinomial regression","volume":"1","author":"Holmes,","year":"2006","journal-title":"Bayesian Anal."},{"key":"2022112918000261700_B23","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1016\/j.jmva.2004.10.007","article-title":"Some results on the multivariate truncated normal distribution","volume":"94","author":"Horrace,","year":"2005","journal-title":"J. Mult. Anal."},{"key":"2022112918000261700_B24","doi-asserted-by":"crossref","first-page":"1394","DOI":"10.1080\/01621459.2018.1505626","article-title":"MCMC for imbalanced categorical data","volume":"114","author":"Johndrow,","year":"2019","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B25","first-page":"1701","article-title":"Non-conjugate variational message passing for multinomial and binary regression","volume-title":"Proc. 24th Int. Conf. Adv. Neural Info. Proces. Syst. (NIPS 2011)","author":"Knowles,","year":"2011"},{"key":"2022112918000261700_B26","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback,","year":"1951","journal-title":"Ann. Math. Statist."},{"key":"2022112918000261700_B27","first-page":"1679","article-title":"Assessing approximate inference for binary Gaussian process classification","volume":"6","author":"Kuss,","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"2022112918000261700_B28","first-page":"633","article-title":"Piecewise bounds for estimating Bernoulli-logistic latent Gaussian models","volume-title":"Proc. 28th Int. Conf. Mach. Learn.","author":"Marlin,","year":"2011"},{"key":"2022112918000261700_B29","doi-asserted-by":"crossref","first-page":"2051","DOI":"10.1109\/TMI.2016.2547947","article-title":"Computer-aided classification of gastrointestinal lesions in regular colonoscopy","volume":"35","author":"Mesejo,","year":"2016","journal-title":"IEEE Trans. Med. Imag."},{"key":"2022112918000261700_B30","first-page":"362","article-title":"Expectation propagation for approximate Bayesian inference","volume-title":"Proc. 17th Conf. Uncertainty in Artif. Intel. (UAI\u201901)","author":"Minka,","year":"2001"},{"key":"2022112918000261700_B31","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1080\/01621459.2013.829001","article-title":"Bayesian inference for logistic models using P\u00f3lya\u2013Gamma latent variables","volume":"108","author":"Polson,","year":"2013","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B32","doi-asserted-by":"crossref","first-page":"2320","DOI":"10.1214\/18-AOS1749","article-title":"Convergence complexity analysis of Albert and Chib\u2019s algorithm for Bayesian probit regression","volume":"47","author":"Qin,","year":"2019","journal-title":"Ann. Statist."},{"key":"2022112918000261700_B33","article-title":"R: A Language and Environment for Statistical Computing","year":"2022"},{"key":"2022112918000261700_B34","doi-asserted-by":"crossref","DOI":"10.1080\/01621459.2020.1847121","article-title":"Variational Bayes for high-dimensional linear regression with sparse priors","author":"Ray,","year":"2022","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B35","first-page":"14423","article-title":"Spike and slab variational Bayes for high dimensional logistic regression","volume-title":"Proc. 34th Int. Conf. Adv. Neural Info. Proces. Systems (NIPS 2020)","author":"Ray,","year":"2020"},{"key":"2022112918000261700_B36","doi-asserted-by":"crossref","first-page":"1957","DOI":"10.1214\/07-AOS525","article-title":"Asymptotic equivalence for nonparametric regression with multivariate and random design","volume":"36","author":"Rei\u00df,","year":"2008","journal-title":"Ann. Statist."},{"key":"2022112918000261700_B37","first-page":"145","article-title":"Nonparametric Bayesian models through probit stick-breaking processes","volume":"6","author":"Rodriguez,","year":"2011","journal-title":"Bayesian Anal."},{"key":"2022112918000261700_B38","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.asoc.2018.10.022","article-title":"A comparative analysis of speech signal processing algorithms for Parkinson\u2019s disease classification and the use of the tunable Q-factor wavelet transform","volume":"74","author":"Sakar,","year":"2019","journal-title":"Appl. Soft Comp."},{"key":"2022112918000261700_B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/16-STS576","article-title":"Penalising model component complexity: a principled, practical approach to constructing priors","volume":"32","author":"Simpson,","year":"2017","journal-title":"Statist. Sci."},{"key":"2022112918000261700_B40","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1109\/TNSRE.2013.2293575","article-title":"Objective automatic assessment of rehabilitative speech treatment in Parkinson\u2019s disease","volume":"22","author":"Tsanas,","year":"2013","journal-title":"IEEE Trans. Neural Syst. Rehab. Eng."},{"key":"2022112918000261700_B41","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1080\/01621459.2018.1473776","article-title":"Frequentist consistency of variational Bayes","volume":"114","author":"Wang,","year":"2019","journal-title":"J. Am. Statist. Assoc."},{"key":"2022112918000261700_B42","first-page":"886","article-title":"$\\alpha$-variational inference with statistical guarantees","volume":"48","author":"Yang,","year":"2022","journal-title":"Ann. Statist."}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asac026\/45715385\/asac026.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/901\/47415778\/asac026.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/901\/47415778\/asac026.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,29]],"date-time":"2022-11-29T18:03:03Z","timestamp":1669744983000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/109\/4\/901\/6581071"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,5]]},"references-count":42,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,5,5]]},"published-print":{"date-parts":[[2022,11,29]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asac026","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,12,1]]},"published":{"date-parts":[[2022,5,5]]}}}