{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T10:25:30Z","timestamp":1740133530722,"version":"3.37.3"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T00:00:00Z","timestamp":1694563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["DMS 1513040 and DMS 2210664"],"award-info":[{"award-number":["DMS 1513040 and DMS 2210664"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,2,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We propose a very fast approximate Markov chain Monte Carlo sampling framework that is applicable to a large class of sparse Bayesian inference problems. The computational cost per iteration in several regression models is of order O(n(s+J)), where n is the sample size, s is the underlying sparsity of the model, and J is the size of a randomly selected subset of regressors. This cost can be further reduced by data sub-sampling when stochastic gradient Langevin dynamics are employed. The algorithm is an extension of the asynchronous Gibbs sampler of Johnson et al. [(2013). Analyzing Hogwild parallel Gaussian Gibbs sampling. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS\u201913) (Vol. 2, pp. 2715\u20132723)], but can be viewed from a statistical perspective as a form of Bayesian iterated sure independent screening [Fan, J., Samworth, R., &amp; Wu, Y. (2009). Ultrahigh dimensional feature selection: Beyond the linear model. Journal of Machine Learning Research, 10, 2013\u20132038]. We show that in high-dimensional linear regression problems, the Markov chain generated by the proposed algorithm admits an invariant distribution that recovers correctly the main signal with high probability under some statistical assumptions. Furthermore, we show that its mixing time is at most linear in the number of regressors. We illustrate the algorithm with several models.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkad078","type":"journal-article","created":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T13:41:31Z","timestamp":1694612491000},"page":"1492-1516","source":"Crossref","is-referenced-by-count":0,"title":["A fast asynchronous Markov chain Monte Carlo sampler for sparse Bayesian inference"],"prefix":"10.1093","volume":"85","author":[{"given":"Yves","family":"Atchad\u00e9","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, Boston University , Boston, MA , USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liwei","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Boston University , Boston, MA , USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,9,13]]},"reference":[{"issue":"3","key":"2024091320573399800_qkad078-B1","doi-asserted-by":"crossref","first-page":"854","DOI":"10.1137\/19M1283082","article-title":"Approximate spectral gaps for Markov chains in high-dimensions","volume":"3","author":"Atchad\u00e9","year":"2021","journal-title":"SIAM Journal on Mathematics of Data Science"},{"year":"2019","author":"Atchad\u00e9","key":"2024091320573399800_qkad078-B2"},{"year":"2011","author":"Atchad\u00e9","key":"2024091320573399800_qkad078-B3"},{"issue":"5","key":"2024091320573399800_qkad078-B4","doi-asserted-by":"crossref","first-page":"2248","DOI":"10.1214\/16-AOS1526","article-title":"On the contraction properties of some high-dimensional quasi-posterior distributions","volume":"45","author":"Atchad\u00e9","year":"2017","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2024091320573399800_qkad078-B5","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1111\/insr.12360","article-title":"Horseshoe regularisation for machine learning in complex and deep models1","volume":"88","author":"Bhadra","year":"2020","journal-title":"International Statistical Review"},{"issue":"4","key":"2024091320573399800_qkad078-B6","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1093\/biomet\/asw042","article-title":"Fast sampling with Gaussian scale mixture priors in high-dimensional regression","volume":"103","author":"Bhattacharya","year":"2016","journal-title":"Biometrika"},{"year":"2021","author":"Biswas","key":"2024091320573399800_qkad078-B7"},{"year":"2019","author":"Biswas","key":"2024091320573399800_qkad078-B8"},{"year":"2016","author":"Blei","key":"2024091320573399800_qkad078-B9"},{"key":"2024091320573399800_qkad078-B10","doi-asserted-by":"crossref","DOI":"10.1201\/b10905","volume-title":"Handbook of Markov chain Monte Carlo","author":"Brooks","year":"2011"},{"key":"2024091320573399800_qkad078-B11","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-20192-9","volume-title":"Statistics for high-dimensional data: Methods, theory and applications","author":"B\u00fchlmann","year":"2011"},{"issue":"5A","key":"2024091320573399800_qkad078-B12","doi-asserted-by":"crossref","first-page":"2145","DOI":"10.1214\/08-AOS653","article-title":"Near-ideal model selection by \u21131 minimization","volume":"37","author":"Cand\u00e8s","year":"2009","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2024091320573399800_qkad078-B13","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1093\/biomet\/asq017","article-title":"The horseshoe estimator for sparse signals","volume":"97","author":"Carvalho","year":"2010","journal-title":"Biometrika"},{"issue":"5","key":"2024091320573399800_qkad078-B14","doi-asserted-by":"crossref","first-page":"1986","DOI":"10.1214\/15-AOS1334","article-title":"Bayesian linear regression with sparse priors","volume":"43","author":"Castillo","year":"2015","journal-title":"Annals of Statistics"},{"year":"2018","author":"Daskalakis","key":"2024091320573399800_qkad078-B15"},{"year":"2016","author":"De Sa","key":"2024091320573399800_qkad078-B16"},{"issue":"5","key":"2024091320573399800_qkad078-B17","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1111\/j.1467-9868.2008.00674.x","article-title":"Sure independence screening for ultrahigh dimensional feature space","volume":"70","author":"Fan","year":"2008","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2024091320573399800_qkad078-B18","first-page":"2013","article-title":"Ultrahigh dimensional feature selection: Beyond the linear model","volume":"10","author":"Fan","year":"2009","journal-title":"Journal of Machine Learning Research"},{"year":"2019","author":"Frankle","key":"2024091320573399800_qkad078-B19"},{"year":"2016","author":"Gal","key":"2024091320573399800_qkad078-B20"},{"year":"2019","author":"Gale","key":"2024091320573399800_qkad078-B21"},{"key":"2024091320573399800_qkad078-B22","first-page":"339","article-title":"Approaches to Bayesian variable selection","volume":"7","author":"George","year":"1997","journal-title":"Statistica Sinica"},{"issue":"182","key":"2024091320573399800_qkad078-B23","first-page":"1","article-title":"Model selection in Bayesian neural networks via horseshoe priors","volume":"20","author":"Ghosh","year":"2019","journal-title":"Journal of Machine Learning Research"},{"key":"2024091320573399800_qkad078-B24","doi-asserted-by":"crossref","DOI":"10.1201\/b18401","volume-title":"Statistical learning with sparsity: The Lasso and generalizations","author":"Hastie","year":"2015"},{"issue":"73","key":"2024091320573399800_qkad078-B25","first-page":"1","article-title":"Scalable approximate MCMC algorithms for the horseshoe prior","volume":"21","author":"Johndrow","year":"2020","journal-title":"Journal of Machine Learning Research"},{"year":"2018","author":"Johndrow","key":"2024091320573399800_qkad078-B26"},{"year":"2013","author":"Johnson","key":"2024091320573399800_qkad078-B27"},{"year":"2014","author":"Kingma","key":"2024091320573399800_qkad078-B28"},{"year":"2010","author":"LeCun","key":"2024091320573399800_qkad078-B29"},{"year":"2017","author":"Louizos","key":"2024091320573399800_qkad078-B30"},{"year":"2015","author":"Ma","key":"2024091320573399800_qkad078-B31"},{"issue":"1","key":"2024091320573399800_qkad078-B32","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1214\/07-AOS582","article-title":"Lasso-type recovery of sparse representations for high-dimensional data","volume":"37","author":"Meinshausen","year":"2009","journal-title":"Annals of Statistics"},{"key":"2024091320573399800_qkad078-B33","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511626630","volume-title":"Markov chains and stochastic stability","author":"Meyn","year":"2009","edition":"2nd ed."},{"issue":"404","key":"2024091320573399800_qkad078-B34","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1080\/01621459.1988.10478694","article-title":"Bayesian variable selection in linear regression","volume":"83","author":"Mitchell","year":"1988","journal-title":"Journal of the American Statistical Association"},{"issue":"527","key":"2024091320573399800_qkad078-B35","doi-asserted-by":"crossref","first-page":"1205","DOI":"10.1080\/01621459.2018.1482754","article-title":"Skinny Gibbs: A consistent and scalable Gibbs sampler for model selection","volume":"114","author":"Narisetty","year":"2019","journal-title":"Journal of the American Statistical Association"},{"issue":"4","key":"2024091320573399800_qkad078-B36","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1214\/12-STS400","article-title":"A unified framework for high-dimensional analysis of m-estimators with decomposable regularizers","volume":"27","author":"Negahban","year":"2012","journal-title":"Statistical Science"},{"year":"2014","author":"Neiswanger","key":"2024091320573399800_qkad078-B37"},{"issue":"2","key":"2024091320573399800_qkad078-B38","doi-asserted-by":"crossref","first-page":"5018","DOI":"10.1214\/17-EJS1337SI","article-title":"Sparsity information and regularization in the horseshoe and other shrinkage priors","volume":"11","author":"Piironen","year":"2017","journal-title":"Electronic Journal of Statistics"},{"year":"2015","author":"Pillai","key":"2024091320573399800_qkad078-B39"},{"issue":"1","key":"2024091320573399800_qkad078-B40","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1080\/10618600.2018.1482767","article-title":"Uncertainty quantification for modern high-dimensional regression via scalable Bayesian methods","volume":"28","author":"Rajaratnam","year":"2019","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"539","key":"2024091320573399800_qkad078-B41","doi-asserted-by":"crossref","first-page":"1270","DOI":"10.1080\/01621459.2020.1847121","article-title":"Variational Bayes for high-dimensional linear regression with sparse priors","volume":"117","author":"Ray","year":"2022","journal-title":"Journal of the American Statistical Association"},{"key":"2024091320573399800_qkad078-B42","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-4145-2","volume-title":"Monte Carlo statistical methods","author":"Robert","year":"2004"},{"issue":"4","key":"2024091320573399800_qkad078-B43","doi-asserted-by":"crossref","first-page":"341","DOI":"10.2307\/3318418","article-title":"Exponential convergence of Langevin distributions and their discrete approximations","volume":"2","author":"Roberts","year":"1996","journal-title":"Bernoulli"},{"issue":"521","key":"2024091320573399800_qkad078-B44","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1080\/01621459.2016.1260469","article-title":"The spike-and-slab lasso","volume":"113","author":"Rockova","year":"2018","journal-title":"Journal of the American Statistical Association"},{"issue":"4A","key":"2024091320573399800_qkad078-B45","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.3150\/17-BEJ938","article-title":"Perturbation theory for Markov chains via Wasserstein distance","volume":"24","author":"Rudolf","year":"2018","journal-title":"Bernoulli"},{"issue":"1\u20132","key":"2024091320573399800_qkad078-B46","doi-asserted-by":"crossref","first-page":"703","DOI":"10.14778\/1920841.1920931","article-title":"An architecture for parallel topic models","volume":"3","author":"Smola","year":"2010","journal-title":"Proceedings of the VLDB Endowment"},{"issue":"3","key":"2024091320573399800_qkad078-B47","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1080\/10618600.2020.1869025","article-title":"Distributed Bayesian inference in linear mixed-effects models","volume":"30","author":"Srivastava","year":"2021","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"1","key":"2024091320573399800_qkad078-B48","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1080\/10618600.2019.1637747","article-title":"Bayesian deep net GLM and GLMM","volume":"29","author":"Tran","year":"2020","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2024091320573399800_qkad078-B49","doi-asserted-by":"crossref","DOI":"10.1017\/9781108627771","volume-title":"High-dimensional statistics: A non-asymptotic viewpoint","author":"Wainwright","year":"2019"},{"year":"2011","author":"Welling","key":"2024091320573399800_qkad078-B50"},{"year":"2017","author":"Xiao","key":"2024091320573399800_qkad078-B51"},{"issue":"1","key":"2024091320573399800_qkad078-B52","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s11222-017-9791-1","article-title":"Double-parallel Monte Carlo for Bayesian analysis of big data","volume":"29","author":"Xue","year":"2019","journal-title":"Statistics and Computing"},{"issue":"6","key":"2024091320573399800_qkad078-B53","doi-asserted-by":"crossref","first-page":"2497","DOI":"10.1214\/15-AOS1417","article-title":"On the computational complexity of high-dimensional Bayesian variable selection","volume":"44","author":"Yang","year":"2016","journal-title":"Annals of Statistics"},{"year":"2019","author":"Zhang","key":"2024091320573399800_qkad078-B54"},{"issue":"4","key":"2024091320573399800_qkad078-B55","doi-asserted-by":"crossref","first-page":"2180","DOI":"10.1214\/19-AOS1883","article-title":"Convergence rates of variational posterior distributions","volume":"48","author":"Zhang","year":"2020","journal-title":"Annals of Statistics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/5\/1492\/56555438\/qkad078.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/5\/1492\/56555438\/qkad078.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,14]],"date-time":"2024-09-14T07:52:03Z","timestamp":1726300323000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/85\/5\/1492\/7272760"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,13]]},"references-count":55,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,9,13]]},"published-print":{"date-parts":[[2024,2,2]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkad078","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"type":"print","value":"1369-7412"},{"type":"electronic","value":"1467-9868"}],"subject":[],"published-other":{"date-parts":[[2023,11]]},"published":{"date-parts":[[2023,9,13]]}}}