{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T07:21:47Z","timestamp":1789888907971,"version":"4.0.1"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2018,3,1]],"date-time":"2018-03-01T00:00:00Z","timestamp":1519862400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,9,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>We introduce a new family of Markov chain Monte Carlo samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalization over the auxiliary variables, yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well-known Metropolis-adjusted Langevin algorithm MALA and preconditioned Crank\u2013Nicolson\u2013Langevin algorithm pCNL are shown to be special cases. We prove that marginal samplers are superior in terms of asymptotic variance and demonstrate cases where they are slower in computing time compared with auxiliary samplers. In the context of latent Gaussian models we propose new auxiliary and marginal samplers whose implementation requires a single tuning parameter, which can be found automatically during the transient phase. Extensive experimentation shows that the increase in efficiency (measured as the effective sample size per unit of computing time) relative to (optimized implementations of) pCNL, elliptical slice sampling and MALA ranges from tenfold in binary classification problems to 25 fold in log-Gaussian Cox processes to 100 fold in Gaussian process regression, and it is on a par with Riemann manifold Hamiltonian Monte Carlo sampling in an example where that algorithm has the same complexity as the aforementioned algorithms. We explain this remarkable improvement in terms of the way that alternative samplers try to approximate the eigenvalues of the target. We introduce a novel Markov chain Monte Carlo sampling scheme for hyperparameter learning that builds on the auxiliary samplers. The MATLAB code for reproducing the experiments in the paper is publicly available and an on-line supplement to this paper contains additional experiments and implementation details.<\/jats:p>","DOI":"10.1111\/rssb.12269","type":"journal-article","created":{"date-parts":[[2018,3,2]],"date-time":"2018-03-02T01:42:45Z","timestamp":1519954965000},"page":"749-767","source":"Crossref","is-referenced-by-count":22,"title":["Auxiliary Gradient-Based Sampling Algorithms"],"prefix":"10.1093","volume":"80","author":[{"given":"Michalis K.","family":"Titsias","sequence":"first","affiliation":[{"name":"Athens University of Economics and Business , Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Omiros","family":"Papaspiliopoulos","sequence":"additional","affiliation":[{"name":"Instituci\u00f3 Catalana de Recerca Estudis Avan\u00e7ats and Universitat Pompeu Fabra , Barcelona, and Barcelona Graduate School of Economics , Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,3,1]]},"reference":[{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1214\/14-AAP1022","article-title":"Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms","volume":"25","author":"Andrieu","year":"2015","journal-title":"Ann. Appl. Probab."},{"key":"2023022002492455500_","volume-title":"Effcient parameterisations of Gaussian process based models for Bayesian computation using MCMC","author":"Bass","year":"2016"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1142\/S0219493708002378","article-title":"MCMC methods for diffusion bridges","volume":"8","author":"Beskos","year":"2008","journal-title":"Stochast. Dyn."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1214\/13-STS421","article-title":"MCMC methods for functions: modifying old algorithms to make them faster","volume":"28","author":"Cotter","year":"2013","journal-title":"Statist. Sci."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.jcp.2015.10.008","article-title":"Dimension-independent likelihood-informed MCMC","volume":"304","author":"Cui","year":"2016","journal-title":"J. Computnl Phys."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1111\/rssb.12183","article-title":"Theoretical guarantees for approximate sampling from smooth and log-concave densities","volume":"79","author":"Dalalyan","year":"2017","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492455500_","article-title":"Effective implementation of Gaussian process regression for machine learning","author":"Davies","year":"2015"},{"key":"2023022002492455500_","article-title":"Non-asymptotic convergence analysis for the unadjusted Langevin algorithm","volume-title":"Preprint","author":"Durmus","year":"2016"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s10994-013-5388-x","article-title":"A comparative evaluation of stochastic-based inference methods for Gaussian process models","volume":"93","author":"Filippone","year":"2013","journal-title":"Mach. Learn."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","article-title":"Riemann manifold Langevin and Hamiltonian Monte Carlo methods (with discussion)","volume":"73","author":"Girolami","year":"2011","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492455500_","article-title":"MCMC for variationally sparse Gaussian processes","volume-title":"Neural and Information Processing Systems","author":"Hensman","year":"2015"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1111\/1467-9469.00308","article-title":"On block updating in Markov random field models for disease mapping","volume":"29","author":"Knorr-Held","year":"2002","journal-title":"Scand. J. Statist."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.cam.2013.07.026","article-title":"Proposals which speed up function-space MCMC","volume":"262","author":"Law","year":"2014","journal-title":"J. Computnl Appl. Math."},{"key":"2023022002492455500_","first-page":"1641","article-title":"An extension of Peskun and Tierney orderings to continuous time Markov chains","volume":"18","author":"Leisen","year":"2008","journal-title":"Statist. Sin."},{"key":"2023022002492455500_","first-page":"1723","article-title":"Slice sampling covariance hyperparameters of latent Gaussian models.","volume-title":"Advances in Neural Information Processing Systems","author":"Murray","year":"2010"},{"key":"2023022002492455500_","first-page":"541","article-title":"Elliptical slice sampling","volume":"9","author":"Murray","year":"2010","journal-title":"J. Mach. Learn. Res. Wrkshp Conf. Proc."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1093\/oso\/9780198504856.003.0021","article-title":"Regression and classification using Gaussian process priors","volume-title":"Bayesian Statistics","author":"Neal","year":"1999"},{"key":"2023022002492455500_","first-page":"307","article-title":"Non-centered parameterizations for hierarchical models and data augmentation","volume-title":"Bayesian Statistics","author":"Papaspiliopoulos","year":"2003"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1093\/biomet\/60.3.607","article-title":"Optimum Monte-Carlo sampling using Markov chains","volume":"60","author":"Peskun","year":"1973","journal-title":"Biometrika"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1214\/ss\/1015346320","article-title":"Optimal scaling for various Metropolis-Hastings algorithms","volume":"16","author":"Roberts","year":"2001","journal-title":"Statist. Sci."},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"341","DOI":"10.2307\/3318418","article-title":"Exponential convergence of Langevin distributions and their discrete approximations","volume":"2","author":"Roberts","year":"1996","journal-title":"Bernoulli"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"774","DOI":"10.3150\/08-BEJ176","article-title":"Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets","volume":"15","author":"Sherlock","year":"2009","journal-title":"Bernoulli"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1111\/j.1467-9469.2010.00709.x","article-title":"On the flexibility of Metropolis-Hastings acceptance probabilities in auxiliary variable proposal generation","volume":"38","author":"Storvik","year":"2011","journal-title":"Scand. J. Statist."},{"key":"2023022002492455500_","first-page":"1723","article-title":"Bayesian learning via stochastic gradient Langevin dynamics","volume-title":"Proc. Int. Conf. Machine Learning","author":"Teh","year":"2011"},{"key":"2023022002492455500_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/aoap\/1027961031","article-title":"A note on Metropolis\u2013Hastings kernels for general state spaces","volume":"8","author":"Tierney","year":"1998","journal-title":"Ann. Appl. Probab."},{"key":"2023022002492455500_","first-page":"197","article-title":"Discussion on \u2018Riemann manifold Langevin and Hamiltonian Monte Carlo methods\u2019","volume":"73","author":"Titsias","year":"2011","journal-title":"J. R. Statist. Soc."}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12269","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/4\/749\/49270656\/jrsssb_80_4_749.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/4\/749\/49270656\/jrsssb_80_4_749.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,1]],"date-time":"2024-07-01T18:04:46Z","timestamp":1719857086000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/80\/4\/749\/7048390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,1]]},"references-count":26,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,3,1]]},"published-print":{"date-parts":[[2018,9,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12269","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,9]]},"published":{"date-parts":[[2018,3,1]]}}}