{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T04:15:42Z","timestamp":1689221742732},"reference-count":62,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,4,24]],"date-time":"2023-04-24T00:00:00Z","timestamp":1682294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,7,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Standard Markov chain Monte Carlo methods struggle to explore distributions that concentrate in the neighbourhood of low-dimensional submanifolds. This pathology naturally occurs in Bayesian inference settings when there is a high signal-to-noise ratio in the observational data but the model is inherently over-parametrised or nonidentifiable. In this paper, we propose a strategy that transforms the original sampling problem into the task of exploring a distribution supported on a manifold embedded in a higher-dimensional space; in contrast to the original posterior this lifted distribution remains diffuse in the limit of vanishing observation noise. We employ a constrained Hamiltonian Monte Carlo method, which exploits the geometry of this lifted distribution, to perform efficient approximate inference. We demonstrate in numerical experiments that, contrarily to competing approaches, the sampling efficiency of our proposed methodology does not degenerate as the target distribution to be explored concentrates near low-dimensional submanifolds. Python code reproducing the results is available at https:\/\/doi.org\/10.5281\/zenodo.6551654.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkad023","type":"journal-article","created":{"date-parts":[[2023,4,24]],"date-time":"2023-04-24T17:19:53Z","timestamp":1682356793000},"page":"757-782","source":"Crossref","is-referenced-by-count":0,"title":["Manifold lifting: scaling Markov chain Monte Carlo to the vanishing noise regime"],"prefix":"10.1093","volume":"85","author":[{"given":"Khai Xiang","family":"Au","sequence":"first","affiliation":[{"name":"Integrative Sciences and Engineering Programme, National University of Singapore , Singapore , Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew M","family":"Graham","sequence":"additional","affiliation":[{"name":"Advanced Research Computing Centre, University College London , London , UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandre H","family":"Thiery","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, National University of Singapore , Singapore , Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,4,24]]},"reference":[{"issue":"100","key":"2023071211303693000_qkad023-B1","first-page":"9","article-title":"The FEniCS project version 1.5","volume":"3","author":"Aln\u00e6s","year":"2015","journal-title":"Archive of Numerical Software"},{"issue":"2","key":"2023071211303693000_qkad023-B2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2566630","article-title":"Unified form language: A domain-specific language for weak formulations of partial differential equations","volume":"40","author":"Aln\u00e6s","year":"2014","journal-title":"ACM Transactions on Mathematical Software (TOMS)"},{"issue":"1","key":"2023071211303693000_qkad023-B3","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/0021-9991(83)90014-1","article-title":"RATTLE: A \u2018velocity\u2019 version of the SHAKE algorithm for molecular dynamics calculations","volume":"52","author":"Andersen","year":"1983","journal-title":"Journal of Computational Physics"},{"issue":"1","key":"2023071211303693000_qkad023-B4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12918-017-0433-1","article-title":"Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems","volume":"11","author":"Ballnus","year":"2017","journal-title":"BMC Systems Biology"},{"issue":"10","key":"2023071211303693000_qkad023-B5","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1002\/jcc.540161003","article-title":"Algorithms for constrained molecular dynamics","volume":"16","author":"Barth","year":"1995","journal-title":"Journal of Computational Chemistry"},{"issue":"4","key":"2023071211303693000_qkad023-B6","first-page":"591","article-title":"Comments on \u2018representations of knowledge in complex systems\u2019 by Grenander and Miller","volume":"56","author":"Besag","year":"1994","journal-title":"Journal of the Royal Statistical Society, Series B"},{"issue":"5A","key":"2023071211303693000_qkad023-B7","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.3150\/12-BEJ414","article-title":"Optimal tuning of the hybrid Monte Carlo algorithm","volume":"19","author":"Beskos","year":"2013","journal-title":"Bernoulli"},{"issue":"5","key":"2023071211303693000_qkad023-B8","doi-asserted-by":"crossref","first-page":"2966","DOI":"10.1214\/18-AAP1380","article-title":"Asymptotic analysis of the random walk Metropolis algorithm on ridged densities","volume":"28","author":"Beskos","year":"2018","journal-title":"The Annals of Applied Probability"},{"key":"2023071211303693000_qkad023-B9","author":"Betancourt","year":"2013"},{"key":"2023071211303693000_qkad023-B10","author":"Betancourt","year":"2017"},{"issue":"4","key":"2023071211303693000_qkad023-B11","doi-asserted-by":"crossref","first-page":"A1556","DOI":"10.1137\/130932740","article-title":"Numerical integrators for the hybrid Monte Carlo method","volume":"36","author":"Blanes","year":"2014","journal-title":"SIAM Journal on Scientific Computing"},{"key":"2023071211303693000_qkad023-B12","author":"Bradbury","year":"2018"},{"key":"2023071211303693000_qkad023-B13","author":"Brubaker","year":"2012"},{"issue":"1","key":"2023071211303693000_qkad023-B14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v076.i01","article-title":"Stan: A probabilistic programming language","volume":"76","author":"Carpenter","year":"2017","journal-title":"Journal of Statistical Software"},{"issue":"12","key":"2023071211303693000_qkad023-B15","doi-asserted-by":"crossref","first-page":"150499","DOI":"10.1098\/rsos.150499","article-title":"Hodgkin\u2013Huxley revisited: Reparametrization and identifiability analysis of the classic action potential model with approximate Bayesian methods","volume":"2","author":"Daly","year":"2015","journal-title":"Royal Society Open Science"},{"issue":"11","key":"2023071211303693000_qkad023-B16","doi-asserted-by":"crossref","first-page":"3380","DOI":"10.1016\/j.jspi.2007.03.018","article-title":"Non-identifiable parametric probability models and reparametrization","volume":"137","author":"Dasgupta","year":"2007","journal-title":"Journal of Statistical Planning and Inference"},{"key":"2023071211303693000_qkad023-B17","author":"Diaconis","year":"2013"},{"issue":"2","key":"2023071211303693000_qkad023-B18","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/0370-2693(87)91197-X","article-title":"Hybrid Monte Carlo","volume":"195","author":"Duane","year":"1987","journal-title":"Physics Letters B"},{"issue":"6","key":"2023071211303693000_qkad023-B19","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/S0006-3495(61)86902-6","article-title":"Impulses and physiological states in theoretical models of nerve membrane","volume":"1","author":"FitzHugh","year":"1961","journal-title":"Biophysical Journal"},{"issue":"2","key":"2023071211303693000_qkad023-B20","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","article-title":"Riemann manifold Langevin and Hamiltonian Monte Carlo methods","volume":"73","author":"Girolami","year":"2011","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023071211303693000_qkad023-B21","author":"Graham","year":"2019"},{"issue":"2","key":"2023071211303693000_qkad023-B22","doi-asserted-by":"crossref","first-page":"5105","DOI":"10.1214\/17-EJS1340SI","article-title":"Asymptotically exact inference in differentiable generative models","volume":"11","author":"Graham","year":"2017","journal-title":"Electronic Journal of Statistics"},{"issue":"3\u20134","key":"2023071211303693000_qkad023-B23","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/0370-2693(90)91790-I","article-title":"The acceptance probability in the hybrid Monte Carlo method","volume":"242","author":"Gupta","year":"1990","journal-title":"Physics Letters, B"},{"issue":"7825","key":"2023071211303693000_qkad023-B24","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"issue":"10","key":"2023071211303693000_qkad023-B25","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1002\/zamm.200410218","article-title":"A constrained hybrid Monte-Carlo algorithm and the problem of calculating the free energy in several variables","volume":"85","author":"Hartmann","year":"2005","journal-title":"ZAMM-Journal of Applied Mathematics and Mechanics\/Zeitschrift f\u00fcr Angewandte Mathematik und Mechanik: Applied Mathematics and Mechanics"},{"key":"2023071211303693000_qkad023-B26","volume-title":"The conservation of the wild life of Canada","author":"Hewitt","year":"1921"},{"issue":"3","key":"2023071211303693000_qkad023-B27","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1085\/jgp.201311116","article-title":"Determination of parameter identifiability in nonlinear biophysical models: A Bayesian approach","volume":"143","author":"Hines","year":"2014","journal-title":"Journal of General Physiology"},{"issue":"4","key":"2023071211303693000_qkad023-B28","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1113\/jphysiol.1952.sp004764","article-title":"A quantitative description of membrane current and its application to conduction and excitation in nerve","volume":"117","author":"Hodgkin","year":"1952","journal-title":"The Journal of physiology"},{"issue":"47","key":"2023071211303693000_qkad023-B29","first-page":"1593","article-title":"The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo","volume":"15","author":"Hoffman","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"2023071211303693000_qkad023-B30","volume-title":"Modeling basics","author":"Howard","year":"2009"},{"issue":"3","key":"2023071211303693000_qkad023-B31","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: A 2d graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Computing in Science & Engineering"},{"issue":"5","key":"2023071211303693000_qkad023-B32","doi-asserted-by":"crossref","first-page":"2626","DOI":"10.1214\/11-AOS920","article-title":"Bayesian inverse problems with Gaussian priors","volume":"39","author":"Knapik","year":"2011","journal-title":"The Annals of Statistics"},{"issue":"33","key":"2023071211303693000_qkad023-B33","doi-asserted-by":"crossref","DOI":"10.21105\/joss.01143","article-title":"ArviZ a unified library for exploratory analysis of Bayesian models in Python","volume":"4","author":"Kumar","year":"2019","journal-title":"The Journal of Open Source Software"},{"issue":"2189","key":"2023071211303693000_qkad023-B34","doi-asserted-by":"crossref","first-page":"20160138","DOI":"10.1098\/rspa.2016.0138","article-title":"Efficient molecular dynamics using geodesic integration and solvent\u2013solute splitting","volume":"472","author":"Leimkuhler","year":"2016","journal-title":"Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences"},{"key":"2023071211303693000_qkad023-B35","volume-title":"Simulating Hamiltonian dynamics","author":"Leimkuhler","year":"2004"},{"issue":"1","key":"2023071211303693000_qkad023-B36","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1006\/jcph.1994.1085","article-title":"Symplectic numerical integrators in constrained Hamiltonian systems","volume":"112","author":"Leimkuhler","year":"1994","journal-title":"Journal of Computational Physics"},{"issue":"2","key":"2023071211303693000_qkad023-B37","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1007\/s00211-019-01056-4","article-title":"Hybrid Monte Carlo methods for sampling probability measures on submanifolds","volume":"143","author":"Leli\u00e8vre","year":"2019","journal-title":"Numerische Mathematik"},{"issue":"4","key":"2023071211303693000_qkad023-B38","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1111\/j.1467-9868.2011.00777.x","article-title":"An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach","volume":"73","author":"Lindgren","year":"2011","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023071211303693000_qkad023-B39","author":"Livingstone","year":"2015"},{"key":"2023071211303693000_qkad023-B40","author":"Livingstone","year":"2019"},{"key":"2023071211303693000_qkad023-B41","volume-title":"Elements of physical biology","author":"Lotka","year":"1925"},{"issue":"6","key":"2023071211303693000_qkad023-B42","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1063\/1.1699114","article-title":"Equation of state calculations by fast computing machines","volume":"21","author":"Metropolis","year":"1953","journal-title":"The Journal of Chemical Physics"},{"issue":"10","key":"2023071211303693000_qkad023-B43","doi-asserted-by":"crossref","first-page":"2061","DOI":"10.1109\/JRPROC.1962.288235","article-title":"An active pulse transmission line simulating nerve axon","volume":"50","author":"Nagumo","year":"1962","journal-title":"Proceedings of the IRE"},{"key":"2023071211303693000_qkad023-B44","doi-asserted-by":"crossref","DOI":"10.1201\/b10905-6","volume-title":"MCMC using Hamiltonian dynamics","author":"Neal","year":"2011"},{"issue":"4","key":"2023071211303693000_qkad023-B45","doi-asserted-by":"crossref","first-page":"A1525","DOI":"10.1137\/130934805","article-title":"A computational framework for infinite-dimensional Bayesian inverse problems, part II: Stochastic newton MCMC with application to ice sheet flow inverse problems","volume":"36","author":"Petra","year":"2014","journal-title":"SIAM Journal on Scientific Computing"},{"issue":"4-5","key":"2023071211303693000_qkad023-B46","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s00422-008-0263-8","article-title":"Minimal Hodgkin\u2013Huxley type models for different classes of cortical and thalamic neurons","volume":"99","author":"Pospischil","year":"2008","journal-title":"Biological cybernetics"},{"issue":"1984","key":"2023071211303693000_qkad023-B47","doi-asserted-by":"crossref","first-page":"20110544","DOI":"10.1098\/rsta.2011.0544","article-title":"Joining forces of Bayesian and frequentist methodology: A study for inference in the presence of non-identifiability","volume":"371","author":"Raue","year":"2013","journal-title":"Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences"},{"issue":"2","key":"2023071211303693000_qkad023-B48","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1137\/0733025","article-title":"Symplectic integration of constrained Hamiltonian systems by composition methods","volume":"33","author":"Reich","year":"1996","journal-title":"SIAM Journal on Numerical Analysis"},{"issue":"1","key":"2023071211303693000_qkad023-B49","first-page":"110","article-title":"Weak convergence and optimal scaling of random walk Metropolis algorithms","volume":"7","author":"Roberts","year":"1997","journal-title":"The Annals of Applied Probability"},{"issue":"4","key":"2023071211303693000_qkad023-B50","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1214\/ss\/1015346320","article-title":"Optimal scaling for various Metropolis\u2013Hastings algorithms","volume":"16","author":"Roberts","year":"2001","journal-title":"Statistical science"},{"issue":"4","key":"2023071211303693000_qkad023-B51","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1023\/A:1023562417138","article-title":"Langevin diffusions and Metropolis\u2013Hastings algorithms","volume":"4","author":"Roberts","year":"2002","journal-title":"Methodology and computing in applied probability"},{"issue":"3","key":"2023071211303693000_qkad023-B52","doi-asserted-by":"crossref","first-page":"577","DOI":"10.2307\/1913267","article-title":"Identification in parametric models","volume":"39","author":"Rothenberg","year":"1971","journal-title":"Econometrica: Journal of the Econometric Society"},{"key":"2023071211303693000_qkad023-B53","volume-title":"Free energy computations: A mathematical perspective","author":"Rousset","year":"2010"},{"key":"2023071211303693000_qkad023-B54","author":"Schillings","year":"2019"},{"issue":"5","key":"2023071211303693000_qkad023-B55","doi-asserted-by":"crossref","first-page":"1919","DOI":"10.5194\/gmd-7-1919-2014","article-title":"Modeling radiocarbon dynamics in soils: SoilR version 1.1","volume":"7","author":"Sierra","year":"2014","journal-title":"Geoscientific Model Development"},{"key":"2023071211303693000_qkad023-B56","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1017\/S0962492910000061","article-title":"Inverse problems: A Bayesian perspective","volume":"19","author":"Stuart","year":"2010","journal-title":"Acta Numerica"},{"key":"2023071211303693000_qkad023-B57","author":"Vehtari","year":"2019"},{"issue":"3","key":"2023071211303693000_qkad023-B58","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nature Methods"},{"issue":"2972","key":"2023071211303693000_qkad023-B59","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1038\/118558a0","article-title":"Fluctuations in the abundance of a species considered mathematically","volume":"118","author":"Volterra","year":"1926","journal-title":"Nature"},{"issue":"8","key":"2023071211303693000_qkad023-B60","doi-asserted-by":"crossref","first-page":"2224","DOI":"10.4249\/scholarpedia.2224","article-title":"Intracellular recording","volume":"5","author":"Weckstrom","year":"2010","journal-title":"Scholarpedia"},{"key":"2023071211303693000_qkad023-B61","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.spl.2014.04.002","article-title":"Langevin diffusions and the Metropolis-adjusted Langevin algorithm","volume":"91","author":"Xifara","year":"2014","journal-title":"Statistics & Probability Letters"},{"issue":"12","key":"2023071211303693000_qkad023-B62","doi-asserted-by":"crossref","first-page":"2609","DOI":"10.1002\/cpa.21783","article-title":"Monte Carlo on manifolds: Sampling densities and integrating functions","volume":"71","author":"Zappa","year":"2018","journal-title":"Communications on Pure and Applied Mathematics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/3\/757\/50859790\/qkad023.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/3\/757\/50859790\/qkad023.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T11:33:23Z","timestamp":1689161603000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/85\/3\/757\/7137399"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,24]]},"references-count":62,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,4,24]]},"published-print":{"date-parts":[[2023,7,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkad023","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,7]]},"published":{"date-parts":[[2023,4,24]]}}}