{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T12:03:28Z","timestamp":1773317008063,"version":"3.50.1"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T00:00:00Z","timestamp":1675123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EPSRC New Investigator","award":["EP\/V002694\/1"],"award-info":[{"award-number":["EP\/V002694\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,2,27]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper introduces a new neural network based prior for real valued functions. Each weight and bias of the neural network has an independent Gaussian prior, with the key novelty that the variances decrease in the width of the network in such a way that the resulting function is well defined in the limit of an infinite width network. We show that the induced posterior over functions is amenable to Monte Carlo sampling using Hilbert space Markov chain Monte Carlo (MCMC) methods. This type of MCMC is stable under mesh refinement, i.e. the acceptance probability does not degenerate as more parameters of the function's prior are introduced, even ad infinitum. We demonstrate these advantages over other function space priors, for example in Bayesian Reinforcement Learning.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkac005","type":"journal-article","created":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T19:31:07Z","timestamp":1675279867000},"page":"46-66","source":"Crossref","is-referenced-by-count":4,"title":["Trace-class Gaussian priors for Bayesian learning of neural networks with MCMC"],"prefix":"10.1093","volume":"85","author":[{"given":"Torben","family":"Sell","sequence":"first","affiliation":[{"name":"School of Mathematics, University of Edinburgh , Edinburgh , UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sumeetpal Sidhu","family":"Singh","sequence":"additional","affiliation":[{"name":"Signal Processing and Communications Laboratory, Department of Engineering, University of Cambridge , Cambridge , UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,1,31]]},"reference":[{"issue":"3","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"1616","DOI":"10.3150\/20-BEJ1285","article-title":"Rates of contraction of posterior distributions based on p-exponential priors","volume":"27","author":"Agapiou","year":"2021","journal-title":"Bernoulli"},{"issue":"10","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"3828","DOI":"10.1016\/j.spa.2013.05.001","article-title":"Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems","volume":"123","author":"Agapiou","year":"2013","journal-title":"Stochastic Processes and Their Applications"},{"issue":"8","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1073\/pnas.38.8.716","article-title":"On the theory of dynamic programming","volume":"38","author":"Bellman","year":"1952","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"2023022804421912900_","volume-title":"Dynamic programming and optimal control","author":"Bertsekas","year":"1995"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1016\/j.jcp.2016.12.041","article-title":"Geometric MCMC for infinite-dimensional inverse problems","volume":"335","author":"Beskos","year":"2017","journal-title":"Journal of Computational Physics"},{"issue":"3","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1142\/S0219493708002378","article-title":"MCMC methods for diffusion bridges","volume":"8","author":"Beskos","year":"2008","journal-title":"Stochastics and Dynamics"},{"key":"2023022804421912900_","author":"Brockman","year":"2016"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","DOI":"10.1201\/b10905","volume-title":"Handbook of Markov chain Monte Carlo","author":"Brooks","year":"2011"},{"issue":"3","key":"2023022804421912900_","first-page":"424","article-title":"MCMC methods for functions: Modifying old algorithms to make them faster","volume":"25","author":"Cotter","year":"2013","journal-title":"Statistical Science"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.jcp.2015.10.008","article-title":"Dimension-independent likelihood-informed MCMC","volume":"304","author":"Cui","year":"2016","journal-title":"Journal of Computational Physics"},{"key":"2023022804421912900_","author":"Damianou","year":"2013"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107295513","volume-title":"Stochastic equations in infinite dimensions","author":"Da Prato","year":"2014"},{"issue":"2","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"183","DOI":"10.3934\/ipi.2012.6.183","article-title":"Besov priors for Bayesian inverse problems","volume":"6","author":"Dashti","year":"2012","journal-title":"Inverse Problems & Imaging"},{"issue":"9","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"095017","DOI":"10.1088\/0266-5611\/29\/9\/095017","article-title":"Map estimators and their consistency in Bayesian nonparametric inverse problems","volume":"29","author":"Dashti","year":"2013","journal-title":"Inverse Problems"},{"key":"2023022804421912900_","first-page":"2100","article-title":"How deep are deep gaussian processes?","volume":"19","author":"Dunlop","year":"2018","journal-title":"The Journal of Machine Learning Research"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1214\/13-AAP926","article-title":"Error bounds for Metropolis-Hastings algorithms applied to perturbations of gaussian measures in high dimensions","volume":"24","author":"Eberle","year":"2014","journal-title":"The Annals of Applied Probability"},{"key":"2023022804421912900_","volume-title":"Geometric measure theory","author":"Federer","year":"1969"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","DOI":"10.1201\/b16018","volume-title":"Bayesian data analysis","author":"Gelman","year":"2013"},{"issue":"2","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1080\/10618600.1992.10477010","article-title":"Numerical computation of multivariate normal probabilities","volume":"1","author":"Genz","year":"1992","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022804421912900_","volume-title":"Mathematical foundations of infinite-dimensional statistical models","author":"Gin\u00e9","year":"2016"},{"issue":"6","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"2455","DOI":"10.1214\/13-AAP982","article-title":"Spectral gaps for a Metropolis-Hastings algorithm in infinite dimensions","volume":"24","author":"Hairer","year":"2014","journal-title":"The Annals of Applied Probability"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/biomet\/57.1.97","article-title":"Monte carlo sampling methods using Markov chains and their applications","volume":"57","author":"Hastings","year":"1970","journal-title":"Biometrika"},{"issue":"2","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/0893-6080(91)90009-T","article-title":"Approximation capabilities of multilayer feedforward networks","volume":"4","author":"Hornik","year":"1991","journal-title":"Neural Networks"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1137\/16M1096372","article-title":"Well-posed Bayesian inverse problems with infinitely divisible and heavy-tailed prior measures","volume":"5","author":"Hosseini","year":"2017","journal-title":"SIAM\/ASA Journal on Uncertainty Quantification"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1137\/16M1076824","article-title":"Well-posed Bayesian inverse problems: Priors with exponential tails","volume":"5","author":"Hosseini","year":"2017","journal-title":"SIAM\/ASA Journal on Uncertainty Quantification"},{"issue":"5","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"851","DOI":"10.1007\/s10596-013-9359-x","article-title":"Evaluation of gaussian approximations for data assimilation in reservoir models","volume":"17","author":"Iglesias","year":"2013","journal-title":"Computational Geosciences"},{"issue":"4","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1093\/imanum\/drn020","article-title":"From high oscillation to rapid approximation III: Multivariate expansions","volume":"29","author":"Iserles","year":"2009","journal-title":"IMA Journal of Numerical Analysis"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"5","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"2626","DOI":"10.1214\/11-AOS920","article-title":"Bayesian inverse problems with gaussian priors","volume":"39","author":"Knapik","year":"2011","journal-title":"The Annals of Statistics"},{"key":"2023022804421912900_","author":"Konidaris","year":"2011"},{"issue":"4","key":"2023022804421912900_","doi-asserted-by":"publisher","first-page":"457","DOI":"10.3934\/fods.2019019","article-title":"Partitioned integrators for thermodynamic parameterization of neural networks","volume":"1","author":"Leimkuhler","year":"2019","journal-title":"Foundations of Data Science"},{"key":"2023022804421912900_","author":"Matthews","year":"2018"},{"issue":"226","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"253","DOI":"10.3189\/2015JoG14J023","article-title":"Early melt season velocity fields of Langj\u00f6kull and Hofsj\u00f6kull, central Iceland","volume":"61","author":"Minchew","year":"2015","journal-title":"Journal of Glaciology"},{"key":"2023022804421912900_","author":"Neal","year":"1995"},{"key":"2023022804421912900_","first-page":"475","article-title":"Regression and classification using gaussian process priors","volume":"6","author":"Neal","year":"1998","journal-title":"Bayesian Statistics"},{"key":"2023022804421912900_","volume-title":"Bayesian learning for neural networks","author":"Neal","year":"2012"},{"issue":"8","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"085001","DOI":"10.1088\/1361-6420\/ab7d2a","article-title":"Consistency of Bayesian inference with gaussian process priors in an elliptic inverse problem","volume":"36","author":"Nickl","year":"2020","journal-title":"Inverse Problems"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1017\/S0962492917000046","article-title":"The cardiovascular system: Mathematical modelling, numerical algorithms and clinical applications","volume":"26","author":"Quarteroni","year":"2017","journal-title":"Acta Numerica"},{"key":"2023022804421912900_","author":"Ramachandran","year":"2007"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1111\/1467-9868.00123","article-title":"Optimal scaling of discrete approximations to Langevin diffusions","volume":"60","author":"Roberts","year":"1998","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"issue":"4","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1214\/ss\/1015346320","article-title":"Optimal scaling for various Metropolis-Hastings algorithms","volume":"16","author":"Roberts","year":"2001","journal-title":"Statistical Science"},{"issue":"4","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"341","DOI":"10.2307\/3318418","article-title":"Exponential convergence of Langevin distributions and their discrete approximations","volume":"2","author":"Roberts","year":"1996","journal-title":"Bernoulli"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/S0893-6080(97)00097-X","article-title":"Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results","volume":"11","author":"Scarselli","year":"1998","journal-title":"Neural Networks"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2414416.2414420","article-title":"Bayesian learning of noisy Markov decision processes","volume":"23","author":"Singh","year":"2013","journal-title":"ACM Transactions on Modeling and Computer Simulation"},{"issue":"4","key":"2023022804421912900_","first-page":"407","article-title":"Sensitivity estimates for nonlinear mathematical models","volume":"1","author":"Sobol","year":"1993","journal-title":"Mathematical Modelling and Computational Experiments"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1017\/S0962492910000061","article-title":"Inverse problems: A Bayesian perspective","volume":"19","author":"Stuart","year":"2010","journal-title":"Acta Numerica"},{"key":"2023022804421912900_","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"2018"},{"issue":"1","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/aoap\/1027961031","article-title":"A note on Metropolis-Hastings kernels for general state spaces","volume":"8","author":"Tierney","year":"1998","journal-title":"The Annals of Applied Probability"},{"key":"2023022804421912900_","author":"Todorov","year":"2012"},{"issue":"3","key":"2023022804421912900_","doi-asserted-by":"crossref","first-page":"1435","DOI":"10.1214\/009053607000000613","article-title":"Rates of contraction of posterior distributions based on gaussian process priors","volume":"36","author":"van der Vaart","year":"2008","journal-title":"The Annals of Statistics"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","DOI":"10.1017\/9781108627771","volume-title":"High-dimensional statistics: A non-asymptotic viewpoint","author":"Wainwright","year":"2019"},{"key":"2023022804421912900_","author":"Welling","year":"2011"},{"key":"2023022804421912900_","author":"Wenzel","year":"2020"},{"key":"2023022804421912900_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511623790","volume-title":"A mathematical introduction to wavelets","author":"Wojtaszczyk","year":"1997"},{"key":"2023022804421912900_","volume-title":"Reinforcement learning: Theory and python implementation","author":"Xiao","year":"2019"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/1\/46\/49347887\/qkac005.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/85\/1\/46\/49347887\/qkac005.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,13]],"date-time":"2024-10-13T12:12:54Z","timestamp":1728821574000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/85\/1\/46\/7017762"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,31]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1,31]]},"published-print":{"date-parts":[[2023,2,27]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkac005","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,2,1]]},"published":{"date-parts":[[2023,1,31]]}}}