{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:45:22Z","timestamp":1740123922361,"version":"3.37.3"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004702","name":"Universit\u00e0 degli Studi di Genova","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004702","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In Bayesian inverse problems, one aims at characterizing the posterior distribution of a set of unknowns, given indirect measurements. For non-linear\/non-Gaussian problems, analytic solutions are seldom available: Sequential Monte Carlo samplers offer a powerful tool for approximating complex posteriors, by constructing an auxiliary sequence of densities that smoothly reaches the posterior. Often the posterior depends on a scalar hyper-parameter, for which limited prior information is available. In this work, we show that properly designed Sequential Monte Carlo (SMC) samplers naturally provide an approximation of the marginal likelihood associated with this hyper-parameter for free, i.e. at a negligible additional computational cost. The proposed method proceeds by constructing the auxiliary sequence of distributions in such a way that each of them can be interpreted as a posterior distribution corresponding to a different value of the hyper-parameter. This can be exploited to perform selection of the hyper-parameter in <jats:italic>Empirical Bayes<\/jats:italic> (EB) approaches, as well as averaging across values of the hyper-parameter according to some hyper-prior distribution in <jats:italic>Fully Bayesian<\/jats:italic> (FB) approaches. For FB approaches, the proposed method has the further benefit of allowing prior sensitivity analysis at a negligible computational cost. In addition, the proposed method exploits particles at all the (relevant) iterations, thus alleviating one of the known limitations of SMC samplers, i.e. the fact that all samples at intermediate iterations are typically discarded. We show numerical results for two distinct cases where the hyper-parameter affects only the likelihood: a toy example, where an SMC sampler is used to approximate the full posterior distribution; and a brain imaging example, where a Rao-Blackwellized SMC sampler is used to approximate the posterior distribution of a subset of parameters in a conditionally linear Gaussian model.<\/jats:p>","DOI":"10.1007\/s11222-023-10294-4","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T14:03:37Z","timestamp":1695823417000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Cost free hyper-parameter selection\/averaging for Bayesian inverse problems with vanilla and Rao-Blackwellized SMC samplers"],"prefix":"10.1007","volume":"33","author":[{"given":"Alessandro","family":"Viani","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3531-7628","authenticated-orcid":false,"given":"Adam M.","family":"Johansen","sequence":"additional","affiliation":[]},{"given":"Alberto","family":"Sorrentino","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"10294_CR1","unstructured":"Bernton, E, Heng, J, Doucet, A, Jacob, P.\u00a0E.: Schr\u00f6dinger bridge samplers. e-print 1912.13170, ArXiv, (2019)"},{"key":"10294_CR2","doi-asserted-by":"crossref","unstructured":"Chopin, N., Papaspiliopoulos, O.: An introduction to sequential Monte Carlo. Springer, (2020)","DOI":"10.1007\/978-3-030-47845-2"},{"key":"10294_CR3","doi-asserted-by":"crossref","unstructured":"Dau, H.-D., Chopin, N.: Waste-free sequential Monte Carlo. J. R. Stat. Soc. Ser. B Stat Methodol. 84(1), 114\u2013148 (2022)","DOI":"10.1111\/rssb.12475"},{"key":"10294_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-020-09986-y","author":"V De Bortoli","year":"2021","unstructured":"De Bortoli, V., Durmus, A., Pereyra, M., Fernandez Vidal, A.: Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Stat. Comput. (2021). https:\/\/doi.org\/10.1007\/s11222-020-09986-y","journal-title":"Stat. Comput."},{"issue":"3","key":"10294_CR5","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1111\/j.1467-9868.2006.00553.x","volume":"68","author":"P Del Moral","year":"2006","unstructured":"Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. Royal Stat. Soci: Series B (Stat. Methodol.) 68(3), 411\u2013436 (2006)","journal-title":"J. Royal Stat. Soci: Series B (Stat. Methodol.)"},{"issue":"1","key":"10294_CR6","first-page":"34","volume":"8","author":"P Del Moral","year":"2007","unstructured":"Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo for Bayesian computation. Bay. Stat. 8(1), 34 (2007)","journal-title":"Bay. Stat."},{"issue":"5","key":"10294_CR7","doi-asserted-by":"publisher","first-page":"1009","DOI":"10.1007\/s11222-011-9271-y","volume":"22","author":"P Del Moral","year":"2012","unstructured":"Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009\u20131020 (2012)","journal-title":"Stat. Comput."},{"key":"10294_CR8","doi-asserted-by":"crossref","unstructured":"Douc, R., Capp\u00e9, O., Moulines, E.: Comparison of resampling schemes for particle filtering. In: ISPA 2005. Proceedings of the 4th International symposium on image and signal processing and analysis, 2005., pp. 64\u201369. IEEE, (2005)","DOI":"10.1109\/ISPA.2005.195385"},{"key":"10294_CR9","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/s10462-011-9236-8","volume":"38","author":"CW Fox","year":"2012","unstructured":"Fox, C.W., Roberts, Stephen J.: A tutorial on variational Bayesian inference. Artif. Intell. Rev. 38, 85\u201395 (2012)","journal-title":"Artif. Intell. Rev."},{"issue":"4","key":"10294_CR10","first-page":"2236","volume":"37","author":"M Gerber","year":"2019","unstructured":"Gerber, M., Chopin, N., Whiteley, N.: Negative association, ordering and convergence of resampling methods. Ann. Stat. 37(4), 2236\u20132260 (2019)","journal-title":"Ann. Stat."},{"key":"10294_CR11","doi-asserted-by":"publisher","DOI":"10.1201\/b14835","volume-title":"Markov chain Monte Carlo in practice","author":"WR Gilks","year":"1995","unstructured":"Gilks, W.R., Richardson, S., Spiegelhalter, D.: Markov chain Monte Carlo in practice. CRC Press, Cambridge (1995)"},{"key":"10294_CR12","volume-title":"The estimation of probabilities: An essay on modern Bayesian methods","author":"IJ Good","year":"1965","unstructured":"Good, I.J.: The estimation of probabilities: An essay on modern Bayesian methods. MIT Press, Cambridge (1965)"},{"issue":"1","key":"10294_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11222-008-9108-5","volume":"20","author":"R Gramacy","year":"2010","unstructured":"Gramacy, R., Samworth, R., King, R.: Importance tempering. Stat. Comput. 20(1), 1\u20137 (2010)","journal-title":"Stat. Comput."},{"issue":"520","key":"10294_CR14","doi-asserted-by":"publisher","first-page":"1636","DOI":"10.1080\/01621459.2016.1222291","volume":"112","author":"P Guarniero","year":"2017","unstructured":"Guarniero, P., Johansen, A.M., Lee, A.: The iterated auxiliary particle filter. J. Am. Stat. Assoc. 112(520), 1636\u20131647 (2017)","journal-title":"J. Am. Stat. Assoc."},{"key":"10294_CR15","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1103\/RevModPhys.65.413","volume":"65","author":"M H\u00e4m\u00e4l\u00e4inen","year":"1993","unstructured":"H\u00e4m\u00e4l\u00e4inen, M., Hari, R., Ilmoniemi, R.J., Knuutila, J., Lounasmaa, O.V.: Magnetoencephalography\u2013theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. Mod. Phys. 65, 413\u2013497 (1993)","journal-title":"Rev. Mod. Phys."},{"key":"10294_CR16","unstructured":"Kuntz, J., Lim, J.\u00a0N., Johansen, A.\u00a0M.: Particle algorithms for maximum likelihood training of latent variable models. In Proceedings on 26th International Confernece on Artificial Intelligence and Statistics (AISTATS), volume 206 of Proceedings of Machine Learning Research, pages 5134\u20135180, April (2023)"},{"issue":"3","key":"10294_CR17","first-page":"135","volume":"14","author":"CC Drovandi","year":"2019","unstructured":"Drovandi, C.C., South, L.F., Pettitt, A.N.: Sequential Monte Carlo samplers with independent Markov chain Monte Carlo proposals. Bay. Anal. 14(3), 135\u2013143 (2019)","journal-title":"Bay. Anal."},{"key":"10294_CR18","volume-title":"Monte Carlo strategies in scientific computing","author":"JS Liu","year":"2008","unstructured":"Liu, J.S.: Monte Carlo strategies in scientific computing. Springer, Berlin (2008)"},{"key":"10294_CR19","doi-asserted-by":"crossref","unstructured":"Murphy, Kevin, Russell, Stuart: Rao-Blackwellised particle filtering for dynamic Bayesian networks. In Sequential Monte Carlo methods in practice, pp. 499\u2013515. Springer (2001)","DOI":"10.1007\/978-1-4757-3437-9_24"},{"issue":"2","key":"10294_CR20","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1023\/A:1008923215028","volume":"11","author":"RM Neal","year":"2001","unstructured":"Neal, R.M.: Annealed importance sampling. Stat. Comput. 11(2), 125\u2013139 (2001)","journal-title":"Stat. Comput."},{"issue":"5","key":"10294_CR21","doi-asserted-by":"publisher","first-page":"1305","DOI":"10.1109\/TSP.2015.2504342","volume":"64","author":"T Le Thu","year":"2016","unstructured":"Le Thu, T., Nguyen, F.S., Peters, G.W., Delignon, Y.: Efficient sequential Monte Carlo samplers for Bayesian inference. IEEE Trans. Signal Process. 64(5), 1305\u20131319 (2016)","journal-title":"IEEE Trans. Signal Process."},{"issue":"2","key":"10294_CR22","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1080\/10618600.2020.1811105","volume":"30","author":"LJ Rendell","year":"2021","unstructured":"Rendell, L.J., Johansen, A.M., Lee, A., Whiteley, N.: Global consensus Monte Carlo. J. Comput. Graph. Stat. 30(2), 249\u2013259 (2021)","journal-title":"J. Comput. Graph. Stat."},{"issue":"7","key":"10294_CR23","doi-asserted-by":"publisher","first-page":"3452","DOI":"10.1109\/TSP.2011.2140111","volume":"59","author":"B Ristic","year":"2011","unstructured":"Ristic, B., Vo, B.-N., Clark, D., Vo, B.-T.: A metric for performance evaluation of multi-target tracking algorithms. IEEE Trans. Signal Process. 59(7), 3452\u20133457 (2011)","journal-title":"IEEE Trans. Signal Process."},{"issue":"2","key":"10294_CR24","first-page":"78","volume":"11","author":"SL Scott","year":"2016","unstructured":"Scott, S.L., Blocker, A.W., Bonassi, F.V., Chipman, H.A., George, E.I., McCulloch, R.E.: Bayes and big data: the consensus Monte Carlo algorithm. Int. J. Manag. Sci. Eng. Manag. 11(2), 78\u201388 (2016)","journal-title":"Int. J. Manag. Sci. Eng. Manag."},{"key":"10294_CR25","doi-asserted-by":"crossref","unstructured":"Sommariva, S., Sorrentino, A.: Sequential Monte Carlo samplers for semi-linear inverse problems and application to magnetoencephalography. Inverse Prob. 30(11), 114020 (2014)","DOI":"10.1088\/0266-5611\/30\/11\/114020"},{"key":"10294_CR26","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1214\/12-AOAS611","volume":"7","author":"A Sorrentino","year":"2013","unstructured":"Sorrentino, A., Johansen, A.M., Aston, J.A.D., Nichols, T.E., Kendall, W.S.: Dynamic filtering of static dipoles in magnetoencephalography. The Annals Appl. Stat. 7, 955\u2013988 (2013)","journal-title":"The Annals Appl. Stat."},{"issue":"4","key":"10294_CR27","doi-asserted-by":"publisher","first-page":"045010","DOI":"10.1088\/0266-5611\/30\/4\/045010","volume":"30","author":"A Sorrentino","year":"2014","unstructured":"Sorrentino, A., Luria, G., Aramini, R.: Bayesian multi-dipole modelling of a single topography in MEG by adaptive sequential Monte Carlo samplers. Inverse Prob. 30(4), 045010 (2014)","journal-title":"Inverse Prob."},{"key":"10294_CR28","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1017\/S0962492910000061","volume":"19","author":"AM Stuart","year":"2010","unstructured":"Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numerica 19, 451\u2013559 (2010)","journal-title":"Acta Numerica"},{"key":"10294_CR29","unstructured":"Syed, S., Romaniello, V., Campbell, T., Bouchard-C.: Alexandre: parallel tempering on optimized paths. In: International conference on machine learning, pp. 10033\u201310042. PMLR (2021)"},{"key":"10294_CR30","doi-asserted-by":"crossref","unstructured":"Viani, A., Luria, G., Bornfleth, H., Sorrentino, A.: Where Bayes tweaks Gauss: conditionally Gaussian priors for stable multi-dipole estimation. Inverse Probl. Imag, 15(5), (2021)","DOI":"10.3934\/ipi.2021030"},{"issue":"3","key":"10294_CR31","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1080\/10618600.2015.1060885","volume":"25","author":"Y Zhou","year":"2016","unstructured":"Zhou, Y., Johansen, A.M., Aston, J.A.D.: Toward automatic model comparison: an adaptive sequential Monte Carlo approach. J. Comput. Graph. Stat 25(3), 701\u2013726 (2016)","journal-title":"J. Comput. Graph. Stat"}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-023-10294-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11222-023-10294-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-023-10294-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,21]],"date-time":"2023-10-21T10:08:40Z","timestamp":1697882920000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11222-023-10294-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":31,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10294"],"URL":"https:\/\/doi.org\/10.1007\/s11222-023-10294-4","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"type":"print","value":"0960-3174"},{"type":"electronic","value":"1573-1375"}],"subject":[],"published":{"date-parts":[[2023,9,27]]},"assertion":[{"value":"22 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 September 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"126"}}