{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T09:03:03Z","timestamp":1770541383236,"version":"3.49.0"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T00:00:00Z","timestamp":1614729600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T00:00:00Z","timestamp":1614729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004052","name":"King Abdullah University of Science and Technology","doi-asserted-by":"publisher","award":["Baseline"],"award-info":[{"award-number":["Baseline"]}],"id":[{"id":"10.13039\/501100004052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We consider the problem of estimating a parameter<jats:inline-formula><jats:alternatives><jats:tex-math>$$\\theta \\in \\Theta \\subseteq {\\mathbb {R}}^{d_{\\theta }}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><mml:mrow><mml:mi>\u03b8<\/mml:mi><mml:mo>\u2208<\/mml:mo><mml:mi>\u0398<\/mml:mi><mml:mo>\u2286<\/mml:mo><mml:msup><mml:mrow><mml:mi>R<\/mml:mi><\/mml:mrow><mml:msub><mml:mi>d<\/mml:mi><mml:mi>\u03b8<\/mml:mi><\/mml:msub><\/mml:msup><\/mml:mrow><\/mml:math><\/jats:alternatives><\/jats:inline-formula>associated with a Bayesian inverse problem. Typically one must resort to a numerical approximation of gradient of the log-likelihood and also adopt a discretization of the problem in space and\/or time. We develop a new methodology to unbiasedly estimate the gradient of the log-likelihood with respect to the unknown parameter, i.e. the expectation of the estimate has no discretization bias. Such a property is not only useful for estimation in terms of the original stochastic model of interest, but can be used in stochastic gradient algorithms which benefit from unbiased estimates. Under appropriate assumptions, we prove that our estimator is not only unbiased but of finite variance. In addition, when implemented on a single processor, we show that the cost to achieve a given level of error is comparable to multilevel Monte Carlo methods, both practically and theoretically. However, the new algorithm is highly amenable to parallel computation.<\/jats:p>","DOI":"10.1007\/s11222-021-09994-6","type":"journal-article","created":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T08:09:43Z","timestamp":1614758983000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Unbiased estimation of the gradient of the log-likelihood in inverse problems"],"prefix":"10.1007","volume":"31","author":[{"given":"Ajay","family":"Jasra","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3133-2537","authenticated-orcid":false,"given":"Kody J. H.","family":"Law","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deng","family":"Lu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,3]]},"reference":[{"key":"9994_CR1","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.3150\/16-BEJ911","volume":"24","author":"S Agapiou","year":"2018","unstructured":"Agapiou, S., Roberts, G.O., Vollmer, S.: Unbiased Monte Carlo: Posterior estimation for intractable\/infinite-dimensional models. Bernoulli 24, 1726\u20131786 (2018)","journal-title":"Bernoulli"},{"key":"9994_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-75894-2","volume-title":"Adaptive Algorithms and Stochastic Approximation","author":"A Benveniste","year":"1990","unstructured":"Benveniste, A., M\u00e9tivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximation. Springer-Verlag, New York (1990)"},{"key":"9994_CR3","doi-asserted-by":"crossref","unstructured":"Beskos, A., Jasra, A., Law, K.J.H., Marzouk, Y., Zhou, Y.: Multilevel sequential Monte Carlo with dimension-independent likelihood-informed proposals. SIAM\/ASA J. Uncertain. Quantif. 6(2), 762\u2013786 (2017)","DOI":"10.1137\/17M1120993"},{"key":"9994_CR4","doi-asserted-by":"crossref","unstructured":"Beskos, A., Jasra, A., Law, K.J.H., Tempone, R., Zhou, Y.: Multilevel Sequential Monte Carlo samplers. Stoch. Proc. Appl. 127, 1417\u20131440 (2017)","DOI":"10.1016\/j.spa.2016.08.004"},{"key":"9994_CR5","volume-title":"The Mathematical Theory of Finite Element Methods","author":"S Brenner","year":"2007","unstructured":"Brenner, S., Scott, R.: The Mathematical Theory of Finite Element Methods. Springer, New York (2007)"},{"key":"9994_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-28982-8","volume-title":"Inference in Hidden Markov Models","author":"O Capp\u00e9","year":"2005","unstructured":"Capp\u00e9, O., Ryden, T., Moulines, \u00c9.: Inference in Hidden Markov Models. Springer, New York (2005)"},{"key":"9994_CR7","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898719208","volume-title":"The Finite Element Method for Elliptic Problems","author":"PG Ciarlet","year":"2002","unstructured":"Ciarlet, P.G.: The Finite Element Method for Elliptic Problems. SIAM, Philadelphia (2002)"},{"issue":"6","key":"9994_CR8","doi-asserted-by":"publisher","first-page":"2524","DOI":"10.1137\/100814664","volume":"49","author":"M Dashti","year":"2011","unstructured":"Dashti, M., Stuart, A.M.: Uncertainty quantification and weak approximation of an elliptic inverse problem. SIAM J. Numer. Anal. 49(6), 2524\u20132542 (2011)","journal-title":"SIAM J. Numer. Anal."},{"key":"9994_CR9","doi-asserted-by":"publisher","DOI":"10.1201\/b14924","volume-title":"Mean Field Simulation for Monte Carlo Integration","author":"P Del Moral","year":"2013","unstructured":"Del Moral, P.: Mean Field Simulation for Monte Carlo Integration. Chapman & Hall, London (2013)"},{"key":"9994_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-9393-1","volume-title":"Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications","author":"P Del Moral","year":"2004","unstructured":"Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004)"},{"key":"9994_CR11","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1111\/j.1467-9868.2006.00553.x","volume":"68","author":"P Del Moral","year":"2006","unstructured":"Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Statist. Soc. B 68, 411\u2013436 (2006)","journal-title":"J. R. Statist. Soc. B"},{"issue":"3","key":"9994_CR12","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1080\/07362994.2016.1272421","volume":"35","author":"P Del Moral","year":"2017","unstructured":"Del Moral, P., Jasra, A., Law, K.J.H.: Multilevel sequential Monte Carlo: Mean square error bounds under verifiable conditions. Stoch. Anal. Appl. 35(3), 478\u2013498 (2017)","journal-title":"Stoch. Anal. Appl."},{"key":"9994_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-1740-8","volume-title":"Regularization of Inverse Problems","author":"HW Engl","year":"1996","unstructured":"Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Springer, New York (1996)"},{"issue":"3","key":"9994_CR14","doi-asserted-by":"publisher","first-page":"682","DOI":"10.1016\/0022-247X(70)90017-X","volume":"31","author":"JN Franklin","year":"1970","unstructured":"Franklin, J.N.: Well-posed stochastic extensions of ill-posed linear problems. J. Math. Anal. App. 31(3), 682\u2013716 (1970)","journal-title":"J. Math. Anal. App."},{"key":"9994_CR15","unstructured":"Gower, R.M., Loizou, N., Qian, X., Sailanbayev, A., Shulgin, E., Richtarik, P.: SGD: General analysis and improved rates. In: Proceedings of the 36th International Conference on Machine Learning, in PMLR 97, 5200-5209, (2019)"},{"key":"9994_CR16","unstructured":"Jasra, A., Law, K.J.H., Yu, F.: Unbiased filtering of a class of partially observed diffusions. arXiv preprint, (2020)"},{"key":"9994_CR17","volume-title":"Stochastic approximation and recursive algorithms and applications","author":"H Kushner","year":"2003","unstructured":"Kushner, H., Yin, G.G.: Stochastic approximation and recursive algorithms and applications, vol. 35. Springer Science & Business Media, Berlin\/Heidelberg (2003)"},{"key":"9994_CR18","doi-asserted-by":"crossref","unstructured":"Le Gland, F., Mevel, M.: Recursive identification in hidden Markov models. In: Proceedings of the 36th IEEE Conference on Decision and Control, pp. 3468-3473, (1997)","DOI":"10.1109\/CDC.1997.652384"},{"key":"9994_CR19","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1515\/mcma.2011.013","volume":"17","author":"D McLeish","year":"2011","unstructured":"McLeish, D.: A general method for debiasing a Monte Carlo estimator. Monte Carlo Meth. Appl. 17, 301\u2013315 (2011)","journal-title":"Monte Carlo Meth. Appl."},{"key":"9994_CR20","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.1287\/opre.2015.1404","volume":"63","author":"CH Rhee","year":"2015","unstructured":"Rhee, C.H., Glynn, P.: Unbiased estimation with square root convergence for SDE models. Oper. Res. 63, 1026\u20131043 (2015)","journal-title":"Oper. Res."},{"key":"9994_CR21","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1017\/S0962492910000061","volume":"19","author":"AM Stuart","year":"2010","unstructured":"Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451\u2013559 (2010)","journal-title":"Acta Numer."},{"key":"9994_CR22","doi-asserted-by":"crossref","unstructured":"Tadic, V., Doucet, A.: Asymptotic properties of recursive maximum likelihood estimators in non-linear state-space models. arXiv preprint, (2018)","DOI":"10.1109\/ISIT.2019.8849337"},{"issue":"3","key":"9994_CR23","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/0041-5553(64)90254-X","volume":"4","author":"AN Tikhonov","year":"1964","unstructured":"Tikhonov, A.N., Glasko, V.B.: The approximate solution of Fredholm integral equations of the first kind. USSR Comput. Math. Math. Phys. 4(3), 236\u2013247 (1964)","journal-title":"USSR Comput. Math. Math. Phys."},{"key":"9994_CR24","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1287\/opre.2017.1670","volume":"66","author":"M Vihola","year":"2018","unstructured":"Vihola, M.: Unbiased estimators and multilevel Monte Carlo. Oper. Res. 66, 448\u2013462 (2018)","journal-title":"Oper. Res."}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-021-09994-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11222-021-09994-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-021-09994-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T06:23:11Z","timestamp":1724566991000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11222-021-09994-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,3]]},"references-count":24,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["9994"],"URL":"https:\/\/doi.org\/10.1007\/s11222-021-09994-6","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"value":"0960-3174","type":"print"},{"value":"1573-1375","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,3]]},"assertion":[{"value":"10 March 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"21"}}