{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T07:38:18Z","timestamp":1776757098615,"version":"3.51.2"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2021,12,28]],"date-time":"2021-12-28T00:00:00Z","timestamp":1640649600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,11,29]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>We introduce a new online algorithm for expected loglikelihood maximization in situations where the objective function is multimodal or has saddle points. The key element underpinning the algorithm is a probability distribution that concentrates on the target parameter value as the sample size increases and can be efficiently estimated by means of a standard particle filter algorithm. This distribution depends on a learning rate, such that the faster the learning rate the quicker the distribution concentrates on the desired element of the search space, but the less likely the algorithm is to escape from a local optimum of the objective function. In order to achieve a fast convergence rate with a slow learning rate, our algorithm exploits the acceleration property of averaging, which is well known from the stochastic gradient literature. Considering several challenging estimation problems, our numerical experiments show that with high probability, the algorithm successfully finds the highest mode of the objective function and converges to the global maximizer at the optimal rate. While the focus of this work is expected loglikelihood maximization, the proposed methodology and its theory apply more generally to optimization of a function defined through an expectation.<\/jats:p>","DOI":"10.1093\/biomet\/asab067","type":"journal-article","created":{"date-parts":[[2021,12,26]],"date-time":"2021-12-26T12:06:08Z","timestamp":1640520368000},"page":"937-955","source":"Crossref","is-referenced-by-count":2,"title":["A global stochastic optimization particle filter algorithm"],"prefix":"10.1093","volume":"109","author":[{"given":"M","family":"Gerber","sequence":"first","affiliation":[{"name":"School of Mathematics, University of Bristol, Woodland Road , Bristol BS8 1UG, U.K"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R","family":"Douc","sequence":"additional","affiliation":[{"name":"Institut Polytechnique de Paris SAMOVAR, Tel\u00e9com SudParis, , 9 rue Charles Fourier, 91000 Evry, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,12,28]]},"reference":[{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1007\/s11222-020-09964-4","article-title":"Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization","volume":"30","author":"Akyildiz,","year":"2020","journal-title":"Statist. Comp."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1214\/06-BA112","article-title":"A one-pass sequential Monte Carlo method for Bayesian analysis of massive datasets","volume":"1","author":"Balakrishnan,","year":"2006","journal-title":"Bayesian Anal."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-47845-2","volume-title":"An Introduction to Sequential Monte Carlo.","author":"Chopin,","year":"2020"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"3039","DOI":"10.3150\/17-BEJ954","article-title":"Nested particle filters for online parameter estimation in discrete-time state-space Markov models","volume":"24","author":"Crisan,","year":"2018","journal-title":"Bernoulli"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.1016\/j.csda.2011.03.019","article-title":"Likelihood-free Bayesian estimation of multivariate quantile distributions","volume":"55","author":"Drovandi,","year":"2011","journal-title":"Comp. Statist. Data Anal."},{"key":"2023022720061809600_","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi,","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1137\/0329055","article-title":"Recursive stochastic algorithms for global optimization in $\\mathbb{R}^d$","volume":"29","author":"Gelfand,","year":"1991","journal-title":"SIAM J. Contr. Optimiz."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"2236","DOI":"10.1214\/18-AOS1746","article-title":"Negative association, ordering and convergence of resampling methods","volume":"47","author":"Gerber,","year":"2019","journal-title":"Ann. Statist."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"3103","DOI":"10.1214\/21-AOS2076","article-title":"Online inference with multi-modal likelihood functions","volume":"49","author":"Gerber,","year":"2021","journal-title":"Ann. Statist."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1007\/978-3-642-41095-6_17","article-title":"On the convergence of quantum and sequential Monte Carlo methods","volume-title":"Monte Carlo and Quasi-Monte Carlo Methods 2012.","author":"Giraud,","year":"2013"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"670","DOI":"10.3150\/14-BEJ680","article-title":"Nonasymptotic analysis of adaptive and annealed Feynman\u2013Kac particle models","volume":"23","author":"Giraud,","year":"2017","journal-title":"Bernoulli"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1080\/10618600.2000.10474866","article-title":"Quantile regression via an MM algorithm","volume":"9","author":"Hunter,","year":"2000","journal-title":"J. Comp. Graph. Statist."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1214\/12-EJS675","article-title":"The Bernstein-von-Mises theorem under misspecification","volume":"6","author":"Kleijn,","year":"2012","journal-title":"Electron. J. Statist."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"105486","DOI":"10.1016\/j.knosys.2020.105486","article-title":"Particle filtering methods for stochastic optimization with application to large-scale empirical risk minimization","volume":"193","author":"Liu,","year":"2020","journal-title":"Knowledge-Based Syst."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/978-3-319-41000-5_10","article-title":"Particle filter optimization: A brief introduction","volume-title":"Advances in Swarm Intelligence.","author":"Liu,","year":"2016"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/978-1-4757-3437-9_10","article-title":"Combined parameter and state estimation in simulation-based filtering","volume-title":"Sequential Monte Carlo Methods in Practice.","author":"Liu,","year":"2001"},{"key":"2023022720061809600_","article-title":"Sgdr: Stochastic gradient descent with warm restarts","author":"Loshchilov,","year":"2017"},{"key":"2023022720061809600_","volume-title":"Frontiers in Massive Data Analysis.","year":"2013"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1214\/aoap\/1027961032","article-title":"Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing","volume":"8","author":"Pelletier,","year":"1998","journal-title":"Ann. Appl. Prob."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"838","DOI":"10.1137\/0330046","article-title":"Acceleration of stochastic approximation by averaging","volume":"30","author":"Polyak,","year":"1992","journal-title":"SIAM J. Contr. Optimiz."},{"key":"2023022720061809600_","article-title":"gk: An R package for the g-and-k and generalised g-and-h distributions","author":"Prangle,","year":"2017"},{"key":"2023022720061809600_","first-page":"3743","article-title":"Stochastic global optimization: Stopping rules","volume-title":"Encyclopedia of Optimization,","author":"Schoen,","year":"2009"},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.1016\/j.spa.2014.11.001","article-title":"Convergence and convergence rate of stochastic gradient search in the case of multiple and non-isolated extrema","volume":"125","author":"Tadi\u0107,","year":"2015","journal-title":"Stoch. Proces. Appl."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1007\/s11222-015-9560-y","article-title":"Scalable estimation strategies based on stochastic approximations: Classical results and new insights","volume":"25","author":"Toulis,","year":"2015","journal-title":"Statist. Comp."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.csda.2018.03.010","article-title":"Objective priors for the number of degrees of freedom of a multivariate $t$ distribution and the $t$-copula","volume":"124","author":"Villa,","year":"2018","journal-title":"Comp. Statist. Data Anal."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.jeconom.2009.05.004","article-title":"Regression density estimation using smooth adaptive Gaussian mixtures","volume":"153","author":"Villani,","year":"2009","journal-title":"J. Economet."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1137\/S1052623497319225","article-title":"Rates of convergence for a class of global stochastic optimization algorithms","volume":"10","author":"Yin,","year":"1999","journal-title":"SIAM J. Optimiz."},{"key":"2023022720061809600_","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1109\/WSC.2008.4736125","article-title":"A particle filtering framework for randomized optimization algorithms","volume-title":"Proc. 2008 Winter Simulation Conference.","author":"Zhou,","year":"2008"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asab067\/43479150\/asab067.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/937\/49346789\/asab067.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/109\/4\/937\/49346789\/asab067.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,15]],"date-time":"2024-09-15T05:53:34Z","timestamp":1726379614000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/109\/4\/937\/6486527"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,28]]},"references-count":28,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,12,28]]},"published-print":{"date-parts":[[2022,11,29]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asab067","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,12,1]]},"published":{"date-parts":[[2021,12,28]]}}}