{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T22:19:44Z","timestamp":1768515584583,"version":"3.49.0"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2017,8,6]],"date-time":"2017-08-06T00:00:00Z","timestamp":1501977600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,1,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Expectation propagation (EP) is a widely successful algorithm for variational inference. EP is an iterative algorithm used to approximate complicated distributions, typically to find a Gaussian approximation of posterior distributions. In many applications of this type, EP performs extremely well. Surprisingly, despite its widespread use, there are very few theoretical guarantees on Gaussian EP, and it is quite poorly understood. To analyse EP, we first introduce a variant of EP: averaged EP, which operates on a smaller parameter space. We then consider averaged EP and EP in the limit of infinite data, where the overall contribution of each likelihood term is small and where posteriors are almost Gaussian. In this limit, we prove that the iterations of both averaged EP and EP are simple: they behave like iterations of Newton\u2019s algorithm for finding the mode of a function. We use this limit behaviour to prove that EP is asymptotically exact, and to obtain other insights into the dynamic behaviour of EP, e.g. that it may diverge under poor initialization exactly like Newton\u2019s method. EP is a simple algorithm to state, but a difficult one to study. Our results should facilitate further research into the theoretical properties of this important method.<\/jats:p>","DOI":"10.1111\/rssb.12241","type":"journal-article","created":{"date-parts":[[2017,8,7]],"date-time":"2017-08-07T04:22:44Z","timestamp":1502079764000},"page":"199-217","source":"Crossref","is-referenced-by-count":15,"title":["Expectation Propagation in the Large Data Limit"],"prefix":"10.1093","volume":"80","author":[{"given":"Guillaume","family":"Dehaene","sequence":"first","affiliation":[{"name":"Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne and University of Geneva \u00a0 Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simon","family":"Barthelm\u00e9","sequence":"additional","affiliation":[{"name":"Grenoble Images Parole Signal Automatique-Lab \u00a0 France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2017,8,6]]},"reference":[{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1080\/01621459.2013.864178","article-title":"Expectation propagation for likelihood-free inference","volume":"109","author":"Barthelm\u00e9","year":"2014","journal-title":"J. Am. Statist. Ass."},{"key":"2023022005320691300_","volume-title":"Pattern Recognition and Machine Learning","author":"Bishop","year":"2007"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex Optimization","author":"Boyd","year":"2004"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/0001-8708(76)90184-5","article-title":"Best constants in Young\u2019s inequality, its converse, and its generalization to more than three functions","volume":"20","author":"Brascamp","year":"1976","journal-title":"Adv. Math."},{"key":"2023022005320691300_","first-page":"244","volume-title":"Advances in Neural Information Processing Systems","author":"Dehaene","year":"2015"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.jcp.2013.12.010","article-title":"Expectation propagation for nonlinear inverse problems\u2014with an application to electrical impedance tomography","volume":"259","author":"Gehre","year":"2014","journal-title":"J. Computnl Phys."},{"key":"2023022005320691300_","volume-title":"Expectation propagation as a way of life","author":"Gelman","year":"2014"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.jfa.2007.07.019","article-title":"Reinforcement of an inequality due to Brascamp and Lieb","volume":"254","author":"Harg\u00e9","year":"2008","journal-title":"J. Functnl Anal."},{"key":"2023022005320691300_","first-page":"1849","article-title":"Expectation propagation for neural networks with sparsity-promoting priors","volume":"15","author":"Jyl\u00e4nki","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","first-page":"3227","article-title":"Robust Gaussian process regression with a Student-t likelihood","volume":"12","author":"Jyl\u00e4nki","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1214\/12-EJS675","article-title":"The Bernstein-Von-Mises theorem under misspecification","volume":"6","author":"Kleijn","year":"2012","journal-title":"Electron. J. Statist."},{"key":"2023022005320691300_","first-page":"1679","article-title":"Assessing approximate inference for binary Gaussian process classification","volume":"6","author":"Kuss","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","first-page":"2323","volume-title":"Advances in Neural Information Processing Systems","author":"Li","year":"2015"},{"key":"2023022005320691300_","volume-title":"Divergence measures and message passing","author":"Minka","year":"2005"},{"key":"2023022005320691300_","first-page":"362","volume-title":"Proc. 17th Conf. Uncertainty in Artificial Intelligence","author":"Minka","year":"2001"},{"key":"2023022005320691300_","first-page":"2035","article-title":"Approximations for binary Gaussian process classification","volume":"9","author":"Nickisch","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","volume-title":"Numerical Optimization","author":"Nocedal","year":"2006"},{"key":"2023022005320691300_","first-page":"363","volume-title":"On-line Learning in Neural Networks","author":"Opper","year":"1998"},{"key":"2023022005320691300_","first-page":"2177","article-title":"Expectation consistent approximate inference","volume":"6","author":"Opper","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","first-page":"1263","article-title":"Perturbation corrections in approximate inference: mixture modelling applications","volume":"10","author":"Paquet","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"2023022005320691300_","volume-title":"Expectation propagation","author":"Raymond","year":"2014"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1162\/NECO_a_00104","article-title":"Expectation propagation with factorizing distributions: a Gaussian approximation and performance results for simple models","volume":"23","author":"Ribeiro","year":"2011","journal-title":"Neurl Computn"},{"key":"2023022005320691300_","first-page":"658","volume-title":"Advances in Neural Information Processing Systems","author":"Ridgway","year":"2014"},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1111\/j.1467-9868.2008.00700.x","article-title":"Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)","volume":"71","author":"Rue","year":"2009","journal-title":"J. R. Statist. Soc."},{"key":"2023022005320691300_","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1214\/14-SS107","article-title":"Log-concavity and strong log-concavity: a review","volume":"8","author":"Saumard","year":"2014","journal-title":"Statist. Surv."},{"key":"2023022005320691300_","volume-title":"Expectation propagation for exponential families","author":"Seeger","year":"2005"},{"key":"2023022005320691300_","first-page":"298","volume-title":"Proc. 18th Eur. Conf. Machine Learning","author":"Seeger","year":"2007"},{"key":"2023022005320691300_","first-page":"1","volume-title":"Mixtures","author":"Titterington","year":"2011"},{"key":"2023022005320691300_","volume-title":"Graphical Models, Exponential Families, and Variational Inference","author":"Wainwright","year":"2008"},{"key":"2023022005320691300_","first-page":"3356","volume-title":"Advances in Neural Information Processing Systems","author":"Xu","year":"2014"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12241","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/1\/199\/49271420\/jrsssb_80_1_199.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/1\/199\/49271420\/jrsssb_80_1_199.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,26]],"date-time":"2024-06-26T01:00:27Z","timestamp":1719363627000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/80\/1\/199\/7048420"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,6]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2017,8,6]]},"published-print":{"date-parts":[[2018,1,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12241","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,1]]},"published":{"date-parts":[[2017,8,6]]}}}