{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:25:38Z","timestamp":1760171138676,"version":"3.37.3"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2020,2,14]],"date-time":"2020-02-14T00:00:00Z","timestamp":1581638400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,2,14]],"date-time":"2020-02-14T00:00:00Z","timestamp":1581638400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000765","name":"University College London","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000765","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In variational inference (VI), coordinate-ascent and gradient-based approaches are two major types of algorithms for approximating difficult-to-compute probability densities. In real-world implementations of complex models, Monte Carlo methods are widely used to estimate expectations in coordinate-ascent approaches and gradients in derivative-driven ones. We discuss a Monte Carlo co-ordinate ascent VI (MC-CAVI) algorithm that makes use of Markov chain Monte Carlo (MCMC) methods in the calculation of expectations required within co-ordinate ascent VI (CAVI). We show that, under regularity conditions, an MC-CAVI recursion will get arbitrarily close to a maximiser of the evidence lower bound with any given high probability. In numerical examples, the performance of MC-CAVI algorithm is compared with that of MCMC and\u2014as a representative of derivative-based VI methods\u2014of Black Box VI (BBVI). We discuss and demonstrate MC-CAVI\u2019s suitability for models with<jats:italic>hard constraints<\/jats:italic>in simulated and real examples. We compare MC-CAVI\u2019s performance with that of MCMC in an important complex model used in nuclear magnetic resonance spectroscopy data analysis\u2014BBVI is nearly impossible to be employed in this setting due to the hard constraints involved in the model.<\/jats:p>","DOI":"10.1007\/s11222-020-09924-y","type":"journal-article","created":{"date-parts":[[2020,2,14]],"date-time":"2020-02-14T18:03:36Z","timestamp":1581703416000},"page":"887-905","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Monte Carlo co-ordinate ascent variational inference"],"prefix":"10.1007","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1195-2069","authenticated-orcid":false,"given":"Lifeng","family":"Ye","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandros","family":"Beskos","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maria","family":"De Iorio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Hao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,2,14]]},"reference":[{"issue":"500","key":"9924_CR1","doi-asserted-by":"publisher","first-page":"1259","DOI":"10.1080\/01621459.2012.695661","volume":"107","author":"W Astle","year":"2012","unstructured":"Astle, W., De Iorio, M., Richardson, S., Stephens, D., Ebbels, T.: A Bayesian model of NMR spectra for the deconvolution and quantification of metabolites in complex biological mixtures. J. Am. Stat. Assoc. 107(500), 1259\u20131271 (2012)","journal-title":"J. Am. Stat. Assoc."},{"issue":"4","key":"9924_CR2","doi-asserted-by":"crossref","first-page":"2025","DOI":"10.1093\/genetics\/162.4.2025","volume":"162","author":"MA Beaumont","year":"2002","unstructured":"Beaumont, M.A., Zhang, W., Balding, D.J.: Approximate Bayesian computation in population genetics. Genetics 162(4), 2025\u20132035 (2002)","journal-title":"Genetics"},{"key":"9924_CR3","volume-title":"Nonlinear Programming","author":"DP Bertsekas","year":"1999","unstructured":"Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)"},{"key":"9924_CR4","volume-title":"Pattern Recognition and Machine Learning","author":"CM Bishop","year":"2006","unstructured":"Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)"},{"issue":"518","key":"9924_CR5","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","volume":"112","author":"DM Blei","year":"2017","unstructured":"Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112(518), 859\u2013877 (2017)","journal-title":"J. Am. Stat. Assoc."},{"issue":"1","key":"9924_CR6","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1111\/1467-9868.00176","volume":"61","author":"JG Booth","year":"1999","unstructured":"Booth, J.G., Hobert, J.P.: Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc. Ser. B (Statistical Methodology) 61(1), 265\u2013285 (1999)","journal-title":"J. R. Stat. Soc. Ser. B (Statistical Methodology)"},{"key":"9924_CR7","first-page":"421","volume-title":"Stochastic Gradient Descent Tricks","author":"L Bottou","year":"2012","unstructured":"Bottou, L.: Stochastic Gradient Descent Tricks, pp. 421\u2013436. Springer, Berlin (2012)"},{"issue":"1","key":"9924_CR8","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1093\/biomet\/83.1.81","volume":"83","author":"G Casella","year":"1996","unstructured":"Casella, G., Robert, C.P.: Rao\u2013Blackwellisation of sampling schemes. Biometrika 83(1), 81\u201394 (1996)","journal-title":"Biometrika"},{"issue":"429","key":"9924_CR9","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1080\/01621459.1995.10476508","volume":"90","author":"K Chan","year":"1995","unstructured":"Chan, K., Ledolter, J.: Monte Carlo EM estimation for time series models involving counts. J. Am. Stat. Assoc. 90(429), 242\u2013252 (1995)","journal-title":"J. Am. Stat. Assoc."},{"issue":"Jul","key":"9924_CR10","first-page":"2121","volume":"12","author":"J Duchi","year":"2011","unstructured":"Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and Stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121\u20132159 (2011)","journal-title":"J. Mach. Learn. Res."},{"issue":"3","key":"9924_CR11","doi-asserted-by":"publisher","first-page":"824","DOI":"10.1109\/TIP.2006.891045","volume":"16","author":"F Forbes","year":"2007","unstructured":"Forbes, F., Fort, G.: Combining Monte Carlo and mean-field-like methods for inference in hidden Markov random fields. IEEE Trans. Image Process. 16(3), 824\u2013837 (2007)","journal-title":"IEEE Trans. Image Process."},{"issue":"4","key":"9924_CR12","doi-asserted-by":"publisher","first-page":"1220","DOI":"10.1214\/aos\/1059655912","volume":"31","author":"G Fort","year":"2003","unstructured":"Fort, G., Moulines, E., et al.: Convergence of the Monte Carlo expectation maximization for curved exponential families. Ann. Stat. 31(4), 1220\u20131259 (2003)","journal-title":"Ann. Stat."},{"issue":"15","key":"9924_CR13","doi-asserted-by":"publisher","first-page":"2088","DOI":"10.1093\/bioinformatics\/bts308","volume":"28","author":"J Hao","year":"2012","unstructured":"Hao, J., Astle, W., De Iorio, M., Ebbels, T.M.: BATMAN\u2014an R package for the automated quantification of metabolites from nuclear magnetic resonance spectra using a Bayesian model. Bioinformatics 28(15), 2088\u20132090 (2012)","journal-title":"Bioinformatics"},{"issue":"1","key":"9924_CR14","first-page":"1303","volume":"14","author":"MD Hoffman","year":"2013","unstructured":"Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. J. Mach. Learn. Res. 14(1), 1303\u20131347 (2013)","journal-title":"J. Mach. Learn. Res."},{"key":"9924_CR15","doi-asserted-by":"crossref","DOI":"10.1093\/hesc\/9780198703419.001.0001","volume-title":"Nuclear Magnetic Resonance","author":"PJ Hore","year":"2015","unstructured":"Hore, P.J.: Nuclear Magnetic Resonance. Oxford University Press, Oxford (2015)"},{"issue":"2","key":"9924_CR16","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1023\/A:1007665907178","volume":"37","author":"MI Jordan","year":"1999","unstructured":"Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183\u2013233 (1999)","journal-title":"Mach. Learn."},{"issue":"1","key":"9924_CR17","first-page":"430","volume":"18","author":"A Kucukelbir","year":"2017","unstructured":"Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., Blei, D.M.: Automatic differentiation variational inference. J. Mach. Learn. Res. 18(1), 430\u2013474 (2017)","journal-title":"J. Mach. Learn. Res."},{"issue":"3","key":"9924_CR18","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1198\/106186001317115045","volume":"10","author":"RA Levine","year":"2001","unstructured":"Levine, R.A., Casella, G.: Implementations of the Monte Carlo EM algorithm. J. Comput. Graph. Stat. 10(3), 422\u2013439 (2001)","journal-title":"J. Comput. Graph. Stat."},{"key":"9924_CR19","first-page":"814","volume":"33","author":"R Ranganath","year":"2014","unstructured":"Ranganath, R., Gerrish, S., Blei, D.: Black box variational inference. Artif. Intell. Stat. 33, 814\u2013822 (2014)","journal-title":"Artif. Intell. Stat."},{"issue":"3","key":"9924_CR20","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","author":"H Robbins","year":"1951","unstructured":"Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400\u2013407 (1951)","journal-title":"Ann. Math. Stat."},{"key":"9924_CR21","volume-title":"Simulation","author":"SM Ross","year":"2002","unstructured":"Ross, S.M.: Simulation. Elsevier, Amsterdam (2002)"},{"issue":"6","key":"9924_CR22","doi-asserted-by":"publisher","first-page":"1760","DOI":"10.1073\/pnas.0607208104","volume":"104","author":"SA Sisson","year":"2007","unstructured":"Sisson, S.A., Fan, Y., Tanaka, M.M.: Sequential Monte Carlo without likelihoods. Proc. Nat. Acad. Sci. 104(6), 1760\u20131765 (2007)","journal-title":"Proc. Nat. Acad. Sci."},{"issue":"2","key":"9924_CR23","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1080\/10618600.2015.1012293","volume":"25","author":"M-N Tran","year":"2016","unstructured":"Tran, M.-N., Nott, D.J., Kuk, A.Y., Kohn, R.: Parallel variational Bayes for large datasets with an application to generalized linear mixed models. J. Comput. Graph. Stat. 25(2), 626\u2013646 (2016)","journal-title":"J. Comput. Graph. Stat."},{"unstructured":"Tran, M.-N., Nguyen, D.H., Nguyen, D.: Variational Bayes on Manifolds (2019). arXiv:1908.03097","key":"9924_CR24"},{"key":"9924_CR25","volume-title":"Graphical Models, Exponential Families, and Variational Inference","author":"MJ Wainwright","year":"2008","unstructured":"Wainwright, M.J., Jordan, M.I., et al.: Graphical Models, Exponential Families, and Variational Inference, vol. 1. Now Publishers, Inc., Hanover (2008)"},{"issue":"411","key":"9924_CR26","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1080\/01621459.1990.10474930","volume":"85","author":"GC Wei","year":"1990","unstructured":"Wei, G.C., Tanner, M.A.: A Monte Carlo implementation of the EM algorithm and the poor man\u2019s data augmentation algorithms. J. Am. Stat. Assoc. 85(411), 699\u2013704 (1990)","journal-title":"J. Am. Stat. Assoc."},{"issue":"suppl1","key":"9924_CR27","doi-asserted-by":"publisher","first-page":"D521","DOI":"10.1093\/nar\/gkl923","volume":"35","author":"DS Wishart","year":"2007","unstructured":"Wishart, D.S., Tzur, D., Knox, C., Eisner, R., Guo, A.C., Young, N., Cheng, D., Jewell, K., Arndt, D., Sawhney, S., et al.: HMDB: the human metabolome database. Nucl. Acids Res. 35(suppl1), D521\u2013D526 (2007)","journal-title":"Nucl. Acids Res."},{"issue":"suppl1","key":"9924_CR28","first-page":"D603","volume":"37","author":"DS Wishart","year":"2008","unstructured":"Wishart, D.S., Knox, C., Guo, A.C., Eisner, R., Young, N., Gautam, B., Hau, D.D., Psychogios, N., Dong, E., Bouatra, S., et al.: HMDB: a knowledgebase for the human metabolome. Nucl. Acids Res. 37(suppl1), D603\u2013D610 (2008)","journal-title":"Nucl. Acids Res."},{"issue":"1","key":"9924_CR29","doi-asserted-by":"publisher","first-page":"D801","DOI":"10.1093\/nar\/gks1065","volume":"41","author":"DS Wishart","year":"2012","unstructured":"Wishart, D.S., Jewison, T., Guo, A.C., Wilson, M., Knox, C., Liu, Y., Djoumbou, Y., Mandal, R., Aziat, F., Dong, E., et al.: HMDB 3.0\u2014the human metabolome database in 2013. Nucl. Acids Res. 41(1), D801\u2013D807 (2012)","journal-title":"Nucl. Acids Res."},{"issue":"1","key":"9924_CR30","first-page":"D608","volume":"46","author":"DS Wishart","year":"2017","unstructured":"Wishart, D.S., Feunang, Y.D., Marcu, A., Guo, A.C., Liang, K., V\u00e1zquez-Fresno, R., Sajed, T., Johnson, D., Li, C., Karu, N., et al.: HMDB 4.0: the human metabolome database for 2018. Nucl. Acids Res. 46(1), D608\u2013D617 (2017)","journal-title":"Nucl. Acids Res."}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-020-09924-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11222-020-09924-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-020-09924-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T22:49:03Z","timestamp":1695768543000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11222-020-09924-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,14]]},"references-count":30,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,7]]}},"alternative-id":["9924"],"URL":"https:\/\/doi.org\/10.1007\/s11222-020-09924-y","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"type":"print","value":"0960-3174"},{"type":"electronic","value":"1573-1375"}],"subject":[],"published":{"date-parts":[[2020,2,14]]},"assertion":[{"value":"10 May 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 January 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 February 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}