{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T23:04:36Z","timestamp":1771455876341,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:00:00Z","timestamp":1642723200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:00:00Z","timestamp":1642723200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004238","name":"Universit\u00e4t Potsdam","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004238","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Found Comput Math"],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We investigate the application of ensemble transform approaches to Bayesian inference of logistic regression problems. Our approach relies on appropriate extensions of the popular ensemble Kalman filter and the feedback particle filter to the cross entropy loss function and is based on a well-established homotopy approach to Bayesian inference. The arising finite particle evolution equations as well as their mean-field limits are affine-invariant. Furthermore, the proposed methods can be implemented in a gradient-free manner in case of nonlinear logistic regression and the data can be randomly subsampled similar to mini-batching of stochastic gradient descent. We also propose a closely related SDE-based sampling method which again is affine-invariant and can easily be made gradient-free. Numerical examples demonstrate the appropriateness of the proposed methodologies.<\/jats:p>","DOI":"10.1007\/s10208-022-09550-2","type":"journal-article","created":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T20:05:04Z","timestamp":1642795504000},"page":"675-708","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Affine-Invariant Ensemble Transform Methods for Logistic Regression"],"prefix":"10.1007","volume":"23","author":[{"given":"Jakiw","family":"Pidstrigach","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebastian","family":"Reich","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,1,21]]},"reference":[{"key":"9550_CR1","doi-asserted-by":"publisher","unstructured":"Adams, R., Murray, I., MacKay, D.: Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML \u201909, pp. 9\u201316. Association for Computing Machinery, New York, NY, USA (2009). https:\/\/doi.org\/10.1145\/1553374.1553376","DOI":"10.1145\/1553374.1553376"},{"key":"9550_CR2","doi-asserted-by":"publisher","first-page":"995","DOI":"10.1002\/qj.2186","volume":"140","author":"J Amezcua","year":"2014","unstructured":"Amezcua, J., Kalnay, E., Ide, K., Reich, S.: Ensemble transform Kalman\u2013Bucy filters. Q.J.R. Meteor. Soc. 140, 995\u20131004 (2014). https:\/\/doi.org\/10.1002\/qj.2186","journal-title":"Q.J.R. Meteor. Soc."},{"key":"9550_CR3","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1002\/qj.591","volume":"136","author":"K Bergemann","year":"2010","unstructured":"Bergemann, K., Reich, S.: A localization technique for ensemble Kalman filters. Quarterly Journal of the Royal Meteorological Society 136, 701\u2013707 (2010). https:\/\/doi.org\/10.1002\/qj.591","journal-title":"Quarterly Journal of the Royal Meteorological Society"},{"key":"9550_CR4","doi-asserted-by":"publisher","first-page":"1396","DOI":"10.1214\/13-AAP951","volume":"24","author":"A Beskos","year":"2014","unstructured":"Beskos, A., Crisan, D., Jasra, A.: On the stability of sequential Monte Carlo methods in high dimensions. The Annals of Applied Probability 24, 1396\u20131445 (2014). https:\/\/doi.org\/10.1214\/13-AAP951","journal-title":"The Annals of Applied Probability"},{"key":"9550_CR5","volume-title":"Pattern Recognition and Machine Learning","author":"CM Bishop","year":"2006","unstructured":"Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)"},{"key":"9550_CR6","doi-asserted-by":"publisher","unstructured":"Daum, F., Huang, J., Noushin, A.: Exact particle flow for nonlinear filters. In: I.\u00a0Kadar (ed.) Signal Processing, Sensor Fusion, and Target Recognition XIX, vol. 7697, pp. 92\u2013110. International Society for Optics and Photonics, SPIE (2010). https:\/\/doi.org\/10.1117\/12.839590","DOI":"10.1117\/12.839590"},{"key":"9550_CR7","unstructured":"Donner, C., Opper, M.: Efficient Bayesian inference of sigmoidal Gaussian Cox processes. J. Mach. Learn. Research 19(67), 1\u201334 (2018). URL http:\/\/jmlr.org\/papers\/v19\/17-759.html"},{"key":"9550_CR8","doi-asserted-by":"publisher","unstructured":"Evensen, G.: Data Assimilation: The Ensemble Kalman Filter, second edn. Springer, Berlin (2009). https:\/\/doi.org\/10.1007\/978-3-642-03711-5","DOI":"10.1007\/978-3-642-03711-5"},{"issue":"1","key":"9550_CR9","doi-asserted-by":"publisher","first-page":"412","DOI":"10.1137\/19M1251655","volume":"19","author":"A Garbuno-Inigo","year":"2020","unstructured":"Garbuno-Inigo, A., Hoffmann, F., Li, W., Stuart, A.: Interacting Langevin diffusions: Gradient structure and ensemble Kalman sampler. SIAM J. Appl. Dyn. Syst. 19(1), 412\u2013441 (2020). https:\/\/doi.org\/10.1137\/19M1251655","journal-title":"SIAM J. Appl. Dyn. Syst."},{"key":"9550_CR10","doi-asserted-by":"publisher","first-page":"1633","DOI":"10.1137\/19M1304891","volume":"19","author":"A Garbuno-Inigo","year":"2020","unstructured":"Garbuno-Inigo, A., N\u00fcsken, N., Reich, S.: Affine invariant interacting Langevin dynamics for Bayesian inference. SIAM J. Appl. Dyn. Syst. 19, 1633\u20131658 (2020). https:\/\/doi.org\/10.1137\/19M1304891","journal-title":"SIAM J. Appl. Dyn. Syst."},{"key":"9550_CR11","doi-asserted-by":"publisher","first-page":"65","DOI":"10.2140\/camcos.2010.5.65","volume":"5","author":"J Goodman","year":"2010","unstructured":"Goodman, J., Weare, J.: Ensemble samplers with affine invariance. Comm. Appl. Math. and Comput. Science 5, 65\u201380 (2010). https:\/\/doi.org\/10.2140\/camcos.2010.5.65","journal-title":"Comm. Appl. Math. and Comput. Science"},{"key":"9550_CR12","doi-asserted-by":"publisher","first-page":"132911","DOI":"10.1016\/j.physd.2021.132911","volume":"423","author":"G Gottwald","year":"2021","unstructured":"Gottwald, G., Reich, S.: Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation. Physica D 423, 132911 (2021). https:\/\/doi.org\/10.1016\/j.physd.2021.132911","journal-title":"Physica D"},{"key":"9550_CR13","unstructured":"Haber, E., Lucka, F., Ruthotto, L.: Never look back\u2014A modified EnKF method and its application to the training of neutral networks without back propagation (2018). arXiv:1805.08034"},{"key":"9550_CR14","unstructured":"Heber, F., \u017d. Trst\u2019anov\u00e1, Leimkuhler, B.: Posterior sampling strategies based on discretized stochastic differential equations for machine learning applications. Journal of Machine Learning Research 21(228), 1\u201333 (2020). URL http:\/\/jmlr.org\/papers\/v21\/19-339.html"},{"key":"9550_CR15","unstructured":"Huang, D., Schneider, T., Stuart, A.: Uncented Kalman inversion (2021). arXiv:2102.01580"},{"key":"9550_CR16","doi-asserted-by":"publisher","first-page":"773","DOI":"10.1080\/01621459.1995.10476572","volume":"90","author":"R Kass","year":"1995","unstructured":"Kass, R., Raftery, A.: Bayes factors. Journal of the American Statistical Association 90, 773\u2013795 (1995). https:\/\/doi.org\/10.1080\/01621459.1995.10476572","journal-title":"Journal of the American Statistical Association"},{"issue":"9","key":"9550_CR17","doi-asserted-by":"publisher","first-page":"095005","DOI":"10.1088\/1361-6420\/ab1c3a","volume":"35","author":"N Kovachki","year":"2019","unstructured":"Kovachki, N., Stuart, A.: Ensemble Kalman inversion: A derivative-free technique for machine learning tasks. Inverse Probl. 35(9), 095005 (2019). https:\/\/doi.org\/10.1088\/1361-6420\/ab1c3a","journal-title":"Inverse Probl."},{"issue":"12","key":"9550_CR18","doi-asserted-by":"publisher","first-page":"124002","DOI":"10.1088\/1742-5468\/abc62b","volume":"2020","author":"J Lee","year":"2020","unstructured":"Lee, J., Xiao, L., Schoenholz, S., Bahri, Y., Novak, R., Sohl-Dickstein, J., Pennington, J.: Wide neural networks of any depth evolve as linear models under gradient descent. Journal of Statistical Mechanics: Theory and Experiment 2020(12), 124002 (2020). https:\/\/doi.org\/10.1088\/1742-5468\/abc62b","journal-title":"Journal of Statistical Mechanics: Theory and Experiment"},{"key":"9550_CR19","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s11222-017-9730-1","volume":"28","author":"B Leimkuhler","year":"2018","unstructured":"Leimkuhler, B., Matthews, C., Weare, J.: Ensemble preconditioning for Markov chain Monte Carlo simulation. Statistics and Computing 28, 277\u2013290 (2018). https:\/\/doi.org\/10.1007\/s11222-017-9730-1","journal-title":"Statistics and Computing"},{"key":"9550_CR20","unstructured":"Li, Q., Tai, C., E, W.: Stochastic modified equations and dynamics of stochastic gradient algorithms I: Mathematical foundations. Journal of Machine Learning Research 20, 1\u201347 (2019). URL http:\/\/jmlr.org\/papers\/v20\/17-526.html"},{"key":"9550_CR21","doi-asserted-by":"crossref","unstructured":"Liu, F., Huang, X., Chen, Y., Suykens, A.: Random features for kernel approximation: A survey on algorithms, theory and beyond (2020). arXiv:2004.11154","DOI":"10.1109\/TPAMI.2021.3097011"},{"key":"9550_CR22","doi-asserted-by":"publisher","first-page":"619","DOI":"10.1002\/mma.3496","volume":"39","author":"A Nadeem","year":"2016","unstructured":"Nadeem, A., Potthast, R.: Transformed and generalized localization for ensemble methods in data assimilation. Mathematical Methods in the Applied Sciences 39, 619\u2013634 (2016). https:\/\/doi.org\/10.1002\/mma.3496","journal-title":"Mathematical Methods in the Applied Sciences"},{"key":"9550_CR23","doi-asserted-by":"publisher","unstructured":"Opper, M.: A Bayesian approach to online learning. In: D.\u00a0Saad (ed.) Online Learning in Neural Networks, pp. 363\u2013378. Cambridge University Press (1999). https:\/\/doi.org\/10.1017\/CBO9780511569920.017","DOI":"10.1017\/CBO9780511569920.017"},{"key":"9550_CR24","doi-asserted-by":"crossref","unstructured":"Pavliotis, G., Stuart, A., Vaes, U.: Derivative-free Bayesian inversion using multiscale dynamics (2021). arXiv:2102.00540","DOI":"10.1137\/21M1397416"},{"key":"9550_CR25","doi-asserted-by":"publisher","unstructured":"Rabovsky, M., McClelland, J.: Quasi-compositional mapping from form to meaning: A neural network based approach to capturing neural responses during human language comprehension. Phil. Trans. Royal Soc. B: Biological Sciences 375, 20190313 (2019). https:\/\/doi.org\/10.1098\/rstb.2019.0313","DOI":"10.1098\/rstb.2019.0313"},{"key":"9550_CR26","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1007\/s10543-010-0302-4","volume":"51","author":"S Reich","year":"2011","unstructured":"Reich, S.: A dynamical systems framework for intermittent data assimilation. BIT Numer. Math. 51, 235\u2013249 (2011). https:\/\/doi.org\/10.1007\/s10543-010-0302-4","journal-title":"BIT Numer. Math."},{"key":"9550_CR27","doi-asserted-by":"publisher","first-page":"A2013","DOI":"10.1137\/130907367","volume":"35","author":"S Reich","year":"2013","unstructured":"Reich, S.: A nonparametric ensemble transform method for Bayesian inference. SIAM J. Sci. Comput. 35, A2013\u2013A2024 (2013). https:\/\/doi.org\/10.1137\/130907367","journal-title":"SIAM J. Sci. Comput."},{"key":"9550_CR28","doi-asserted-by":"publisher","unstructured":"Reich, S., Cotter, C.: Ensemble filter techniques for intermittent data assimilation. In: M.\u00a0Cullen, F.M. A., S.\u00a0Kindermann, R.\u00a0Scheichl (eds.) Large Scale Inverse Problems. Computational Methods and Applications in the Earth Sciences, Radon Ser. Comput. Appl. Math., vol.\u00a013, pp. 91\u2013134. Walter de Gruyter, Berlin (2013). https:\/\/doi.org\/10.1515\/9783110282269","DOI":"10.1515\/9783110282269"},{"key":"9550_CR29","doi-asserted-by":"publisher","unstructured":"Reich, S., Cotter, C.: Probabilistic forecasting and Bayesian data assimilation. Cambridge University Press, New York (2015). https:\/\/doi.org\/10.1017\/CBO9781107706804","DOI":"10.1017\/CBO9781107706804"},{"key":"9550_CR30","doi-asserted-by":"publisher","unstructured":"Reich, S., Weissmann, S.: Fokker\u2013Planck particle systems for Bayesian inference: Computational approaches. SIAM\/ASA J. Uncertainty Quantification 9, 446\u2013482 (2021). https:\/\/doi.org\/10.1137\/19M1303162","DOI":"10.1137\/19M1303162"},{"key":"9550_CR31","doi-asserted-by":"publisher","first-page":"1988","DOI":"10.1175\/MWR-D-11-00176.1","volume":"140","author":"P Sakov","year":"2012","unstructured":"Sakov, P., Oliver, D., Bertino, L.: An iterative EnKF for strongly nonlinear systems. Monthly Weather Review 140, 1988\u20132004 (2012). https:\/\/doi.org\/10.1175\/MWR-D-11-00176.1","journal-title":"Monthly Weather Review"},{"key":"9550_CR32","unstructured":"Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(56), 1929\u20131958 (2014). URL http:\/\/jmlr.org\/papers\/v15\/srivastava14a.html"},{"key":"9550_CR33","doi-asserted-by":"publisher","unstructured":"Sullivan, T.: Introduction to Uncertainty Quantification, Texts in Applied Mathematics, vol.\u00a063. Springer (2015). https:\/\/doi.org\/10.1007\/978-3-319-23395-6","DOI":"10.1007\/978-3-319-23395-6"},{"issue":"3","key":"9550_CR34","doi-asserted-by":"publisher","first-page":"1090","DOI":"10.1137\/19M124513X","volume":"8","author":"A Taghvaei","year":"2020","unstructured":"Taghvaei, A., Mehta, P., Meyn, S.: Diffusion map-based algorithm for gain function approximation in the feedback particle filter. SIAM\/ASA J. Uncertain. Quantif. 8(3), 1090\u20131117 (2020). https:\/\/doi.org\/10.1137\/19M124513X","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"9550_CR35","doi-asserted-by":"publisher","unstructured":"Taghvaei, A., de\u00a0Wiljes, J., Mehta, P., Reich, S.: Kalman filter and its modern extensions for the continuous-time nonlinear filtering problem. ASME. J. Dyn. Sys., Meas., Control. 140, 030904\u2013030904\u201311 (2017). https:\/\/doi.org\/10.1115\/1.4037780","DOI":"10.1115\/1.4037780"},{"key":"9550_CR36","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1175\/MWR-D-15-0073.1","volume":"144","author":"J T\u00f6dter","year":"2016","unstructured":"T\u00f6dter, J., Kirchgessner, P., Nerger, L., Ahrens, B.: Assessment of a nonlinear ensemble transform filter for high-dimensional data assimilation. Monthly Weather Review 144, 409\u2013427 (2016). https:\/\/doi.org\/10.1175\/MWR-D-15-0073.1","journal-title":"Monthly Weather Review"},{"key":"9550_CR37","doi-asserted-by":"publisher","first-page":"1687","DOI":"10.1137\/20M1344093","volume":"59","author":"C Wormell","year":"2021","unstructured":"Wormell, C., Reich, S.: Spectral convergence of diffusion maps: Improved error bounds and an alternative normalisation. SIAM J. Numer. Anal. 59, 1687\u20131734 (2021). https:\/\/doi.org\/10.1137\/20M1344093","journal-title":"SIAM J. Numer. Anal."},{"key":"9550_CR38","doi-asserted-by":"publisher","unstructured":"Yang, T., Blom, H.A.P., Mehta, P.G.: The Continuous-Discrete Time Feedback Particle Filter. In: 2014 American Control Conference, pp. 648\u2013653. American Automatic Control Council (2014). https:\/\/doi.org\/10.1109\/ACC.2014.6859259","DOI":"10.1109\/ACC.2014.6859259"},{"key":"9550_CR39","doi-asserted-by":"publisher","first-page":"2465","DOI":"10.1109\/TAC.2013.2258825","volume":"58","author":"T Yang","year":"2013","unstructured":"Yang, T., Mehta, P.G., Meyn, S.P.: Feedback particle filter. IEEE Trans. Automat. Control 58, 2465\u20132480 (2013). https:\/\/doi.org\/10.1109\/TAC.2013.2258825","journal-title":"IEEE Trans. Automat. Control"}],"container-title":["Foundations of Computational Mathematics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10208-022-09550-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10208-022-09550-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10208-022-09550-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T15:09:02Z","timestamp":1680102542000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10208-022-09550-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,21]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["9550"],"URL":"https:\/\/doi.org\/10.1007\/s10208-022-09550-2","relation":{},"ISSN":["1615-3375","1615-3383"],"issn-type":[{"value":"1615-3375","type":"print"},{"value":"1615-3383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,21]]},"assertion":[{"value":"16 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}