{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T17:09:27Z","timestamp":1777655367932,"version":"3.51.4"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"11-12","license":[{"start":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T00:00:00Z","timestamp":1727222400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T00:00:00Z","timestamp":1727222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"publisher","award":["19-PI3A-0004"],"award-info":[{"award-number":["19-PI3A-0004"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["19-PI3A-0004"],"award-info":[{"award-number":["19-PI3A-0004"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["20-CHIA-0031-01"],"award-info":[{"award-number":["20-CHIA-0031-01"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["16-IDEX-0004"],"award-info":[{"award-number":["16-IDEX-0004"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100007256","name":"Institut National Polytechnique de Toulouse","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007256","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Normalizing flows (NF) use a continuous generator to map a simple latent (e.g. Gaussian) distribution, towards an empirical target distribution associated with a training data set. Once trained by minimizing a variational objective, the learnt map provides an approximate generative model of the target distribution. Since standard NF implement differentiable maps, they may suffer from pathological behaviors when targeting complex distributions. For instance, such problems may appear for distributions on multi-component topologies or characterized by multiple modes with high probability regions separated by very unlikely areas. A typical symptom is the explosion of the Jacobian norm of the transformation in very low probability areas. This paper proposes to overcome this issue thanks to a new Markov chain Monte Carlo algorithm to sample from the target distribution in the latent domain before transporting it back to the target domain. The approach relies on a Metropolis adjusted Langevin algorithm whose dynamics explicitly exploits the Jacobian of the transformation. Contrary to alternative approaches, the proposed strategy preserves the tractability of the likelihood and it does not require a specific training. Notably, it can be straightforwardly used with any pre-trained NF network, regardless of the architecture. Experiments conducted on synthetic and high-dimensional real data sets illustrate the efficiency of the method.<\/jats:p>","DOI":"10.1007\/s10994-024-06623-x","type":"journal-article","created":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T21:26:33Z","timestamp":1727299593000},"page":"8301-8326","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Normalizing flow sampling with Langevin dynamics in the latent space"],"prefix":"10.1007","volume":"113","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2100-1438","authenticated-orcid":false,"given":"Florentin","family":"Coeurdoux","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicolas","family":"Dobigeon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pierre","family":"Chainais","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,9,25]]},"reference":[{"key":"6623_CR1","unstructured":"Ardizzone, L., Mackowiak, R., Rother, C., & K\u00f6the, U. (2020). Training normalizing flows with the information bottleneck for competitive generative classification. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR2","unstructured":"Arjovsky, M., Chintala, S., & Bottou, L. (2017) Wasserstein generative adversarial networks. In: Precup, D., Teh, Y. W. (Eds.) In Proceedings of international conference on machine learning (ICML). PMLR, Proceedings of Machine Learning Research."},{"key":"6623_CR3","unstructured":"Arvanitidis, G., Hansen, L. K., & Hauberg, S. (2018). Latent space oddity: On the curvature of deep generative models. In Proceedings of IEEE International conference on learning representation (ICLR)."},{"key":"6623_CR4","unstructured":"Behrmann, J., Vicol, P., Wang, K. C., Grosse, R. B., & Jacobsen, J. H. (2019). On the invertibility of invertible neural networks. https:\/\/openreview.net\/forum?id=BJlVeyHFwH."},{"issue":"8","key":"6623_CR5","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","volume":"35","author":"Y Bengio","year":"2013","unstructured":"Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798\u20131828.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"6623_CR6","doi-asserted-by":"crossref","unstructured":"Coeurdoux, F., Dobigeon, N., & Chainais, P. (2022) Sliced-Wasserstein normalizing flows: Beyond maximum likelihood training. In Proceedings of European symposium on artificial neural networks, computational intelligence and machine learning (ESANN), Bruges, Belgium.","DOI":"10.14428\/esann\/2022.ES2022-101"},{"key":"6623_CR7","unstructured":"Cornish, R., Caterini, A., Deligiannidis, G., & Doucet, A. (2020) Relaxing bijectivity constraints with continuously indexed normalising flows. In Proceedings of international conference machine learning (ICML), PMLR, pp. 2133\u20132143."},{"key":"6623_CR8","unstructured":"Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016). Density estimation using real NVP. arXiv preprint arXiv:1605.08803"},{"key":"6623_CR9","doi-asserted-by":"crossref","unstructured":"Dudley, R. M. (2002). Real analysis and probability. Cambridge University Press.","DOI":"10.1017\/CBO9780511755347"},{"key":"6623_CR10","doi-asserted-by":"crossref","unstructured":"Gabri\u00e9, M., Rotskoff, G. M., & Vanden-Eijnden, E. (2022). Adaptive monte carlo augmented with normalizing flows. Proceedings of the National Academy of Sciences (PNAS), 119(10):e2109420119.","DOI":"10.1073\/pnas.2109420119"},{"issue":"2","key":"6623_CR11","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","volume":"73","author":"M Girolami","year":"2011","unstructured":"Girolami, M., & Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 73(2), 123\u2013214.","journal-title":"Journal of the Royal Statistical Society Series B: Statistical Methodology"},{"key":"6623_CR12","unstructured":"Gomez, A. N., Ren, M., Urtasun, R., & Grosse, R. B (2017) The reversible residual network: Backpropagation without storing activations. In Advances in neural information processing systems (NeurIPS)."},{"issue":"11","key":"6623_CR13","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1145\/3422622","volume":"63","author":"I Goodfellow","year":"2020","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Bing, Xu., David, Warde-Farley., Sherjil, Ozair, Aaron, Courville, & Yoshua, Bengio. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139\u2013144.","journal-title":"Communications of the ACM"},{"issue":"4","key":"6623_CR14","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1111\/j.2517-6161.1994.tb02000.x","volume":"56","author":"U Grenander","year":"1994","unstructured":"Grenander, U., & Miller, M. I. (1994). Representations of knowledge in complex systems. Journal of the Royal Statistical Society: Series B (Methodological), 56(4), 549\u2013581.","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"key":"6623_CR15","unstructured":"Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, Vincent., & Courville, A. C. (2017). Improved training of Wasserstein GANs. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR16","doi-asserted-by":"crossref","unstructured":"Hagemann, P., & Neumayer, S. (2021) Stabilizing invertible neural networks using mixture models. Inverse Problems, 37(8), 085002.","DOI":"10.1088\/1361-6420\/abe928"},{"key":"6623_CR17","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, Bernhard., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR18","unstructured":"Ho, J., Jain, A., & Abbeel, P. (2020) Denoising diffusion probabilistic models. In Advances in neural information processing systems (NeurIPS), pp. 6840\u20136851."},{"key":"6623_CR19","unstructured":"Hoffman, M., Sountsov, P., Dillon, J. V., Langmore, I., Tran, D., & Vasudevan, S. (2019). NeuTra-lizing bad geometry in Hamiltonian Monte Carlo using neural transport. In Proceedings of 1st symposium advances in approximate bayesian inference."},{"key":"6623_CR20","unstructured":"Huang, C. W., Dinh, L., & Courville, A. (2020). Augmented normalizing flows: Bridging the gap between generative flows and latent variable models. arXiv preprint arXiv:2002.07101."},{"key":"6623_CR21","unstructured":"Issenhuth, T., Tanielian, U., Mary, J., & Picard, D. (2022) On the optimal precision for GANs. arXiv preprint arXiv:2207.10541"},{"key":"6623_CR22","unstructured":"Izmailov, P., Kirichenko, P., Finzi, M., & Wilson, A. G. (2020) Semi-supervised learning with normalizing flows. In Proceedings international conference machine learning (ICML), PMLR."},{"issue":"3","key":"6623_CR23","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/S0167-7152(97)00020-5","volume":"35","author":"A Justel","year":"1997","unstructured":"Justel, A., Pe\u00f1a, D., & Zamar, R. (1997). A multivariate Kolmogorov\u2013Smirnov test of goodness of fit. Statistics & Probability Letters, 35(3), 251\u2013259.","journal-title":"Statistics & Probability Letters"},{"key":"6623_CR24","unstructured":"Kingma, D. P., & Dhariwal, P. (2018) Glow: Generative flow with invertible $$1\\times 1$$ convolutions. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR25","doi-asserted-by":"crossref","unstructured":"Kraskov, A., St\u00f6gbauer, H., & Grassberger, P. (2004) Estimating mutual information. Physical Review E\u2014Statistical, Nonlinear, and Soft Matter Physics, 69(6):066138.","DOI":"10.1103\/PhysRevE.69.066138"},{"key":"6623_CR26","unstructured":"Krizhevsky, A., Nair, V., & Hinton, G. (2010) Cifar-10 (Canadian Institute for Advanced Research)."},{"key":"6623_CR27","unstructured":"Kumar, A., Sattigeri, P., & Fletcher, T. (2017) Semi-supervised learning with GANs: Manifold invariance with improved inference. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR28","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., & Tang, Xiaoou. (2015) Deep learning face attributes in the wild. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 3730\u20133738.","DOI":"10.1109\/ICCV.2015.425"},{"key":"6623_CR29","first-page":"2","volume":"1","author":"Y Marzouk","year":"2016","unstructured":"Marzouk, Y., Moselhy, T., Parno, M., & Alessio, Spantini. (2016). Sampling via measure transport: An introduction. Handbook of Uncertainty Quantification, 1, 2.","journal-title":"Handbook of Uncertainty Quantification"},{"key":"6623_CR30","doi-asserted-by":"crossref","unstructured":"Milman, E., & Neeman, J. (2022) The Gaussian double-bubble and multi-bubble conjectures. Annals of Mathematics, 195.","DOI":"10.4007\/annals.2022.195.1.2"},{"issue":"5","key":"6623_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3341156","volume":"38","author":"T M\u00fcller","year":"2019","unstructured":"M\u00fcller, T., McWilliams, B., Rousselle, F., Markus, Gross, & Jan, Nov\u00e1k. (2019). Neural importance sampling. ACM Transactions on Graphics (ToG), 38(5), 1\u201319.","journal-title":"ACM Transactions on Graphics (ToG)"},{"key":"6623_CR32","unstructured":"Nalisnick, E., Matsukawa, A., Teh, Y. W., Gorur, D., & Lakshminarayanan, B. (2018). Do deep generative models know what they don\u2019t know? arXiv preprint arXiv:1810.09136"},{"key":"6623_CR33","unstructured":"Nielsen, D., Jaini, P., Hoogeboom, E., Winther, O., & Welling, M. (2020). SurVAE flows: Surjections to bridge the gap between VAEs and flows. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR34","doi-asserted-by":"crossref","unstructured":"No\u00e9, F., Olsson, S., K\u00f6hler, J., & Wu, H. (2019) Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science 365(6457), eaaw1147.","DOI":"10.1126\/science.aaw1147"},{"key":"6623_CR35","doi-asserted-by":"crossref","unstructured":"\u00d8ksendal, B., & \u00d8ksendal, B. (2003). Stochastic differential equations. Springer.","DOI":"10.1007\/978-3-642-14394-6"},{"key":"6623_CR36","unstructured":"Papamakarios, G., Pavlakou, T., & Murray, I. (2017) Masked autoregressive flow for density estimation. In Advances in neural information processing systems (NeurIPS)."},{"issue":"57","key":"6623_CR37","first-page":"1","volume":"22","author":"G Papamakarios","year":"2021","unstructured":"Papamakarios, G., Nalisnick, E. T., Rezende, D. J., Shakir, Mohamed, & Balaji, Lakshminarayanan. (2021). Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57), 1\u201364.","journal-title":"Journal of Machine Learning Research"},{"issue":"6","key":"6623_CR38","doi-asserted-by":"publisher","first-page":"2320","DOI":"10.1214\/11-AAP828","volume":"22","author":"NS Pillai","year":"2012","unstructured":"Pillai, N. S., Stuart, A. M., & Thi\u00e9ry, A. H. (2012). Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions. The Annals of Applied Probability, 22(6), 2320\u20132356.","journal-title":"The Annals of Applied Probability"},{"key":"6623_CR39","unstructured":"Pires, G. G., & Figueiredo, M. A. (2020) Variational mixture of normalizing flows. In Proceedings of European symposium on artificial neural networks, computational intelligence and machine learning (ESANN)."},{"key":"6623_CR40","doi-asserted-by":"crossref","unstructured":"Rifai, S., Mesnil, G., Vincent, P., Muller, X., Bengio, Y., Dauphin, Y., & Glorot, X. (2011) Higher order contractive auto-encoder. In Proceedings of European conference on machine learning and principles and practice of knowledge discovery in databases (ECML-PKDD), Springer.","DOI":"10.1007\/978-3-642-23783-6_41"},{"key":"6623_CR41","doi-asserted-by":"crossref","unstructured":"Runde, V., Ribet, K., & Axler, S. (2005). A taste of topology. Springer.","DOI":"10.1007\/0-387-28387-0"},{"key":"6623_CR42","unstructured":"Samsonov, S., Lagutin, E., Gabri\u00e9, M., Durmus, A., Naumov, A., & Moulines, E. (2022). Local-global MCMC kernels: The best of both worlds. In Advances in neural information processing systems (NeurIPS), pp. 5178\u20135193."},{"key":"6623_CR43","unstructured":"Stimper, V., Sch\u00f6lkopf, B., & Hern\u00e1ndez-Lobato, J. M. (2022) Resampling base distributions of normalizing flows. In Proceedings of international conference on artificial intelligence and statistics (AISTATS), PMLR, pp. 4915\u20134936."},{"issue":"1","key":"6623_CR44","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1137\/20M1371026","volume":"64","author":"M Vono","year":"2022","unstructured":"Vono, M., Dobigeon, N., & Chainais, P. (2022). High-dimensional Gaussian sampling: A review and a unifying approach based on a stochastic proximal point algorithm. SIAM Review, 64(1), 3\u201356.","journal-title":"SIAM Review"},{"key":"6623_CR45","unstructured":"Wu, H., K\u00f6hler, J., & No\u00e9, F. (2020) Stochastic normalizing flows. In Advances in neural information processing systems (NeurIPS)."},{"key":"6623_CR46","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.spl.2014.04.002","volume":"91","author":"T Xifara","year":"2014","unstructured":"Xifara, T., Sherlock, C., Livingstone, S., Simon, Byrne, & Mark, Girolami. (2014). Langevin diffusions and the Metropolis-adjusted Langevin algorithm. Statistics & Probability Letters, 91, 14\u201319.","journal-title":"Statistics & Probability Letters"},{"key":"6623_CR47","unstructured":"Yu, F., Zhang, Y., Song, S., Seff, A., & Xiao, J. (2015) LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06623-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06623-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06623-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T16:05:54Z","timestamp":1735574754000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06623-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,25]]},"references-count":47,"journal-issue":{"issue":"11-12","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["6623"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06623-x","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,25]]},"assertion":[{"value":"20 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 August 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}}]}}