{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T03:41:36Z","timestamp":1788838896751,"version":"build-2803163510"},"reference-count":70,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T00:00:00Z","timestamp":1743379200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"project MUSECOM2\u2014AI-enabled MUltimodal SEmantic COMmunications and COMputing, in the Machine Learning-based Communication Systems, towards Wireless AI (WAI), Call 2022, ChistERA","award":["101054746"],"award-info":[{"award-number":["101054746"]}]},{"name":"AdG ELISA project","award":["101054746"],"award-info":[{"award-number":["101054746"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. Our theory is based on a new formulation of joint (state and measurement) dynamics and an information-theoretic measure of state influence on the measurement process. We show that diffusion models can be interpreted as a system of SDE, describing a non-linear filter where unobservable latent abstractions steer the dynamics of an observable measurement process. Additionally, we present an empirical study validating our theory and supporting previous findings on the emergence of latent abstractions at different generative stages.<\/jats:p>","DOI":"10.3390\/e27040371","type":"journal-article","created":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T08:21:41Z","timestamp":1743495701000},"page":"371","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Latent Abstractions in Generative Diffusion Models"],"prefix":"10.3390","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4244-2053","authenticated-orcid":false,"given":"Giulio","family":"Franzese","sequence":"first","affiliation":[{"name":"Department of Data Science, EURECOM, 06410 Biot, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9326-3234","authenticated-orcid":false,"given":"Mattia","family":"Martini","sequence":"additional","affiliation":[{"name":"Laboratoire J. A. Dieudonn\u00e9, CNRS, Universit\u00e9 C\u00f4te d\u2019Azur, 06108 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-5561-5479","authenticated-orcid":false,"given":"Giulio","family":"Corallo","sequence":"additional","affiliation":[{"name":"Department of Data Science, EURECOM, 06410 Biot, France"},{"name":"SAP Labs, 805 Avenue du Dr Donat-Font de l\u2019Orme, 06259 Mougins, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0651-4128","authenticated-orcid":false,"given":"Paolo","family":"Papotti","sequence":"additional","affiliation":[{"name":"Department of Data Science, EURECOM, 06410 Biot, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4675-7677","authenticated-orcid":false,"given":"Pietro","family":"Michiardi","sequence":"additional","affiliation":[{"name":"Department of Data Science, EURECOM, 06410 Biot, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,31]]},"reference":[{"key":"ref_1","first-page":"8780","article-title":"Diffusion Models Beat GANs on Image Synthesis","volume":"Volume 34","author":"Ranzato","year":"2021","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_2","unstructured":"Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D.P., Poole, B., Norouzi, M., and Fleet, D.J. (2025, February 21). Imagen Video: High Definition Video Generation with Diffusion Models. Available online: https:\/\/imagen.research.google\/video\/paper.pdf."},{"key":"ref_3","unstructured":"He, Y., Yang, T., Zhang, Y., Shan, Y., and Chen, Q. (2022). Latent Video Diffusion Models for High-Fidelity Video Generation with Arbitrary Lengths. arXiv."},{"key":"ref_4","unstructured":"Li, X.L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T. (December, January 28). Diffusion-LM Improves Controllable Text Generation. Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA. Oh, A.H., Agarwal, A., Belgrave, D., Cho, K., Eds."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"He, Z., Sun, T., Tang, Q., Wang, K., Huang, X., and Qiu, X. (2023, January 9\u201314). DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, ON, Canada. Long Papers.","DOI":"10.18653\/v1\/2023.acl-long.248"},{"key":"ref_6","unstructured":"Gulrajani, I., and Hashimoto, T. (2023, January 10\u201316). Likelihood-Based Diffusion Language Models. Proceedings of the Thirty-Seventh Conference on Neural Information Processing Systems, New Orleans, LA, USA."},{"key":"ref_7","unstructured":"Lou, A., Meng, C., and Ermon, S. (2024, January 21\u201327). Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. Proceedings of the International Conference on Machine Learning, Vienna, Austria."},{"key":"ref_8","unstructured":"Kong, Z., Ping, W., Huang, J., Zhao, K., and Catanzaro, B. (2021, January 4). DiffWave: A Versatile Diffusion Model for Audio Synthesis. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_9","first-page":"11020","article-title":"DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism","volume":"36","author":"Liu","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_10","unstructured":"Trippe, B.L., Yim, J., Tischer, D., Baker, D., Broderick, T., Barzilay, R., and Jaakkola, T. (2022). Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem. arXiv."},{"key":"ref_11","unstructured":"Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (2022, January 17\u201323). Equivariant Diffusion for Molecule Generation in 3D. Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA. Proceedings of Machine Learning Research."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Luo, S., and Hu, W. (2021, January 20\u201325). Diffusion Probabilistic Models for 3D Point Cloud Generation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00286"},{"key":"ref_13","unstructured":"Zeng, X., Vahdat, A., Williams, F., Gojcic, Z., Litany, O., Fidler, S., and Kreis, K. (December, January 28). LION: Latent Point Diffusion Models for 3D Shape Generation. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA."},{"key":"ref_14","unstructured":"Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., and Poole, B. (2021, January 4). Score-Based Generative Modeling through Stochastic Differential Equations. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_15","first-page":"6840","article-title":"Denoising Diffusion Probabilistic Models","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_16","unstructured":"Albergo, M.S., Boffi, N.M., and Vanden-Eijnden, E. (2023). Stochastic interpolants: A unifying framework for flows and diffusions. arXiv."},{"key":"ref_17","unstructured":"Chen, Y., Vi\u00e9gas, F., and Wattenberg, M. (2023). Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model. arXiv."},{"key":"ref_18","unstructured":"Linhardt, L., Morik, M., Bender, S., and Borras, N.E. (2024). An Analysis of Human Alignment of Latent Diffusion Models. arXiv."},{"key":"ref_19","first-page":"1363","article-title":"Emergent correspondence from image diffusion","volume":"36","author":"Tang","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","unstructured":"Berner, J., Richter, L., and Ullrich, K. (2022). An optimal control perspective on diffusion-based generative modeling. arXiv."},{"key":"ref_21","unstructured":"Richter, L., and Berner, J. (2023). Improved sampling via learned diffusions. arXiv."},{"key":"ref_22","first-page":"27280","article-title":"First hitting diffusion models for generating manifold, graph and categorical data","volume":"35","author":"Ye","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Raginsky, M. (2024). A variational approach to sampling in diffusion processes. arXiv.","DOI":"10.1109\/CDC56724.2024.10886629"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bain, A., and Crisan, D. (2009). Fundamentals of Stochastic Filtering, Springer.","DOI":"10.1007\/978-0-387-76896-0"},{"key":"ref_25","unstructured":"Van Handel, R. (2007). Filtering, Stability, and Robustness. [Ph.D. Thesis, California Institute of Technology]."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"102307","DOI":"10.1016\/j.jmp.2019.102307","article-title":"The Hitchhiker\u2019s guide to nonlinear filtering","volume":"94","author":"Kutschireiter","year":"2020","journal-title":"J. Math. Psychol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., and Nisnevich, A. (2020). Experience grounds language. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-main.703"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bender, E.M., and Koller, A. (2020, January 5\u201310). Climbing towards NLU: On meaning, form, and understanding in the age of data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.463"},{"key":"ref_29","unstructured":"Li, K., Hopkins, A.K., Bau, D., Vi\u00e9gas, F., Pfister, H., and Wattenberg, M. (2022). Emergent world representations: Exploring a sequence model trained on a synthetic task. arXiv."},{"key":"ref_30","first-page":"24129","article-title":"Understanding the latent space of diffusion models through the lens of riemannian geometry","volume":"36","author":"Park","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_31","unstructured":"Kwon, M., Jeong, J., and Uh, Y. (2023). Diffusion Models already have a Semantic Latent Space. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xiang, W., Yang, H., Huang, D., and Wang, Y. (2023, January 2\u20133). Denoising diffusion autoencoders are unified self-supervised learners. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.01448"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Haas, R., Huberman-Spiegelglas, I., Mulayoff, R., Gra\u00dfhof, S., Brandt, S.S., and Michaeli, T. (2024). Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models. arXiv.","DOI":"10.1109\/FG59268.2024.10581912"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sclocchi, A., Favero, A., and Wyart, M. (2024). A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data. arXiv.","DOI":"10.1073\/pnas.2408799121"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"104025","DOI":"10.1088\/1742-5468\/ad64bd","article-title":"Spontaneous symmetry breaking in generative diffusion models","volume":"2024","author":"Raya","year":"2024","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_36","unstructured":"Ambrogioni, L. (2023). The statistical thermodynamics of generative diffusion models. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1007\/s10955-008-9622-z","article-title":"Interactive statistical mechanics and nonlinear filtering","volume":"133","author":"Newton","year":"2008","journal-title":"J. Stat. Phys."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1137\/0119020","article-title":"On the calculation of mutual information","volume":"19","author":"Duncan","year":"1970","journal-title":"SIAM J. Appl. Math."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/S0019-9958(71)90135-5","article-title":"Mutual information for stochastic differential equations","volume":"19","author":"Duncan","year":"1971","journal-title":"Inf. Control"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1813","DOI":"10.1137\/S0363012901393894","article-title":"A variational approach to nonlinear estimation","volume":"42","author":"Mitter","year":"2003","journal-title":"SIAM J. Control Optim."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"ksendal, B. (2003). Stochastic Differential Equations, Springer.","DOI":"10.1007\/978-3-642-14394-6"},{"key":"ref_42","first-page":"1","article-title":"Nonlinear filtering with degenerate noise","volume":"22","author":"Pardoux","year":"2017","journal-title":"Electron. Commun. Probabability"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/0304-4149(82)90051-5","article-title":"Reverse-time diffusion equation models","volume":"12","author":"Anderson","year":"1982","journal-title":"Stoch. Process. Their Appl."},{"key":"ref_44","unstructured":"Pardoux, E. (2006). Grossissement d\u2019une filtration et retournement du temps d\u2019une diffusion. S\u00e9minaire de Probabilit\u00e9s XX 1984\/85: Proceedings, Springer."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rogers, L.C.G., and Williams, D. (2000). Diffusions, Markov Processes, and Martingales: It\u00f4 Calculus, Cambridge University Press.","DOI":"10.1017\/CBO9781107590120"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Aksamit, A., and Jeanblanc, M. (2017). Enlargement of Filtration with Finance in View, Springer.","DOI":"10.1007\/978-3-319-41255-9"},{"key":"ref_47","unstructured":"Ouwehand, P. (2022). Enlargement of Filtrations\u2014A Primer. arXiv."},{"key":"ref_48","unstructured":"Grigorian, K., and Jarrow, R.A. (2023). Enlargement of Filtrations: An Exposition of Core Ideas with Financial Examples. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.spa.2015.09.015","article-title":"Markov bridges: SDE representation","volume":"126","author":"Danilova","year":"2016","journal-title":"Stoch. Process. Their Appl."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1214\/aop\/1176992362","article-title":"Time Reversal of Diffusions","volume":"14","author":"Haussmann","year":"1986","journal-title":"Ann. Probab."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Shreve, S.E. (2004). Stochastic Calculus for Finance II: Continuous-Time Models, Springer.","DOI":"10.1007\/978-1-4757-4296-1"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/0304-4149(87)90031-7","article-title":"Enlarged filtrations for diffusions","volume":"24","author":"Elliott","year":"1987","journal-title":"Stoch. Process. Their Appl."},{"key":"ref_53","unstructured":"Franzese, G., Bounoua, M., and Michiardi, P. (2024, January 7\u201311). MINDE: Mutual Information Neural Diffusion Estimation. Proceedings of the The Twelfth International Conference on Learning Representations, Vienna Austria."},{"key":"ref_54","unstructured":"Kidger, P., Foster, J., Li, X., and Lyons, T.J. (2021, January 18\u201324). Neural sdes as infinite-dimensional gans. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_55","first-page":"17695","article-title":"Diffusion schr\u00f6dinger bridge with applications to score-based generative modeling","volume":"34","author":"Thornton","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_56","unstructured":"Hodgkinson, L., van der Heide, C., Roosta, F., and Mahoney, M.W. (2021, January 27\u201330). Stochastic continuous normalizing flows: Training SDEs as ODEs. Proceedings of the Uncertainty in Artificial Intelligence, Online."},{"key":"ref_57","unstructured":"Kim, H., and Mnih, A. (2018, January 10\u201315). Disentangling by factorising. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Sechidis, K., Tsoumakas, G., and Vlahavas, I. (2011). On the stratification of multi-label data. Machine Learning and Knowledge Discovery in Databases, Springer.","DOI":"10.1007\/978-3-642-23808-6_10"},{"key":"ref_60","unstructured":"Torgo, L., Krawczyk, B., Branco, P., and Moniz, N. A Network Perspective on Stratification of Multi-Label Data. Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications, Proceedings of Machine Learning Research."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Xiong, J. (2008). An Introduction to Stochastic Filtering Theory, Oxford University Press.","DOI":"10.1093\/oso\/9780199219704.001.0001"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Kallenberg, O. (2002). Foundations of Modern Probability, Springer. [2nd ed.]. Probability and its Applications (New York).","DOI":"10.1007\/978-1-4757-4015-8"},{"key":"ref_64","first-page":"934","article-title":"Invariance properties of the conditional independence relation","volume":"13","year":"1985","journal-title":"Ann. Probab."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"093302","DOI":"10.1063\/1.5000077","article-title":"Constraint ornstein-uhlenbeck bridges","volume":"58","author":"Mazzolo","year":"2017","journal-title":"J. Math. Phys."},{"key":"ref_66","unstructured":"Corlay, S. (2013). Properties of the Ornstein-Uhlenbeck bridge. arXiv."},{"key":"ref_67","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_68","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations (ICLR), San Diega, CA, USA."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7\u201313). Deep Learning Face Attributes in the Wild. Proceedings of the Proceedings of International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.425"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Lingenfelter, B., Davis, S.R., and Hand, E.M. (2022, January 3\u20135). A quantitative analysis of labeling issues in the CelebA dataset. Proceedings of the Advances in Visual Computing: 17th International Symposium, ISVC 2022, San Diego, CA, USA. Proceedings, Part I.","DOI":"10.1007\/978-3-031-20713-6_10"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/4\/371\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:06:58Z","timestamp":1760029618000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/4\/371"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,31]]},"references-count":70,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["e27040371"],"URL":"https:\/\/doi.org\/10.3390\/e27040371","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,31]]}}}