{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T19:05:57Z","timestamp":1770750357703,"version":"3.50.0"},"reference-count":48,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T00:00:00Z","timestamp":1770681600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Generative diffusion models have emerged as a powerful class of models in machine learning, yet a unified theoretical understanding of their operation is still developing. This paper provides an integrated perspective on generative diffusion by connecting the information-theoretic, dynamical, and thermodynamic aspects. We demonstrate that the rate of conditional entropy production during generation (i.e., the generative bandwidth) is directly governed by the expected divergence of the score function\u2019s vector field. This divergence, in turn, is linked to the branching of trajectories and generative bifurcations, which we characterize as symmetry-breaking phase transitions in the energy landscape. Beyond ensemble averages, we demonstrate that symmetry-breaking decisions are revealed by peaks in the variance of pathwise conditional entropy, capturing heterogeneity in how individual trajectories resolve uncertainty. Together, these results establish generative diffusion as a process of controlled, noise-induced symmetry breaking, in which the score function acts as a dynamic nonlinear filter that regulates both the rate and variability of information flow from noise to data.<\/jats:p>","DOI":"10.3390\/e28020195","type":"journal-article","created":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T11:16:38Z","timestamp":1770722198000},"page":"195","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["The Information Dynamics of Generative Diffusion"],"prefix":"10.3390","volume":"28","author":[{"given":"Dejan","family":"Stan\u010devi\u0107","sequence":"first","affiliation":[{"name":"Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 GD Nijmegen, The Netherlands"}]},{"given":"Luca","family":"Ambrogioni","sequence":"additional","affiliation":[{"name":"Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 GD Nijmegen, The Netherlands"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,10]]},"reference":[{"key":"ref_1","unstructured":"Sohl-Dickstein, J., Weiss, E.A., Maheswaranathan, N., and Ganguli, S. (2015). Deep Unsupervised Learning Using Nonequilibrium Thermodynamics. arXiv."},{"key":"ref_2","unstructured":"Ho, J., Jain, A., and Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems 33, MIT Press."},{"key":"ref_3","unstructured":"Dhariwal, P., and Nichol, A. (2021). Diffusion Models Beat GANs on Image Synthesis. arXiv."},{"key":"ref_4","unstructured":"Kong, Z., Ping, W., Huang, J., Zhao, K., and Catanzaro, B. (2021, January 3\u20137). DiffWave: A Versatile Diffusion Model for Audio Synthesis. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event."},{"key":"ref_5","unstructured":"Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., and Gafni, O. (2022). Make-A-Video: Text-to-Video Generation without Text-Video Data. arXiv."},{"key":"ref_6","unstructured":"Ho, J., and Salimans, T. (2022). Classifier-Free Diffusion Guidance. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. arXiv.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_8","unstructured":"Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., and Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv."},{"key":"ref_9","unstructured":"Song, J., Meng, C., and Ermon, S. (2022). Denoising Diffusion Implicit Models. arXiv."},{"key":"ref_10","unstructured":"Kong, X., Brekelmans, R., and Ver Steeg, G. (2023). Information-Theoretic Diffusion. arXiv."},{"key":"ref_11","unstructured":"Kong, X., Liu, O., Li, H., Yogatama, D., and Ver Steeg, G. (2023). Interpretable Diffusion via Information Decomposition. arXiv."},{"key":"ref_12","unstructured":"Stancevic, D., Handke, F., and Ambrogioni, L. (2025). Entropic Time Schedulers for Generative Diffusion Models. arXiv."},{"key":"ref_13","unstructured":"Dieleman, S., Sartran, L., Roshannai, A., Savinov, N., Ganin, Y., Richemond, P.H., Doucet, A., Strudel, R., Dyer, C., and Durkan, C. (2022). Continuous Diffusion for Categorical Data. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Franzese, G., Martini, M., Corallo, G., Papotti, P., and Michiardi, P. (2025). Latent Abstractions in Generative Diffusion Models. Entropy, 27.","DOI":"10.3390\/e27040371"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Raya, G., and Ambrogioni, L. (2023). Spontaneous Symmetry Breaking in Generative Diffusion Models. arXiv.","DOI":"10.1088\/1742-5468\/ad64bd"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ambrogioni, L. (2024). The Statistical Thermodynamics of Generative Diffusion Models: Phase Transitions, Symmetry Breaking and Critical Instability. arXiv.","DOI":"10.3390\/e27030291"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"093402","DOI":"10.1088\/1742-5468\/acf8ba","article-title":"Generative Diffusion in Very Large Dimensions","volume":"2023","author":"Biroli","year":"2023","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_18","unstructured":"Alaoui, A.E., Montanari, A., and Sellke, M. (2023). Sampling from Mean-Field Gibbs Measures via Diffusion Processes. arXiv."},{"key":"ref_19","unstructured":"Huang, B., Montanari, A., and Pham, H.T. (2024). Sampling from Spherical Spin Glasses in Total Variation via Algorithmic Stochastic Localization. arXiv."},{"key":"ref_20","unstructured":"Montanari, A. (2023). Sampling, Diffusions, and Stochastic Localization. arXiv."},{"key":"ref_21","unstructured":"Benton, J., De Bortoli, V., Doucet, A., and Deligiannidis, G. (2024, January 7\u201311). Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e2408799121","DOI":"10.1073\/pnas.2408799121","article-title":"A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data","volume":"122","author":"Sclocchi","year":"2025","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"084005","DOI":"10.1088\/1742-5468\/aded6c","article-title":"Probing the Latent Hierarchical Structure of Data via Diffusion Models","volume":"2025","author":"Sclocchi","year":"2025","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"9957","DOI":"10.1038\/s41467-024-54281-3","article-title":"Dynamical Regimes of Diffusion Models","volume":"15","author":"Biroli","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_25","unstructured":"Bonnaire, T., Urfin, R., Biroli, G., and M\u00e9zard, M. (2025). Why Diffusion Models Don\u2019t Memorize: The Role of Implicit Dynamical Regularization in Training. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"073401","DOI":"10.1088\/1742-5468\/ade136","article-title":"Memorization and Generalization in Generative Diffusion under the Manifold Hypothesis","volume":"2025","author":"Achilli","year":"2025","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_27","unstructured":"Achilli, B., Ventura, E., Silvestri, G., Pham, B., Raya, G., Krotov, D., Lucibello, C., and Ambrogioni, L. (2024). Losing Dimensions: Geometric Memorization in Generative Diffusion. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ambrogioni, L. (2024). In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks. Entropy, 26.","DOI":"10.3390\/e26050381"},{"key":"ref_29","unstructured":"Hoover, B., Strobelt, H., Krotov, D., Hoffman, J., Kira, Z., and Chau, D.H. (2023). Memory in Plain Sight: A Survey of the Uncanny Resemblances Between Diffusion Models and Associative Memories. Associative Memory & Hopfield Networks in 2023, NeurIPS."},{"key":"ref_30","unstructured":"Hess, J., and Morris, Q. (2025). Associative Memory and Generative Diffusion in the Zero-Noise Limit. arXiv."},{"key":"ref_31","unstructured":"Jeon, D., Kim, D., and No, A. (2024). Understanding Memorization in Generative Models via Sharpness in Probability Landscapes. arXiv."},{"key":"ref_32","unstructured":"Pham, B., Raya, G., Negri, M., Zaki, M.J., Ambrogioni, L., and Krotov, D. (2025). Memorization to Generalization: Emergence of Diffusion Models from Associative Memory. arXiv."},{"key":"ref_33","unstructured":"Premkumar, A. (2024). Neural Entropy. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"040602","DOI":"10.1103\/PhysRevLett.95.040602","article-title":"Entropy production along a stochastic trajectory and an integral fluctuation theorem","volume":"95","author":"Seifert","year":"2005","journal-title":"Phys. Rev. Lett."},{"key":"ref_35","first-page":"031031","article-title":"Speed-accuracy relations for diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transport","volume":"15","author":"Ikeda","year":"2025","journal-title":"Phys. Rev. X"},{"key":"ref_36","unstructured":"Lou, A., Meng, C., and Ermon, S. (2023). Discrete Diffusion Language Modeling by Estimating the Ratios of the Data Distribution. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sahoo, S., Arriola, M., Schiff, Y., Gokaslan, A., Marroquin, E., Chiu, J., Rush, A., and Kuleshov, V. (2024). Simple and Effective Masked Diffusion Language Models. Advances in Neural Information Processing Systems 37, MIT Press.","DOI":"10.52202\/079017-4135"},{"key":"ref_38","unstructured":"Karras, T., Aittala, M., Aila, T., and Laine, S. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. PNIPS\u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems, Curran Associates Inc."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/0304-4149(82)90051-5","article-title":"Reverse-time diffusion equation models","volume":"12","author":"Anderson","year":"1982","journal-title":"Stoch. Process. Their Appl."},{"key":"ref_40","first-page":"695","article-title":"Estimation of Non-Normalized Statistical Models by Score Matching","volume":"6","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1661","DOI":"10.1162\/NECO_a_00142","article-title":"A connection between score matching and denoising autoencoders","volume":"23","author":"Vincent","year":"2011","journal-title":"Neural Comput."},{"key":"ref_42","first-page":"1318","article-title":"Mode-finding for mixtures of Gaussian distributions","volume":"22","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Carreira-Perpin\u00e1n, M.A., and Williams, C.K. (2003). On the number of modes of a Gaussian mixture. Scale Space Methods in Computer Vision, Springer.","DOI":"10.1007\/3-540-44935-3_44"},{"key":"ref_44","first-page":"587","article-title":"Maximum number of modes of Gaussian mixtures","volume":"9","author":"Haase","year":"2020","journal-title":"Inf. Inference J. IMA"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Amari, S.i. (2016). Information Geometry and Its Applications, Springer. Applied Mathematical Sciences (AMS).","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_46","unstructured":"Kadkhodaie, Z., Guth, F., Simoncelli, E.P., and Mallat, S. (2023). Generalization in diffusion models arises from geometry-adaptive harmonic representations. arXiv."},{"key":"ref_47","unstructured":"Stanczuk, J.P., Batzolis, G., Deveney, T., and Sch\u00f6nlieb, C.B. (2024, January 21\u201327). Diffusion Models Encode the Intrinsic Dimension of Data Manifolds. Proceedings of the International Conference on Machine Learning, Vienna, Austria."},{"key":"ref_48","unstructured":"Ventura, E., Achilli, B., Silvestri, G., Lucibello, C., and Ambrogioni, L. (2024). Manifolds, Random Matrices and Spectral Gaps: The Geometric Phases of Generative Diffusion. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/2\/195\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T11:24:34Z","timestamp":1770722674000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/2\/195"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,10]]},"references-count":48,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["e28020195"],"URL":"https:\/\/doi.org\/10.3390\/e28020195","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,10]]}}}