{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T06:34:03Z","timestamp":1775802843898,"version":"3.50.1"},"reference-count":60,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100000271","name":"Science and Technology Facilities Council","doi-asserted-by":"crossref","award":["ST\/X000648\/1"],"award-info":[{"award-number":["ST\/X000648\/1"]}],"id":[{"id":"10.13039\/501100000271","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"crossref","award":["EP\/S023992\/1"],"award-info":[{"award-number":["EP\/S023992\/1"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100022813","name":"Chinese University of Hong Kong, Shenzhen","doi-asserted-by":"crossref","award":["2023TC0179"],"award-info":[{"award-number":["2023TC0179"]}],"id":[{"id":"10.13039\/100022813","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected <jats:italic>n<\/jats:italic>-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory.<\/jats:p>","DOI":"10.1088\/2632-2153\/adc53a","type":"journal-article","created":{"date-parts":[[2025,3,25]],"date-time":"2025-03-25T23:48:32Z","timestamp":1742946512000},"page":"025004","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["On learning higher-order cumulants in diffusion models"],"prefix":"10.1088","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6038-3782","authenticated-orcid":true,"given":"Gert","family":"Aarts","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4942-1516","authenticated-orcid":true,"given":"Diaa E","family":"Habibi","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3757-3403","authenticated-orcid":true,"given":"Lingxiao","family":"Wang","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9859-1758","authenticated-orcid":false,"given":"Kai","family":"Zhou","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2025,4,3]]},"reference":[{"key":"mlstadc53abib1","first-page":"pp 2256","article-title":"Deep unsupervised learning using nonequilibrium thermodynamics","volume":"vol 37","author":"Sohl-Dickstein","year":"2015"},{"key":"mlstadc53abib2","first-page":"pp 6840","article-title":"Denoising diffusion probabilistic models","author":"Ho","year":"2020"},{"key":"mlstadc53abib3","article-title":"Generative modeling by estimating gradients of the data distribution","author":"Song","year":"2020"},{"key":"mlstadc53abib4","article-title":"Score-based generative modeling through stochastic differential equations","author":"Song","year":"2021"},{"key":"mlstadc53abib5","article-title":"Diffusion models: a comprehensive survey of methods and applications","author":"Yang","year":"2022"},{"key":"mlstadc53abib6","first-page":"695","article-title":"Estimation of non-normalized statistical models by score matching","volume":"6","author":"Hyv\u00e4rinen","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"mlstadc53abib7","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR52688.2022.01042","article-title":"High-resolution image synthesis with latent diffusion models","author":"Rombach","year":"2022"},{"key":"mlstadc53abib8","article-title":"Hierarchical text-conditional image generation with CLIP latents","author":"Ramesh","year":"2022"},{"key":"mlstadc53abib9","volume":"vol 15","author":"Smit","year":"2011"},{"key":"mlstadc53abib10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-01850-3)","volume":"vol 788","author":"Gattringer","year":"2010"},{"key":"mlstadc53abib11","doi-asserted-by":"publisher","first-page":"2308","DOI":"10.1103\/PhysRevD.21.2308","article-title":"Monte Carlo study of quantized SU(2) gauge theory","volume":"21","author":"Creutz","year":"1980","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib12","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1016\/0370-2693(87)91197-X","article-title":"Hybrid Monte Carlo","volume":"195","author":"Duane","year":"1987","journal-title":"Phys. Lett. B"},{"key":"mlstadc53abib13","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.nuclphysb.2010.11.020","article-title":"Critical slowing down and error analysis in lattice QCD simulations","volume":"845","author":"(ALPHA Collaboration)","year":"2011","journal-title":"Nucl. Phys. B"},{"key":"mlstadc53abib14","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1038\/s42254-023-00616-w","article-title":"Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics","volume":"5","author":"Cranmer","year":"2023","journal-title":"Nat. Rev. Phys."},{"key":"mlstadc53abib15","first-page":"pp 1530","article-title":"Variational inference with normalizing flows","author":"Rezende","year":"2015"},{"key":"mlstadc53abib16","doi-asserted-by":"publisher","first-page":"eaaw1147","DOI":"10.1126\/science.aaw1147","article-title":"Boltzmann generators: sampling equilibrium states of many-body systems with deep learning","volume":"365","author":"No\u00e9","year":"2019","journal-title":"Science"},{"key":"mlstadc53abib17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.100.034515","article-title":"Flow-based generative models for Markov chain Monte Carlo in lattice field theory","volume":"100","author":"Albergo","year":"2019","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib18","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.101.023304","article-title":"Asymptotically unbiased estimation of physical observables with neural samplers","volume":"101","author":"Nicoli","year":"2020","journal-title":"Phys. Rev. E"},{"key":"mlstadc53abib19","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.125.121601","article-title":"Equivariant flow-based sampling for lattice gauge theory","volume":"125","author":"Kanwar","year":"2020","journal-title":"Phys. Rev. Lett."},{"key":"mlstadc53abib20","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.126.032001","article-title":"Estimation of thermodynamic observables in lattice field theories with deep generative models","volume":"126","author":"Nicoli","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"mlstadc53abib21","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.108.114501","article-title":"Detecting and mitigating mode-collapse for flow-based sampling of lattice field theories","volume":"108","author":"Nicoli","year":"2023","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib22","article-title":"Neural ordinary differential equations","volume":"vol 31","author":"Chen","year":"2018"},{"key":"mlstadc53abib23","article-title":"Scaling up machine learning for quantum field theory with equivariant continuous flows","author":"de Haan","year":"2021"},{"key":"mlstadc53abib24","doi-asserted-by":"publisher","first-page":"238","DOI":"10.21468\/SciPostPhys.15.6.238","article-title":"Learning lattice quantum field theories with equivariant continuous flows","volume":"15","author":"Gerdes","year":"2023","journal-title":"SciPost Phys."},{"key":"mlstadc53abib25","doi-asserted-by":"publisher","first-page":"JHEP02(2024)048","DOI":"10.1007\/JHEP02(2024)048","article-title":"Sampling the lattice Nambu-Goto string using continuous normalizing flows","author":"Caselle","year":"2024","journal-title":"J. High Energy Phys."},{"key":"mlstadc53abib26","first-page":"p 5933","article-title":"Stochastic normalizing flows","volume":"vol 33","author":"Wu","year":"2020"},{"key":"mlstadc53abib27","doi-asserted-by":"publisher","first-page":"JHEP07(2022)015","DOI":"10.1007\/JHEP07(2022)015","article-title":"Stochastic normalizing flows as non-equilibrium transformations","author":"Caselle","year":"2022","journal-title":"J. High Energy Phys."},{"key":"mlstadc53abib28","doi-asserted-by":"publisher","first-page":"JHEP05(2024)060","DOI":"10.1007\/JHEP05(2024)060","article-title":"Diffusion models as stochastic quantization in lattice field theory","author":"Wang","year":"2024","journal-title":"J. High Energy Phys."},{"key":"mlstadc53abib29","article-title":"Generative diffusion models for lattice field theory","volume":"vol 11","author":"Wang","year":"2023"},{"key":"mlstadc53abib30","article-title":"Diffusion models for lattice gauge field simulations","author":"Zhu","year":"2024"},{"key":"mlstadc53abib31","article-title":"NCoder\u2014a quantum field theory approach to encoding data","author":"Berman","year":"2024"},{"key":"mlstadc53abib32","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abeca3","article-title":"Neural networks and quantum field theory","volume":"2","author":"Halverson","year":"2021","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc53abib33","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad17d3","article-title":"Neural network field theories: non-Gaussianity, actions and locality","volume":"5","author":"Demirtas","year":"2024","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc53abib34","article-title":"Bayesian learning for neural networks","author":"Neal","year":"1995"},{"key":"mlstadc53abib35","article-title":"Deep neural networks as Gaussian processes","author":"Lee","year":"2018"},{"key":"mlstadc53abib36","article-title":"Gaussian process behaviour in wide deep neural networks","author":"Matthews","year":"2018"},{"key":"mlstadc53abib37","article-title":"Tensor programs I: wide feedforward or recurrent neural networks of any architecture are Gaussian processes","author":"Yang","year":"2021"},{"key":"mlstadc53abib38","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad81ac","article-title":"Neural network representation of quantum systems","volume":"5","author":"Hashimoto","year":"2024","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc53abib39","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.98.046019","article-title":"Deep learning and the AdS\/CFT correspondence","volume":"98","author":"Hashimoto","year":"2018","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib40","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.103.074510","article-title":"Quantum field-theoretic machine learning","volume":"103","author":"Bachtis","year":"2021","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib41","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.109.034521","article-title":"Scalar field restricted Boltzmann machine as an ultraviolet regulator","volume":"109","author":"Aarts","year":"2024","journal-title":"Phys. Rev. D"},{"key":"mlstadc53abib42","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ac87e9","article-title":"A duality connecting neural network and cosmological dynamics","volume":"3","author":"Krippendorf","year":"2022","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc53abib43","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.111.015303","article-title":"Stochastic weight matrix dynamics during learning and Dyson Brownian motion","volume":"111","author":"Aarts","year":"2025","journal-title":"Phys. Rev. E"},{"key":"mlstadc53abib44","first-page":"483","article-title":"Perturbation theory without gauge fixing","volume":"24","author":"Parisi","year":"1980","journal-title":"Sci. China A"},{"key":"mlstadc53abib45","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/0370-1573(87)90144-X","article-title":"Stochastic quantization","volume":"152","author":"Damgaard","year":"1987","journal-title":"Phys. Rep."},{"key":"mlstadc53abib46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1143\/PTPS.111.1","article-title":"Basic ideas of stochastic quantization","volume":"111","author":"Namiki","year":"1993","journal-title":"Prog. Theor. Phys. Suppl."},{"key":"mlstadc53abib47","article-title":"Understanding diffusion models by Feynman\u2019s path integral","author":"Hirono","year":"2024"},{"key":"mlstadc53abib48","article-title":"Improved techniques for training score-based generative models","author":"Song","year":"2020"},{"key":"mlstadc53abib49","article-title":"Maximum likelihood training of score-based diffusion models","author":"Song","year":"2021"},{"key":"mlstadc53abib50","article-title":"Elucidating the design space of diffusion-based generative models","author":"Karras","year":"2022"},{"key":"mlstadc53abib51","article-title":"Step-by-step diffusion: an elementary tutorial","author":"Nakkiran","year":"2024"},{"key":"mlstadc53abib52","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1080\/00031305.1995.10476146","article-title":"A recursive formulation of the old problem of obtaining moments from cumulants and vice versa","volume":"49","author":"Smith","year":"1995","journal-title":"Am. Stat."},{"key":"mlstadc53abib53","article-title":"Renormalizing diffusion models","author":"Cotler","year":"2023"},{"key":"mlstadc53abib54","doi-asserted-by":"publisher","first-page":"389","DOI":"10.3390\/e26050389","article-title":"The inverse of exact renormalization group flows as statistical inference","volume":"26","author":"Berman","year":"2024","journal-title":"Entropy"},{"key":"mlstadc53abib55","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad0102","article-title":"Bayesian renormalization","volume":"4","author":"Berman","year":"2023","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc53abib56","article-title":"A phase transition in diffusion models reveals the hierarchical nature of data","author":"Sclocchi","year":"2024"},{"key":"mlstadc53abib57","article-title":"On the importance of noise scheduling for diffusion models","author":"Chen","year":"2023"},{"key":"mlstadc53abib58","article-title":"Improved noise schedule for diffusion training","author":"Hang","year":"2024"},{"key":"mlstadc53abib59","article-title":"Speed-accuracy trade-off for the diffusion models: wisdom from nonequilibrium thermodynamics and optimal transport","author":"Ikeda","year":"2024"},{"key":"mlstadc53abib60","article-title":"Fourier features let networks learn high frequency functions in low dimensional domains","author":"Tancik","year":"2020"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T20:47:10Z","timestamp":1743713230000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc53a"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,3]]},"references-count":60,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,4,3]]},"published-print":{"date-parts":[[2025,6,30]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/adc53a","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,3]]},"assertion":[{"value":"On learning higher-order cumulants in diffusion models","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-11-20","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-03-25","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-04-03","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}