{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T22:36:07Z","timestamp":1787697367414,"version":"build-2784847793"},"reference-count":52,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:00:00Z","timestamp":1747785600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:00:00Z","timestamp":1747785600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100021745","name":"Institut P\u00e9rim\u00e8tre de physique th\u00e9orique","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100021745","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"crossref","award":["GBMF7392"],"award-info":[{"award-number":["GBMF7392"]}],"id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Science Foundation","award":["NSF PHY-1748958"],"award-info":[{"award-number":["NSF PHY-1748958"]}]},{"DOI":"10.13039\/501100003977","name":"Israel Science Foundation","doi-asserted-by":"crossref","award":["2250\/19"],"award-info":[{"award-number":["2250\/19"]}],"id":[{"id":"10.13039\/501100003977","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Separating relevant and irrelevant information is key to any modeling process or scientific inquiry. Theoretical physics offers a powerful tool for achieving this in the form of the renormalization group (RG). Here we demonstrate a practical approach to performing Wilsonian RG in the context of Gaussian Process (GP) Regression. We systematically integrate out the unlearnable modes of the GP kernel, thereby obtaining an RG flow of the GP in which the data sets the IR scale. In simple cases, this results in a universal flow of the ridge parameter, which becomes input-dependent in the richer scenario in which non-Gaussianities are included. In addition to being analytically tractable, this approach goes beyond structural analogies between RG and neural networks by providing a natural connection between RG flow and learnable vs. unlearnable modes. Studying such flows may improve our understanding of feature learning in deep neural networks, and enable us to identify potential universality classes in these models.<\/jats:p>","DOI":"10.1088\/2632-2153\/adc8fc","type":"journal-article","created":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T22:55:03Z","timestamp":1743720903000},"page":"025038","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Wilsonian renormalization of neural network Gaussian processes<sup>*<\/sup>"],"prefix":"10.1088","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8295-7727","authenticated-orcid":true,"given":"Jessica N","family":"Howard","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2603-0350","authenticated-orcid":false,"given":"Ro","family":"Jefferson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4712-6626","authenticated-orcid":true,"given":"Anindita","family":"Maiti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2555-7695","authenticated-orcid":false,"given":"Zohar","family":"Ringel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2025,5,21]]},"reference":[{"key":"mlstadc8fcbib1","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2022"},{"key":"mlstadc8fcbib2","article-title":"Scaling laws for neural language models","author":"Kaplan","year":"2020"},{"key":"mlstadc8fcbib3","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2311878121","article-title":"Explaining neural scaling laws","volume":"121","author":"Bahri","year":"2024","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstadc8fcbib4","first-page":"pp 1024","article-title":"Spectrum dependent learning curves in kernel regression and wide neural networks","author":"Bordelon","year":"2020"},{"key":"mlstadc8fcbib5","doi-asserted-by":"publisher","first-page":"2914","DOI":"10.1038\/s41467-021-23103-1","article-title":"Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks","volume":"12","author":"Canatar","year":"2021","journal-title":"Nat. Commun."},{"key":"mlstadc8fcbib6","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.11.031059","article-title":"Statistical mechanics of deep linear neural networks: the backpropagating kernel renormalization","volume":"11","author":"Li","year":"2021","journal-title":"Phys. Rev. X"},{"key":"mlstadc8fcbib7","article-title":"Scaling and renormalization in high-dimensional regression","author":"Atanasov","year":"2024"},{"key":"mlstadc8fcbib8","article-title":"Pruning convolutional neural networks for resource efficient inference","author":"Molchanov","year":"2017"},{"key":"mlstadc8fcbib9","author":"Peskin","year":"1995"},{"key":"mlstadc8fcbib10","author":"Cardy","year":"1996"},{"key":"mlstadc8fcbib11","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/0370-1573(74)90023-4","article-title":"The renormalization group and the \u03b5 expansion","volume":"12","author":"Wilson","year":"1974","journal-title":"Phys. Rep."},{"key":"mlstadc8fcbib12","doi-asserted-by":"publisher","first-page":"1343","DOI":"10.1103\/PhysRevLett.30.1343","article-title":"Ultraviolet behavior of non-Abelian gauge theories","volume":"30","author":"Gross","year":"1973","journal-title":"Phys. Rev. Lett."},{"key":"mlstadc8fcbib13","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.1088\/0022-3719\/6\/7\/010","article-title":"Ordering, metastability and phase transitions in two-dimensional systems","volume":"6","author":"Kosterlitz","year":"1973","journal-title":"J. Phys. C: Solid State Phys."},{"key":"mlstadc8fcbib14","article-title":"An exact mapping between the variational renormalization group and deep learning","author":"Mehta","year":"2014"},{"key":"mlstadc8fcbib15","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1038\/s41567-018-0081-4","article-title":"Mutual information, neural networks and the renormalization group","volume":"14","author":"Koch-Janusz","year":"2018","journal-title":"Nat. Phys."},{"key":"mlstadc8fcbib16","article-title":"Deep learning and the renormalization group","author":"B\u00e9ny","year":"2013"},{"key":"mlstadc8fcbib17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.108.025003","article-title":"Renormalization group flow as optimal transport","volume":"108","author":"Cotler","year":"2023","journal-title":"Phys. Rev. D"},{"key":"mlstadc8fcbib18","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ac4f69","article-title":"Non-perturbative renormalization for the neural network-QFT correspondence","volume":"3","author":"Erbin","year":"2022","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc8fcbib19","article-title":"Functional renormalization group for signal detection and stochastic ergodicity breaking","author":"Erbin","year":"2023"},{"key":"mlstadc8fcbib20","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abeca3","article-title":"Neural networks and quantum field theory","volume":"2","author":"Halverson","year":"2021","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc8fcbib21","doi-asserted-by":"publisher","first-page":"081","DOI":"10.21468\/SciPostPhys.12.3.081","article-title":"The edge of chaos: quantum field theory and deep neural networks","volume":"12","author":"Grosvenor","year":"2022","journal-title":"SciPost Phys."},{"key":"mlstadc8fcbib22","doi-asserted-by":"publisher","author":"Roberts","year":"2022","DOI":"10.1017\/9781009023405)"},{"key":"mlstadc8fcbib23","article-title":"On the dynamics of inference and learning","author":"Berman","year":"2022"},{"key":"mlstadc8fcbib24","article-title":"The inverse of exact renormalization group flows as statistical inference","author":"Berman","year":"2022"},{"key":"mlstadc8fcbib25","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad0102","article-title":"Bayesian renormalization","volume":"4","author":"Berman","year":"2023","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc8fcbib26","article-title":"NCoder\u2014a quantum field theory approach to encoding data","author":"Berman","year":"2024"},{"key":"mlstadc8fcbib27","doi-asserted-by":"publisher","first-page":"027","DOI":"10.21468\/SciPostPhysCore.8.1.027","article-title":"Bayesian RG flow in neural network field theories","volume":"8","author":"Howard","year":"2024","journal-title":"SciPost Phys. Core"},{"key":"mlstadc8fcbib28","doi-asserted-by":"publisher","first-page":"462","DOI":"10.1007\/s10955-017-1770-6","article-title":"PCA meets RG","volume":"167","author":"Bradde","year":"2017","journal-title":"J. Stat. Phys."},{"key":"mlstadc8fcbib29","article-title":"Bayesian learning for neural networks","author":"Neal","year":"1995"},{"key":"mlstadc8fcbib30","article-title":"Neural tangent kernel: convergence and generalization in neural networks","volume":"vol 31","author":"Jacot","year":"2018"},{"key":"mlstadc8fcbib31","article-title":"Deep neural networks as Gaussian processes","author":"Lee","year":"2018"},{"key":"mlstadc8fcbib32","article-title":"Gaussian process behaviour in wide deep neural networks","author":"Matthews","year":"2018"},{"key":"mlstadc8fcbib33","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.3.023034","article-title":"Learning curves for overparametrized deep neural networks: a field theory perspective","volume":"3","author":"Cohen","year":"2021","journal-title":"Phys. Rev. Res."},{"key":"mlstadc8fcbib34","doi-asserted-by":"publisher","first-page":"908","DOI":"10.1038\/s41467-023-36361-y","article-title":"Separation of scales and a thermodynamic description of feature learning in some CNNs","volume":"14","author":"Seroussi","year":"2023","journal-title":"Nat. Commun."},{"key":"mlstadc8fcbib35","article-title":"Statistical mechanics of deep learning beyond the infinite-width limit","author":"Ariosto","year":"2022"},{"key":"mlstadc8fcbib36","author":"Rasmussen","year":"2006"},{"key":"mlstadc8fcbib37","article-title":"A variational approach to learning curves","volume":"vol 14","author":"Malzahn","year":"2001"},{"key":"mlstadc8fcbib38","doi-asserted-by":"publisher","first-page":"pp 211","DOI":"10.1007\/11559887_13)","article-title":"Understanding Gaussian process regression using the equivalent kernel","author":"Sollich","year":"2004"},{"key":"mlstadc8fcbib39","doi-asserted-by":"publisher","first-page":"898","DOI":"10.1214\/aos\/1176346710","article-title":"Spline smoothing: the equivalent variable kernel method","volume":"12","author":"Silverman","year":"1984","journal-title":"Ann. Stat."},{"key":"mlstadc8fcbib40","article-title":"Mnist handwritten digit database","volume":"2","author":"LeCun","year":"2010","journal-title":"ATT Labs"},{"key":"mlstadc8fcbib41","author":"Krizhevsky","year":"2009"},{"key":"mlstadc8fcbib42","article-title":"Mnist handwritten digit database.","volume":"2","author":"","year":"2010"},{"key":"mlstadc8fcbib43","article-title":"Neural tangents: fast and easy infinite neural networks in Python","author":"Novak","year":"2020"},{"key":"mlstadc8fcbib44","article-title":"Fast finite width neural tangent kernel","author":"Novak","year":"2022"},{"key":"mlstadc8fcbib45","article-title":"Infinite attention: NNGP and NTK for deep attention networks","author":"Hron","year":"2020"},{"key":"mlstadc8fcbib46","article-title":"On the infinite width limit of neural networks with a standard parameterization","author":"Sohl-Dickstein","year":"2020"},{"key":"mlstadc8fcbib47","article-title":"Fast neural kernel embeddings for general activations","author":"Han","year":"2022"},{"key":"mlstadc8fcbib48","article-title":"The eigenlearning framework: a conservation law perspective on kernel regression and wide neural networks","author":"Simon","year":"2021"},{"key":"mlstadc8fcbib49","article-title":"Exact marginal prior distributions of finite Bayesian neural networks","author":"Zavatone-Veth","year":"2021"},{"key":"mlstadc8fcbib50","article-title":"Failure and success of the spectral bias prediction for kernel ridge regression: the case of low-dimensional data","author":"Tomasini","year":"2022"},{"key":"mlstadc8fcbib51","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.104.064301","article-title":"Predicting the outputs of finite deep neural networks trained with noisy gradients","volume":"104","author":"Naveh","year":"2021","journal-title":"Phys. Rev. E"},{"key":"mlstadc8fcbib52","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/0370-1573(74)90029-5","article-title":"Electrons in disordered systems and the theory of localization","volume":"13","author":"Thouless","year":"1974","journal-title":"Phys. Rep."}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T10:35:57Z","timestamp":1747823757000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc8fc"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,21]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,5,21]]},"published-print":{"date-parts":[[2025,6,30]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/adc8fc","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,21]]},"assertion":[{"value":"Wilsonian renormalization of neural network Gaussian processes*","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-09-26","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-04-03","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-05-21","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}