{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T21:36:52Z","timestamp":1793396212828,"version":"4.5.0"},"reference-count":41,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2021,4,27]],"date-time":"2021-04-27T00:00:00Z","timestamp":1619481600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,27]],"date-time":"2021-04-27T00:00:00Z","timestamp":1619481600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from \u2019t Hooft\u2019s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.<\/jats:p>","DOI":"10.1088\/2632-2153\/abeca3","type":"journal-article","created":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T22:30:43Z","timestamp":1615242643000},"page":"035002","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":68,"title":["Neural networks and quantum field theory"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0535-2622","authenticated-orcid":false,"given":"James","family":"Halverson","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4712-6626","authenticated-orcid":false,"given":"Anindita","family":"Maiti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1708-2773","authenticated-orcid":false,"given":"Keegan","family":"Stoner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2021,4,27]]},"reference":[{"key":"mlstabeca3bib1","article-title":"Bayesian learning for neural networks","author":"Neal","year":"1995"},{"key":"mlstabeca3bib2","first-page":"pp 295","article-title":"Computing with infinite networks","author":"Williams","year":"1997"},{"key":"mlstabeca3bib3","article-title":"Deep neural networks as Gaussian processes","author":"Lee","year":"2017"},{"key":"mlstabeca3bib4","article-title":"Gaussian process behaviour in wide deep neural networks","author":"Matthews","year":"2018"},{"key":"mlstabeca3bib5","article-title":"Bayesian convolutional neural networks with many channels are Gaussian processes","author":"Novak","year":"2018"},{"key":"mlstabeca3bib6","article-title":"Deep convolutional networks as shallow Gaussian processes","author":"Garriga-Alonso","year":"2019"},{"key":"mlstabeca3bib7","article-title":"Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation","author":"Yang","year":"2019"},{"key":"mlstabeca3bib8","article-title":"Tensor programs I: wide feedforward or recurrent neural networks of any architecture are Gaussian processes","author":"Yang","year":"2019"},{"key":"mlstabeca3bib9","article-title":"Tensor programs II: neural tangent kernel for any architecture","author":"Yang","year":"2020"},{"key":"mlstabeca3bib10","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"mlstabeca3bib11","article-title":"Densely connected convolutional networks","author":"Huang","year":"2016"},{"key":"mlstabeca3bib12","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1007\/BF00342633","article-title":"Cognitron: a self-organizing multilayered neural network","volume":"20","author":"Fukushima","year":"1975","journal-title":"Biol. Cybern."},{"key":"mlstabeca3bib13","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/BF00344251","article-title":"Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position","volume":"36","author":"Fukushima","year":"1980","journal-title":"Biol. Cybern."},{"key":"mlstabeca3bib14","doi-asserted-by":"crossref","DOI":"10.21236\/ADA164453","article-title":"Learning internal representations by error propagation","author":"Rumelhart","year":"1985"},{"key":"mlstabeca3bib15","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"mlstabeca3bib16","first-page":"pp 319","author":"LeCun","year":"1999"},{"key":"mlstabeca3bib17","article-title":"Spectral networks and locally connected networks on graphs","author":"Bruna","year":"2013"},{"key":"mlstabeca3bib18","article-title":"Deep convolutional networks on graph-structured data","author":"Henaff","year":"2015"},{"key":"mlstabeca3bib19","first-page":"pp 2224","article-title":"Convolutional networks on graphs for learning molecular fingerprints","author":"Duvenaud","year":"2015"},{"key":"mlstabeca3bib20","article-title":"Gated graph sequence neural networks","author":"Yujia","year":"2015"},{"key":"mlstabeca3bib21","first-page":"pp 3844","article-title":"Convolutional neural networks on graphs with fast localized spectral filtering","author":"Defferrard","year":"2016"},{"key":"mlstabeca3bib22","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf","year":"2016"},{"key":"mlstabeca3bib23","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","author":"Ioffe","year":"2015"},{"key":"mlstabeca3bib24","article-title":"Layer normalization","author":"Ba","year":"2016"},{"key":"mlstabeca3bib25","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau","year":"2014"},{"key":"mlstabeca3bib26","first-page":"pp 5998","article-title":"Attention is all you need","author":"Vaswani","year":"2017"},{"key":"mlstabeca3bib27","article-title":"Wide neural networks of any depth evolve as linear models under gradient descent","author":"Lee","year":"2019"},{"key":"mlstabeca3bib28","article-title":"Neural tangent kernel: convergence and generalization in neural networks","author":"Jacot","year":"2018"},{"key":"mlstabeca3bib29","article-title":"Finite size corrections for neural network Gaussian processes","author":"Antognini","year":"2019"},{"key":"mlstabeca3bib30","article-title":"Predicting the outputs of finite networks trained with noisy gradients","author":"Naveh","year":"2020"},{"key":"mlstabeca3bib31","article-title":"Non-Gaussian processes and neural networks at finite widths","author":"Yaida","year":"2019"},{"key":"mlstabeca3bib32","article-title":"Learning curves for deep neural networks: a Gaussian field theory perspective","author":"Cohen","year":"2020"},{"key":"mlstabeca3bib33","article-title":"Asymptotics of wide networks from Feynman diagrams","author":"Dyer","year":"2020"},{"key":"mlstabeca3bib34","article-title":"Tensor programs III: neural matrix laws","author":"Yang","year":"2020"},{"key":"mlstabeca3bib35","article-title":"Feature learning in infinite-width neural networks","author":"Yang","year":"2020"},{"key":"mlstabeca3bib36","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.69.044026","article-title":"Chameleon cosmology","volume":"69","author":"Khoury","year":"2004","journal-title":"Phys. Rev. D"},{"key":"mlstabeca3bib37","article-title":"A fine-grained spectral perspective on neural networks","author":"Yang","year":"2019"},{"key":"mlstabeca3bib38","first-page":"pp 342","article-title":"Kernel methods for deep learning","author":"Cho","year":"2009"},{"key":"mlstabeca3bib39","author":"Schwartz","year":"2014"},{"key":"mlstabeca3bib40","first-page":"491","article-title":"Tentativo di una teoria dell\u2019emissione dei raggi \u2018beta\u2019","volume":"4","author":"Fermi","year":"1933","journal-title":"Ric. Sci."},{"key":"mlstabeca3bib41","volume":"vol 11","author":"Zee","year":"2003"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,13]],"date-time":"2021-12-13T14:58:59Z","timestamp":1639407539000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abeca3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,27]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,4,27]]},"published-print":{"date-parts":[[2021,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abeca3","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,27]]},"assertion":[{"value":"Neural networks and quantum field theory","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2020-08-22","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-03-08","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-04-27","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}