{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T09:21:10Z","timestamp":1759742470004,"version":"3.37.3"},"reference-count":50,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2024,8,20]],"date-time":"2024-08-20T00:00:00Z","timestamp":1724112000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2024,8,20]],"date-time":"2024-08-20T00:00:00Z","timestamp":1724112000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design and training protocol requires careful craftsmanship. Here, we suggest a comprehensive theoretical framework that sheds light on this important problem. Leveraging an equivalence between infinitely over-parameterized neural networks and Gaussian process regression, we derive an integro-differential equation that governs PINN prediction in the large data-set limit\u2014the neurally-informed equation. This equation augments the original one by a kernel term reflecting architecture choices. It allows quantifying implicit bias induced by the network via a spectral decomposition of the source term in the original differential equation.<\/jats:p>","DOI":"10.1088\/2632-2153\/ad652d","type":"journal-article","created":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T22:58:58Z","timestamp":1721343538000},"page":"035048","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Spectral-bias and kernel-task alignment in physically informed neural networks"],"prefix":"10.1088","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5209-5839","authenticated-orcid":true,"given":"Inbar","family":"Seroussi","sequence":"first","affiliation":[]},{"given":"Asaf","family":"Miron","sequence":"additional","affiliation":[]},{"given":"Zohar","family":"Ringel","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2024,8,20]]},"reference":[{"key":"mlstad652dbib1","first-page":"p 25","article-title":"Imagenet classification with deep convolutional neural networks","author":"Krizhevsky","year":"2012"},{"key":"mlstad652dbib2","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with alphafold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"mlstad652dbib3","first-page":"pp 1877","article-title":"Language models are few-shot learners","volume":"vol 33","author":"Brown","year":"2020"},{"key":"mlstad652dbib4","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"mlstad652dbib5","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2019.112789","article-title":"Physics-informed neural networks for high-speed flows","volume":"360","author":"Mao","year":"2020","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"mlstad652dbib6","doi-asserted-by":"publisher","first-page":"2616","DOI":"10.6052\/0459-1879-21-373","article-title":"Physics-enhanced deep learning methods for modelling and simulating flow fields","volume":"53","author":"Jin","year":"2021","journal-title":"Chin. J. Theor. Appl. Mech."},{"key":"mlstad652dbib7","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1007\/s00348-022-03554-y","article-title":"Estimating density, velocity and pressure fields in supersonic flows using physics-informed bos","volume":"64","author":"Molnar","year":"2023","journal-title":"Exp. Fluids"},{"key":"mlstad652dbib8","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110754","article-title":"Thermodynamically consistent physics-informed neural networks for hyperbolic systems","volume":"449","author":"Patel","year":"2022","journal-title":"J. Comput. Phys."},{"key":"mlstad652dbib9","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2023.112265","article-title":"Physics-informed neural networks with adaptive localized artificial viscosity","volume":"489","author":"Jose Rocha Coutinho","year":"2023","journal-title":"J. Comput. Phys."},{"key":"mlstad652dbib10","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1063\/5.0123466","article-title":"Tom Gruenwald and Carlo Scalo. Neural-network-based riemann solver for real fluids and high explosives; application to computational fluid dynamics","volume":"34","author":"Ruggeri","year":"2022","journal-title":"Phys. Fluids"},{"key":"mlstad652dbib11","doi-asserted-by":"publisher","DOI":"10.1111\/str.12431","article-title":"Calibrating constitutive models with full-field data via physics informed neural networks","volume":"59","author":"Hamel","year":"2023","journal-title":"Strain"},{"key":"mlstad652dbib12","doi-asserted-by":"publisher","first-page":"1727","DOI":"10.1007\/s10409-021-01148-1","article-title":"Physics-informed neural networks (pinns) for fluid mechanics: A review","volume":"37","author":"Cai","year":"2021","journal-title":"Acta Mech. Sin."},{"key":"mlstad652dbib13","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1038\/s42254-021-00314-5","article-title":"Physics-informed machine learning","volume":"3","author":"Em Karniadakis","year":"2021","journal-title":"Nat. Rev. Phys."},{"article-title":"Physics-informed machine learning: a survey on problems, methods and applications","year":"2022","author":"Hao","key":"mlstad652dbib14"},{"key":"mlstad652dbib15","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1007\/s10915-022-01939-z","article-title":"Scientific machine learning through physics\u2013informed neural networks: where we are and what\u2019s next","volume":"92","author":"Cuomo","year":"2022","journal-title":"J. Sci. Comput."},{"article-title":"Partial differential equations meet deep neural networks: a survey","year":"2022","author":"Huang","key":"mlstad652dbib16"},{"article-title":"State-of-the-art review of design of experiments for physics-informed deep learning","year":"2022","author":"Das","key":"mlstad652dbib17"},{"key":"mlstad652dbib18","first-page":"pp 4376","article-title":"Infinite attention: Nngp and ntk for deep attention networks","author":"Hron","year":"2020"},{"key":"mlstad652dbib19","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.104.064301","article-title":"Predicting the outputs of finite deep neural networks trained with noisy gradients","volume":"104","author":"Naveh","year":"2021","journal-title":"Phys. Rev. E"},{"key":"mlstad652dbib20","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.11.031059","article-title":"Statistical mechanics of deep linear neural networks: the backpropagating kernel renormalization","volume":"11","author":"Li","year":"2021","journal-title":"Phys. Rev. X"},{"key":"mlstad652dbib21","first-page":"pp 21352","volume":"vol 34","author":"Naveh","year":"2021"},{"key":"mlstad652dbib22","doi-asserted-by":"publisher","first-page":"908","DOI":"10.1038\/s41467-023-36361-y","article-title":"Separation of scales and a thermodynamic description of feature learning in some cnns","volume":"14","author":"Seroussi","year":"2023","journal-title":"Nat. Commun."},{"key":"mlstad652dbib23","article-title":"Bayesian interpolation with deep linear networks","volume":"vol 120","author":"Hanin","year":"2023"},{"key":"mlstad652dbib24","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/ac9829","article-title":"Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime","volume":"2022","author":"Cui","year":"2022","journal-title":"J. Stat. Mech."},{"article-title":"Droplets of good representations: Grokking as a first order phase transition in two layer networks","year":"2023","author":"Rubin","key":"mlstad652dbib25"},{"key":"mlstad652dbib26","first-page":"pp 21548","article-title":"Failure and success of the spectral bias prediction for laplace kernel ridge regression: the case of low-dimensional data","author":"Tomasini","year":"2022"},{"article-title":"Tuning large neural networks via zero-shot hyperparameter transfer","year":"2021","author":"Yang","key":"mlstad652dbib27"},{"article-title":"Explaining scaling laws of neural network generalization","year":"2022","author":"Bahri","key":"mlstad652dbib28"},{"article-title":"A solvable model of neural scaling laws","year":"2022","author":"Maloney","key":"mlstad652dbib29"},{"key":"mlstad652dbib30","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.3.023034","article-title":"Learning curves for overparametrized deep neural networks: a field theory perspective","volume":"3","author":"Cohen","year":"2021","journal-title":"Phys. Rev. Res."},{"key":"mlstad652dbib31","doi-asserted-by":"publisher","first-page":"2914","DOI":"10.1038\/s41467-021-23103-1","article-title":"Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks","volume":"12","author":"Canatar","year":"2021","journal-title":"Nat. Commun."},{"key":"mlstad652dbib32","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2021.113938","article-title":"On the eigenvector bias of fourier feature networks: from regression to solving multi-scale pdes with physics-informed neural networks","volume":"384","author":"Wang","year":"2021","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"mlstad652dbib33","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110768","article-title":"When and why PINNs fail to train: a neural tangent kernel perspective","volume":"449","author":"Wang","year":"2022","journal-title":"J. Comput. Phys."},{"article-title":"Improved architectures and training algorithms for deep operator networks","year":"2021","author":"Wang","key":"mlstad652dbib34"},{"key":"mlstad652dbib35","first-page":"p 31","article-title":"Neural tangent kernel: convergence and generalization in neural networks","author":"Jacot","year":"2018"},{"article-title":"Stochastic gradient descent as approximate bayesian inference","year":"2017","author":"Mandt","key":"mlstad652dbib36"},{"key":"mlstad652dbib37","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2015.123","article-title":"Delving deep into rectifiers: surpassing human-level performance on imagenet classification","author":"He","year":"2015"},{"key":"mlstad652dbib38","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/j.jcp.2017.07.050","article-title":"Machine learning of linear differential equations using gaussian processes","volume":"348","author":"Raissi","year":"2017","journal-title":"J. Comput. Phys."},{"volume":"vol 2","year":"2006","author":"Williams","key":"mlstad652dbib39"},{"article-title":"Deep neural networks as gaussian processes","year":"2018","author":"Lee","key":"mlstad652dbib40"},{"key":"mlstad652dbib41","first-page":"4629","article-title":"What are Bayesian neural network posteriors really like?","author":"Izmailov","year":"2021"},{"article-title":"Bayesian deep convolutional networks with many channels are gaussian processes","year":"2018","author":"Novak","key":"mlstad652dbib42"},{"key":"mlstad652dbib43","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1137\/19M1274067","article-title":"Deepxde: a deep learning library for solving differential equations","volume":"63","author":"Lu","year":"2021","journal-title":"SIAM Rev."},{"key":"mlstad652dbib44","doi-asserted-by":"publisher","first-page":"7557","DOI":"10.1038\/s41598-022-11058-2","article-title":"Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the buckley\u2013leverett problem","volume":"12","author":"Rodriguez-Torrado","year":"2022","journal-title":"Sci. Rep."},{"key":"mlstad652dbib45","doi-asserted-by":"publisher","DOI":"10.1142\/S0129183123500821","article-title":"A hybrid physics-informed neural network for nonlinear partial differential equation","volume":"34","author":"Lv","year":"2023","journal-title":"Int. J. Mod. Phys. C"},{"key":"mlstad652dbib46","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1615\/JMachLearnModelComput.2020033905","article-title":"Limitations of physics informed machine learning for nonlinear two-phase transport in porous media","volume":"1","author":"Fuks","year":"2020","journal-title":"J. Mach. Learn. Model. Comput."},{"key":"mlstad652dbib47","first-page":"pp 15344","article-title":"Learning data-driven discretizations for partial differential equations","volume":"vol 116","author":"Bar-Sinai","year":"2019"},{"key":"mlstad652dbib48","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1007\/978-3-030-44992-6_14","article-title":"Physics-informed learning machines for partial differential equations: Gaussian processes versus neural networks","volume":"32","author":"Pang","year":"2020","journal-title":"Emerg. Front. Nonlinear Sci."},{"year":"2009","author":"Mezard","key":"mlstad652dbib49"},{"key":"mlstad652dbib50","first-page":"p 14","article-title":"A variational approach to learning curves","author":"Malzahn","year":"2001"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,20]],"date-time":"2024-08-20T09:27:46Z","timestamp":1724146066000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad652d"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,20]]},"references-count":50,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,8,20]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ad652d","relation":{},"ISSN":["2632-2153"],"issn-type":[{"type":"electronic","value":"2632-2153"}],"subject":[],"published":{"date-parts":[[2024,8,20]]},"assertion":[{"value":"Spectral-bias and kernel-task alignment in physically informed neural networks","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-03-11","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-07-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-08-20","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}