{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T17:24:11Z","timestamp":1777915451976,"version":"3.51.4"},"reference-count":64,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T00:00:00Z","timestamp":1690848000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T00:00:00Z","timestamp":1690848000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"National Science Foundation","award":["DMS-1818772"],"award-info":[{"award-number":["DMS-1818772"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2023,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We consider the simulation of Bayesian statistical inverse problems governed by large-scale linear and nonlinear partial differential equations (PDEs). Markov chain Monte Carlo (MCMC) algorithms are standard techniques to solve such problems. However, MCMC techniques are computationally challenging as they require a prohibitive number of forward PDE solves. The goal of this paper is to introduce a fractional deep neural network (fDNN) based approach for the forward solves within an MCMC routine. Moreover, we discuss some approximation error estimates. We illustrate the efficiency of fDNN on inverse problems governed by nonlinear elliptic PDEs and the unsteady Navier\u2013Stokes equations. In the former case, two examples are discussed, respectively depending on two and 100 parameters, with significant observed savings. The unsteady Navier\u2013Stokes example illustrates that fDNN can outperform existing DNNs, doing a better job of capturing essential features such as vortex shedding.<\/jats:p>","DOI":"10.1088\/2632-2153\/ace67c","type":"journal-article","created":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T22:41:25Z","timestamp":1689115285000},"page":"035015","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["A deep neural network approach for parameterized PDEs and Bayesian inverse problems"],"prefix":"10.1088","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6641-1449","authenticated-orcid":true,"given":"Harbir","family":"Antil","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Howard C","family":"Elman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2110-8881","authenticated-orcid":true,"given":"Akwum","family":"Onwunta","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deepanshu","family":"Verma","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2023,8,1]]},"reference":[{"key":"mlstace67cbib1","first-page":"pp 101","article-title":"Application of the discrete empirical interpolation method to reduced order modeling of nonlinear and parametric systems","author":"Antil","year":"2014"},{"key":"mlstace67cbib2","author":"Antil","year":"2020"},{"key":"mlstace67cbib3","first-page":"pp 3","article-title":"A brief introduction to PDE-constrained optimization","author":"Antil","year":"2018"},{"key":"mlstace67cbib4","author":"Bardsley","year":"2018"},{"key":"mlstace67cbib5","doi-asserted-by":"publisher","first-page":"A1895","DOI":"10.1137\/140964023","article-title":"Randomize-then-optimize: A method for sampling from posterior distributions in nonlinear inverse problems","volume":"36","author":"Bardsley","year":"2014","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib6","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1016\/j.crma.2004.08.006","article-title":"An \u2018empirical interpolation\u2019 method: Application to efficient reduced-basis discretization of partial differential equations","volume":"339","author":"Barrault","year":"2004","journal-title":"C. R. Math."},{"key":"mlstace67cbib7","doi-asserted-by":"publisher","first-page":"171","DOI":"10.3934\/jcd.2019009","article-title":"Deep learning as optimal control problems: models and numerical methods","volume":"6","author":"Benning","year":"2019","journal-title":"J. Comp. Dyn."},{"key":"mlstace67cbib8","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1016\/j.jcp.2016.12.041","article-title":"Geometric MCMC for infinite-dimensional inverse problems","volume":"335","author":"Beskos","year":"2017","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib9","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1007\/s00211-022-01294-z","article-title":"Error estimates for deep learning methods in fluid dynamics","volume":"151","author":"Biswas","year":"2022","journal-title":"Numer. Math."},{"key":"mlstace67cbib10","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-90539-2_2","article-title":"Novel DNNs for stiff ODEs with applications to chemically reacting flows","author":"Brown","year":"2021"},{"key":"mlstace67cbib11","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/30\/11\/114014","article-title":"Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold hamiltonian Monte Carlo","volume":"30","author":"Bui-Thanh","year":"2014","journal-title":"Inverse Problems"},{"key":"mlstace67cbib12","author":"Calvetti","year":"2007"},{"key":"mlstace67cbib13","article-title":"Nonlinear model reduction via discrete empirical interpolation","author":"Chaturantabut","year":"2011"},{"key":"mlstace67cbib14","doi-asserted-by":"publisher","first-page":"2737","DOI":"10.1137\/090766498","article-title":"Nonlinear model reduction via discrete empirical interpolation","volume":"32","author":"Chaturantabut","year":"2010","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib15","doi-asserted-by":"publisher","first-page":"11618","DOI":"10.1364\/OE.384875","article-title":"Physics-informed neural networks for inverse problems in nano-optics and metamaterials","volume":"28","author":"Chen","year":"2020","journal-title":"Opt. Express"},{"key":"mlstace67cbib16","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1007\/s10915-022-01939-z","article-title":"Scientific machine learning through physics-informed neural networks: where we are and what\u2019s next","volume":"92","author":"Cuomo","year":"2022","journal-title":"J. Sci. Comput."},{"key":"mlstace67cbib17","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/BF02288367","article-title":"The approximation of one matrix by another of lower rank","volume":"1","author":"Eckart","year":"1936","journal-title":"Psychometrika"},{"key":"mlstace67cbib18","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1017\/S0962492900002439","article-title":"Aspects of the numerical analysis of neural networks","volume":"3","author":"Ellacott","year":"1994","journal-title":"Acta Numer."},{"key":"mlstace67cbib19","doi-asserted-by":"publisher","first-page":"S177","DOI":"10.1137\/140970859","article-title":"Preconditioning techniques for reduced basis methods for parameterized elliptic partial differential equations","volume":"37","author":"Elman","year":"2015","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib20","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1016\/j.cma.2016.12.011","article-title":"Numerical solution of the steady-state Navier-Stokes equations using empirical interpolation methods","volume":"317","author":"Elman","year":"2017","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"mlstace67cbib21","first-page":"1","article-title":"Reduced-order modeling for nonlinear Bayesian statistical inverse problems","author":"Elman","year":"2019"},{"key":"mlstace67cbib22","author":"Elman","year":"2014","edition":"2nd edn"},{"key":"mlstace67cbib23","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1137\/090780717","article-title":"Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations","volume":"33","author":"Flath","year":"2011","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib24","article-title":"Iterative Solution Methods for Reduced-order Models of Parameterized Partial Differential Equations","author":"Forstall","year":"2015"},{"key":"mlstace67cbib25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2020.110079","article-title":"PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain","volume":"428","author":"Gao","year":"2021","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib26","author":"Golub","year":"1996"},{"key":"mlstace67cbib27","author":"Goodfellow","year":"2016"},{"key":"mlstace67cbib28","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1051\/m2an:2007031","article-title":"Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations","volume":"41","author":"Grepl","year":"2007","journal-title":"Math. Modelling Numer. Anal."},{"key":"mlstace67cbib29","doi-asserted-by":"publisher","first-page":"A2485","DOI":"10.1137\/18M1204991","article-title":"Machine learning of space-fractional differential equations","volume":"41","author":"Gulian","year":"2019","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1137\/19M1247620","article-title":"Layer-parallel training of deep residual neural networks","volume":"2","author":"G\u00fcnther","year":"2020","journal-title":"SIAM J. Math. Data Sci."},{"key":"mlstace67cbib31","doi-asserted-by":"publisher","first-page":"223","DOI":"10.2307\/3318737","article-title":"An adaptive Metropolis algorithms","volume":"7","author":"Haario","year":"2001","journal-title":"Bernoulli"},{"key":"mlstace67cbib32","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/aa9a90","article-title":"Stable architectures for deep neural networks","volume":"34","author":"Haber","year":"2017","journal-title":"Inverse Problems"},{"key":"mlstace67cbib33","doi-asserted-by":"publisher","first-page":"8505","DOI":"10.1073\/pnas.1718942115","article-title":"Solving high-dimensional partial differential equations using deep learning","volume":"115","author":"Han","year":"2018","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstace67cbib34","doi-asserted-by":"publisher","first-page":"502","DOI":"10.4208\/jcm.1901-m2018-0160","article-title":"ReLU deep neural networks and linear finite elements","volume":"38","author":"He","year":"2020","journal-title":"J. Comput. Math."},{"key":"mlstace67cbib35","first-page":"pp 770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"mlstace67cbib36","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.jcp.2018.02.037","article-title":"Non-intrusive reduced order modeling of nonlinear problems using neural networks","volume":"363","author":"Hesthaven","year":"2018","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib37","author":"Hesthaven","year":"2016"},{"key":"mlstace67cbib38","doi-asserted-by":"publisher","first-page":"1262","DOI":"10.1162\/neco.1994.6.6.1262","article-title":"Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives","volume":"6","author":"Hornik","year":"1994","journal-title":"Neural Comput."},{"key":"mlstace67cbib39","first-page":"pp 4700","article-title":"Densely connected convolutional networks","author":"Huang","year":"2017"},{"key":"mlstace67cbib40","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1002\/fld.679","article-title":"Reference values for drag and lift of a two-dimensional time-dependent flow around a cylinder","volume":"44","author":"John","year":"2004","journal-title":"Int. J. Numer. Methods Fluids"},{"key":"mlstace67cbib41","author":"Kaipio","year":"2005"},{"key":"mlstace67cbib42","author":"Kelley","year":"1995"},{"key":"mlstace67cbib43","author":"Kelley","year":"1999"},{"key":"mlstace67cbib44","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/S0893-6080(05)80131-5","article-title":"Multilayer feedforward networks with a nonpolynomial activation function can approximate any function","volume":"6","author":"Leshno","year":"1993","journal-title":"Neural Netw."},{"key":"mlstace67cbib45","doi-asserted-by":"publisher","first-page":"A1163","DOI":"10.1137\/130938189","article-title":"Adaptive construction of surrogates for the bayesian solution of inverse problems","volume":"36","author":"Li","year":"2014","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib46","article-title":"Fourier neural operator for parametric partial differential equations","author":"Li","year":"2020"},{"key":"mlstace67cbib47","doi-asserted-by":"publisher","first-page":"A1460","DOI":"10.1137\/110845598","article-title":"A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion","volume":"34","author":"Martin","year":"2012","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib48","doi-asserted-by":"publisher","first-page":"981","DOI":"10.1093\/imanum\/drab032","article-title":"Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs","volume":"42","author":"Mishra","year":"2022","journal-title":"IMA J. Numer. Anal."},{"key":"mlstace67cbib49","doi-asserted-by":"publisher","first-page":"A2603","DOI":"10.1137\/18M1229845","article-title":"Fpinns: fractional physics-informed neural networks","volume":"41","author":"Pang","year":"2019","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstace67cbib50","volume":"vol 92","author":"Quarteroni","year":"2015"},{"key":"mlstace67cbib51","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib52","first-page":"981","article-title":"Error estimates for physics informed neural networks approximating the Navier-Stokes equations","author":"De Ryck","year":"2023"},{"key":"mlstace67cbib53","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.neunet.2020.05.019","article-title":"Approximation rates for neural networks with general activation functions","volume":"128","author":"Siegel","year":"2020","journal-title":"Neural Netw."},{"key":"mlstace67cbib54","author":"Smith","year":"2014"},{"key":"mlstace67cbib55","first-page":"pp 131","article-title":"Monte Carlo methods in statistical mechanics: foundations and new algorithms","author":"Sokal","year":"1997"},{"key":"mlstace67cbib56","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1016\/j.jcp.2018.08.036","article-title":"Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification","volume":"375","author":"Tripathy","year":"2018","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib57","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1007\/s10915-006-9083-y","article-title":"Numerical study of a modified time-stepping \u03b8-scheme for incompressible flow simulations","volume":"28","author":"Turek","year":"2006","journal-title":"J. Sci. Comput."},{"key":"mlstace67cbib58","first-page":"pp 550","article-title":"Residual networks behave like ensembles of relatively shallow networks","author":"Veit","year":"2016"},{"key":"mlstace67cbib59","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1515\/IJNSNS.2009.10.3.273","article-title":"Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling","volume":"10","author":"Vrugt","year":"2009","journal-title":"Commun. Nonlinear Sci. Numer. Simul."},{"key":"mlstace67cbib60","doi-asserted-by":"publisher","first-page":"2180","DOI":"10.4208\/cicp.OA-2020-0186","article-title":"An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems","volume":"28","author":"Yan","year":"2020","journal-title":"Commun. Comput. Phys."},{"key":"mlstace67cbib61","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2020.109913","article-title":"B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data","volume":"425","author":"Yang","year":"2021","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib62","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.neunet.2017.07.002","article-title":"Error bounds for approximations with deep ReLU networks","volume":"94","author":"Yarotsky","year":"2017","journal-title":"Neural Netw."},{"key":"mlstace67cbib63","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1016\/j.jcp.2018.04.018","article-title":"Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification","volume":"366","author":"Zhu","year":"2018","journal-title":"J. Comput. Phys."},{"key":"mlstace67cbib64","article-title":"Adaptive particle-based approximations of the Gibbs posterior for inverse problems","author":"Zou","year":"2019"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T10:34:52Z","timestamp":1690886092000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace67c"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,1]]},"references-count":64,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,8,1]]},"published-print":{"date-parts":[[2023,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ace67c","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,1]]},"assertion":[{"value":"A deep neural network approach for parameterized PDEs and Bayesian inverse problems","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2023 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2023-02-28","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-07-11","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-08-01","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}