{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T14:47:21Z","timestamp":1776955641470,"version":"3.51.4"},"reference-count":78,"publisher":"ASME International","issue":"1","content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In this paper, we introduce a physics-driven regularization method for training of deep neural networks (DNNs) for use in engineering design and analysis problems. In particular, we focus on the prediction of a physical system, for which in addition to training data, partial or complete information on a set of governing laws is also available. These laws often appear in the form of differential equations, derived from first principles, empirically validated laws, or domain expertise, and are usually neglected in a data-driven prediction of engineering systems. We propose a training approach that utilizes the known governing laws and regularizes data-driven DNN models by penalizing divergence from those laws. The first two numerical examples are synthetic examples, where we show that in constructing a DNN model that best fits the measurements from a physical system, the use of our proposed regularization results in DNNs that are more interpretable with smaller generalization errors, compared with other common regularization methods. The last two examples concern metamodeling for a random Burgers\u2019 system and for aerodynamic analysis of passenger vehicles, where we demonstrate that the proposed regularization provides superior generalization accuracy compared with other common alternatives.<\/jats:p>","DOI":"10.1115\/1.4044507","type":"journal-article","created":{"date-parts":[[2019,9,5]],"date-time":"2019-09-05T09:45:56Z","timestamp":1567676756000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":65,"title":["Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis"],"prefix":"10.1115","volume":"20","author":[{"given":"Mohammad Amin","family":"Nabian","sequence":"first","affiliation":[{"name":"Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, 205 N Mathews Avenue, Urbana, IL 61801"}]},{"given":"Hadi","family":"Meidani","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, 205 N Mathews Avenue, Urbana, IL 61801"}]}],"member":"33","published-online":{"date-parts":[[2019,9,10]]},"reference":[{"key":"2019110410333066500_CIT0001","volume-title":"Deep Learning","author":"Goodfellow","year":"2016"},{"issue":"7553","key":"2019110410333066500_CIT0002","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"1","key":"2019110410333066500_CIT0003","first-page":"1929","article-title":"Dropout: a Simple Way to Prevent Neural Networks From Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"2019110410333066500_CIT0004","first-page":"481","article-title":"Convolutional Neural Networks for Steady Flow Approximation","author":"Guo","year":"2016"},{"key":"2019110410333066500_CIT0005","article-title":"Lat-Net: Compressing Lattice Boltzmann Flow Simulations Using Deep Neural Networks","author":"Hennigh","year":"2017"},{"key":"2019110410333066500_CIT0006","article-title":"Automated Design Using Neural Networks and Gradient Descent","author":"Hennigh","year":"2017"},{"issue":"1","key":"2019110410333066500_CIT0007","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.paerosci.2005.02.001","article-title":"Surrogate-Based Analysis and Optimization","volume":"41","author":"Queipo","year":"2005","journal-title":"Prog. Aerosp. Sci."},{"issue":"4","key":"2019110410333066500_CIT0008","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1007\/s10869-010-9183-4","article-title":"Polynomial Regression With Response Surface Analysis: A Powerful Approach for Examining Moderation and Overcoming Limitations of Difference Scores","volume":"25","author":"Shanock","year":"2010","journal-title":"J. Bus. Psychol."},{"key":"2019110410333066500_CIT0009","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0962492900000015","article-title":"Radial Basis Functions","volume":"9","author":"Buhmann","year":"2000","journal-title":"Acta Numer."},{"issue":"6","key":"2019110410333066500_CIT0010","doi-asserted-by":"crossref","first-page":"3197","DOI":"10.1137\/070691814","article-title":"Orbit: Optimization by Radial Basis Function Interpolation in Trust-regions","volume":"30","author":"Wild","year":"2008","journal-title":"SIAM J. Sci. Comput."},{"issue":"2","key":"2019110410333066500_CIT0011","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1137\/S1064827501387826","article-title":"The Wiener\u2013askey Polynomial Chaos for Stochastic Differential Equations","volume":"24","author":"Xiu","year":"2002","journal-title":"SIAM J. Sci. Comput."},{"issue":"6","key":"2019110410333066500_CIT0012","doi-asserted-by":"crossref","first-page":"1862","DOI":"10.1016\/j.jcp.2008.11.024","article-title":"Dimensionality Reduction and Polynomial Chaos Acceleration of Bayesian Inference in Inverse Problems","volume":"228","author":"Marzouk","year":"2009","journal-title":"J. Comput. Phys."},{"issue":"12","key":"2019110410333066500_CIT0013","doi-asserted-by":"crossref","first-page":"2233","DOI":"10.2514\/2.1234","article-title":"Kriging Models for Global Approximation in Simulation-based Multidisciplinary Design Optimization","volume":"39","author":"Simpson","year":"2001","journal-title":"AIAA J."},{"issue":"2","key":"2019110410333066500_CIT0014","doi-asserted-by":"crossref","first-page":"413","DOI":"10.2514\/1.6386","article-title":"Efficient Optimization Design Method Using Kriging Model","volume":"42","author":"Jeong","year":"2005","journal-title":"J. Aircr."},{"issue":"1","key":"2019110410333066500_CIT0015","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/s00366-018-0590-x","article-title":"Gradient-Enhanced Kriging for High-Dimensional Problems","volume":"35","author":"Bouhlel","year":"2019","journal-title":"Eng. Comput."},{"key":"2019110410333066500_CIT0016","article-title":"Efficient Uncertainty Quantification With Gradient-Enhanced Kriging: Applications in FSI","volume-title":"ECCOMAS","author":"De Baar","year":"2012"},{"issue":"3","key":"2019110410333066500_CIT0017","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1023\/B:STCO.0000035301.49549.88","article-title":"A Tutorial on Support Vector Regression","volume":"14","author":"Smola","year":"2004","journal-title":"Stat. Comput."},{"issue":"6","key":"2019110410333066500_CIT0018","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.1115\/1.1897403","article-title":"Analysis of Support Vector Regression for Approximation of Complex Engineering Analyses","volume":"127","author":"Clarke","year":"2005","journal-title":"ASME J. Mech. Des."},{"issue":"4","key":"2019110410333066500_CIT0019","doi-asserted-by":"crossref","first-page":"e1602614","DOI":"10.1126\/sciadv.1602614","article-title":"Data-Driven Discovery of Partial Differential Equations","volume":"3","author":"Rudy","year":"2017","journal-title":"Sci. Adv."},{"key":"2019110410333066500_CIT0020","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.jcp.2017.11.039","article-title":"Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations","volume":"357","author":"Raissi","year":"2018","journal-title":"J. Comput. Phys."},{"key":"2019110410333066500_CIT0021","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.jcp.2019.01.030","article-title":"Numerical Aspects for Approximating Governing Equations Using Data","volume":"384","author":"Wu","year":"2019","journal-title":"J. Comput. Phys."},{"issue":"17","key":"2019110410333066500_CIT0022","doi-asserted-by":"crossref","first-page":"3422","DOI":"10.1103\/PhysRevLett.83.3422","article-title":"Amplitude Equations From Spatiotemporal Binary-Fluid Convection Data","volume":"83","author":"Voss","year":"1999","journal-title":"Phys. Rev. Lett."},{"key":"2019110410333066500_CIT0023","first-page":"1","article-title":"Study on a 3D Possion\u2019s Equation Slover Based on Deep Learning Technique","author":"Shan","year":"2018"},{"issue":"6","key":"2019110410333066500_CIT0024","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1111\/mice.12359","article-title":"Deep Learning for Accelerated Seismic Reliability Analysis of Transportation Networks","volume":"33","author":"Nabian","year":"2018","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"2019110410333066500_CIT0025","article-title":"Data Driven Governing Equations Approximation using Deep Neural Networks","author":"Qin","year":"2018"},{"issue":"5","key":"2019110410333066500_CIT0026","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1109\/72.712178","article-title":"Artificial Neural Networks for Solving Ordinary and Partial Differential Equations","volume":"9","author":"Lagaris","year":"1998","journal-title":"IEEE Trans. Neural Networks"},{"issue":"1","key":"2019110410333066500_CIT0027","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/0021-9991(90)90007-N","article-title":"Neural Algorithm for Solving Differential Equations","volume":"91","author":"Lee","year":"1990","journal-title":"J. Comput. Phys."},{"key":"2019110410333066500_CIT0028","first-page":"125","article-title":"Structured Trainable Networks for Matrix Algebra","author":"Wang","year":"1990"},{"issue":"4","key":"2019110410333066500_CIT0029","doi-asserted-by":"crossref","first-page":"042113","DOI":"10.1103\/PhysRevA.96.042113","article-title":"Deep Learning and the Schr\u00f6dinger Equation","volume":"96","author":"Mills","year":"2017","journal-title":"Phys. Rev. A"},{"key":"2019110410333066500_CIT0030","article-title":"Learning A Physical Long-Term Predictor","author":"Ehrhardt","year":"2017"},{"key":"2019110410333066500_CIT0031","article-title":"Solving Poisson\u2019s Equation Using Deep Learning in Particle Simulation of PN Junction","author":"Zhang","year":"2018"},{"key":"2019110410333066500_CIT0032","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"2019110410333066500_CIT0033","doi-asserted-by":"crossref","first-page":"S965","DOI":"10.1016\/S0098-1354(98)00191-4","article-title":"Identification of Distributed Parameter Systems: A Neural Net Based Approach","volume":"22","author":"Gonzalez-Garcia","year":"1998","journal-title":"Comput. Chem. Eng."},{"key":"2019110410333066500_CIT0034","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1016\/j.jcp.2018.08.029","article-title":"DGM: A Deep Learning Algorithm for Solving Partial Differential Equations","volume":"375","author":"Sirignano","year":"2018","journal-title":"J. Comput. Phys."},{"key":"2019110410333066500_CIT0035","article-title":"A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations","author":"Nabian","year":"2018"},{"key":"2019110410333066500_CIT0036","unstructured":"Rudd, K.\n          , 2013, \u201cSolving Partial Differential Equations using Artificial Neural Networks,\u201d Ph.D. thesis, Duke University, Durham, NC."},{"issue":"4","key":"2019110410333066500_CIT0037","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1007\/s40304-017-0117-6","article-title":"Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations","volume":"5","author":"Weinan","year":"2017","journal-title":"Commun. Math. Stat."},{"issue":"34","key":"2019110410333066500_CIT0038","doi-asserted-by":"crossref","first-page":"8505","DOI":"10.1073\/pnas.1718942115","article-title":"Solving High-Dimensional Partial Differential Equations Using Deep Learning","volume":"115","author":"Han","year":"2018","journal-title":"Proc. Natl. Acad. Sci. U. S. A."},{"key":"2019110410333066500_CIT0039","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.neucom.2018.06.056","article-title":"A Unified Deep Artificial Neural Network Approach to Partial Differential Equations in Complex Geometries","volume":"317","author":"Berg","year":"2018","journal-title":"Neurocomputing"},{"key":"2019110410333066500_CIT0040","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1017\/jfm.2018.872","article-title":"Deep Learning of Vortex-Induced Vibrations","volume":"861","author":"Raissi","year":"2019","journal-title":"J. Fluid Mech."},{"key":"2019110410333066500_CIT0041","article-title":"Hidden Fluid Mechanics: A Navier\u2013Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data","author":"Raissi","year":"2018"},{"issue":"3","key":"2019110410333066500_CIT0042","doi-asserted-by":"crossref","first-page":"1118","DOI":"10.1137\/040615201","article-title":"High-order Collocation Methods for Differential Equations With Random Inputs","volume":"27","author":"Xiu","year":"2005","journal-title":"SIAM J. Sci. Comput."},{"key":"2019110410333066500_CIT0043","volume-title":"Stochastic Finite Elements: A Spectral Approach","author":"Ghanem","year":"2003"},{"key":"2019110410333066500_CIT0044","volume-title":"Monte Carlo: Concepts, Algorithms, and Applications","author":"Fishman","year":"2013"},{"key":"2019110410333066500_CIT0045","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1007\/978-3-642-35289-8_25","volume-title":"Neural Networks: Tricks of the Trade","author":"Bottou","year":"2012"},{"key":"2019110410333066500_CIT0046","article-title":"Adam: A Method for Stochastic Optimization","author":"Kingma","year":"2014"},{"issue":"Jul","key":"2019110410333066500_CIT0047","first-page":"2121","article-title":"Adaptive Subgradient Methods for Online Learning and Stochastic Optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"2019110410333066500_CIT0048","article-title":"Adadelta: An Adaptive Learning Rate Method","author":"Zeiler","year":"2012"},{"key":"2019110410333066500_CIT0049","first-page":"1139","article-title":"On the Importance of Initialization and Momentum in Deep Learning","author":"Sutskever","year":"2013"},{"key":"2019110410333066500_CIT0050","first-page":"1058","article-title":"Regularization of Neural Networks Using Dropconnect","author":"Wan","year":"2013"},{"key":"2019110410333066500_CIT0051","first-page":"402","article-title":"Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping","author":"Caruana","year":"2001","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3","key":"2019110410333066500_CIT0052","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/LSP.2017.2657381","article-title":"Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification","volume":"24","author":"Salamon","year":"2017","journal-title":"IEEE Signal Process Lett."},{"key":"2019110410333066500_CIT0053","article-title":"Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors","author":"Hinton","year":"2012"},{"key":"2019110410333066500_CIT0054","first-page":"8609","article-title":"Improving Deep Neural Networks for LVCSR using Rectified Linear Units and Dropout","author":"Dahl","year":"2013"},{"key":"2019110410333066500_CIT0055","first-page":"1097","article-title":"Imagenet Classification With Deep Convolutional Neural Networks","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2019110410333066500_CIT0056","first-page":"285","article-title":"Dropout Improves Recurrent Neural Networks for Handwriting Recognition","author":"Pham","year":"2014"},{"key":"2019110410333066500_CIT0057","first-page":"351","article-title":"Dropout Training as Adaptive Regularization","volume-title":"Advances in Neural Information Processing Systems","author":"Wager","year":"2013"},{"key":"2019110410333066500_CIT0058","first-page":"1","article-title":"Automatic Differentiation in Machine Learning: A Survey","volume":"18","author":"Baydin","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"2019110410333066500_CIT0059","doi-asserted-by":"crossref","DOI":"10.1137\/1.9780898717761","volume-title":"Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation","author":"Griewank","year":"2008"},{"key":"2019110410333066500_CIT0060","article-title":"Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations","author":"Raissi","year":"2018"},{"issue":"5","key":"2019110410333066500_CIT0061","doi-asserted-by":"crossref","first-page":"1743","DOI":"10.1137\/S1064827503430126","article-title":"An Extension of MATLAB to Continuous Functions and Operators","volume":"25","author":"Battles","year":"2004","journal-title":"SIAM J. Sci. Comp."},{"issue":"4","key":"2019110410333066500_CIT0062","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/0893-9659(95)00048-U","article-title":"Vorticity-velocity Formulation for the Stationary Navier\u2013Stokes Equations: The Three-Dimensional Case","volume":"8","author":"Medjo","year":"1995","journal-title":"Appl. Math. Lett."},{"issue":"2","key":"2019110410333066500_CIT0063","doi-asserted-by":"crossref","first-page":"2118","DOI":"10.1016\/j.jcp.2007.03.005","article-title":"The Immersed Boundary Method: A Projection Approach","volume":"225","author":"Taira","year":"2007","journal-title":"J. Comput. Phys."},{"issue":"25-28","key":"2019110410333066500_CIT0064","doi-asserted-by":"crossref","first-page":"2131","DOI":"10.1016\/j.cma.2007.08.014","article-title":"A Fast Immersed Boundary Method Using a Nullspace Approach and Multi-Domain Far-Field Boundary Conditions","volume":"197","author":"Colonius","year":"2008","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"2019110410333066500_CIT0065","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611974508","volume-title":"Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems","author":"Kutz","year":"2016"},{"key":"2019110410333066500_CIT0066","volume-title":"Computational Fluid Dynamics: A Practical Approach","author":"Tu","year":"2018"},{"issue":"4","key":"2019110410333066500_CIT0067","doi-asserted-by":"crossref","first-page":"04016102","DOI":"10.1061\/(ASCE)HY.1943-7900.0001275","article-title":"Multiphase Mesh-free Particle Method for Simulating Granular Flows and Sediment Transport","volume":"143","author":"Nabian","year":"2016","journal-title":"J. Hydraul. Eng."},{"key":"2019110410333066500_CIT0068","article-title":"Steady-State-Flow-with-Neural-Nets","author":"Hennigh","year":"2018"},{"issue":"1","key":"2019110410333066500_CIT0069","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1146\/annurev.fluid.30.1.329","article-title":"Lattice Boltzmann Method for Fluid Flows","volume":"30","author":"Chen","year":"1998","journal-title":"Annu. Rev. Fluid Mech."},{"key":"2019110410333066500_CIT0070","first-page":"770","article-title":"Deep Residual Learning for Image Recognition","author":"He","year":"2016"},{"key":"2019110410333066500_CIT0071","first-page":"4790","article-title":"Conditional Image Generation with PixelCNN Decoders","volume-title":"Advances in Neural Information Processing Systems","author":"van den Oord","year":"2016"},{"key":"2019110410333066500_CIT0072","article-title":"PixelCNN++: A PixelCNN Implementation With Discretized Logistic Mixture Likelihood and Other Modifications","author":"Salimans","year":"2017"},{"key":"2019110410333066500_CIT0073","article-title":"NIPS 2016 Tutorial: Generative Adversarial Networks","author":"Goodfellow","year":"2016"},{"key":"2019110410333066500_CIT0074","article-title":"Regularisation of Neural Networks by Enforcing Lipschitz Continuity","author":"Gouk","year":"2018"},{"key":"2019110410333066500_CIT0075","first-page":"5767","article-title":"Improved Training of Wasserstein Gans","volume-title":"Advances in Neural Information Processing Systems","author":"Gulrajani","year":"2017"},{"key":"2019110410333066500_CIT0076","article-title":"Spectral Normalization for Generative Adversarial Networks","author":"Miyato","year":"2018"},{"key":"2019110410333066500_CIT0077","article-title":"Towards Deep Neural Network Architectures Robust to Adversarial Examples","author":"Gu","year":"2014"},{"key":"2019110410333066500_CIT0078","article-title":"Explaining and Harnessing Adversarial Examples","author":"Goodfellow","year":"2014"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4044507\/6437409\/jcise_20_1_011006.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4044507\/6437409\/jcise_20_1_011006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,11,4]],"date-time":"2019-11-04T15:34:16Z","timestamp":1572881656000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4044507\/958395\/PhysicsDriven-Regularization-of-Deep-Neural"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,10]]},"references-count":78,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,2,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4044507","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,10]]},"article-number":"011006"}}