{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T12:41:53Z","timestamp":1778935313577,"version":"3.51.4"},"reference-count":23,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,3,12]],"date-time":"2020-03-12T00:00:00Z","timestamp":1583971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSTRF","award":["80NSSC19K1152"],"award-info":[{"award-number":["80NSSC19K1152"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This article presents a new methodology called Deep Theory of Functional Connections (TFC) that estimates the solutions of partial differential equations (PDEs) by combining neural networks with the TFC. The TFC is used to transform PDEs into unconstrained optimization problems by analytically embedding the PDE\u2019s constraints into a \u201cconstrained expression\u201d containing a free function. In this research, the free function is chosen to be a neural network, which is used to solve the now unconstrained optimization problem. This optimization problem consists of minimizing a loss function that is chosen to be the square of the residuals of the PDE. The neural network is trained in an unsupervised manner to minimize this loss function. This methodology has two major differences when compared with popular methods used to estimate the solutions of PDEs. First, this methodology does not need to discretize the domain into a grid, rather, this methodology can randomly sample points from the domain during the training phase. Second, after training, this methodology produces an accurate analytical approximation of the solution throughout the entire training domain. Because the methodology produces an analytical solution, it is straightforward to obtain the solution at any point within the domain and to perform further manipulation if needed, such as differentiation. In contrast, other popular methods require extra numerical techniques if the estimated solution is desired at points that do not lie on the discretized grid, or if further manipulation to the estimated solution must be performed.<\/jats:p>","DOI":"10.3390\/make2010004","type":"journal-article","created":{"date-parts":[[2020,3,13]],"date-time":"2020-03-13T08:58:59Z","timestamp":1584089939000},"page":"37-55","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":81,"title":["Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3750-5067","authenticated-orcid":false,"given":"Carl","family":"Leake","sequence":"first","affiliation":[{"name":"Aerospace Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0787-4547","authenticated-orcid":false,"given":"Daniele","family":"Mortari","sequence":"additional","affiliation":[{"name":"Aerospace Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1108\/eb032482","article-title":"Energy Theorems and Structural Analysis: A Generalized Discourse with Applications on Energy Principles of Structural Analysis Including the Effects of Temperature and Non-Linear Stress-Strain Relations","volume":"26","author":"Argyris","year":"1954","journal-title":"Aircr. Eng. Aerosp. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"805","DOI":"10.2514\/8.3664","article-title":"Stiffness and Deflection Analysis of Complex Structures","volume":"23","author":"Turner","year":"1956","journal-title":"J. Aeronaut. Sci."},{"key":"ref_3","unstructured":"Clough, R.W. (1960). The Finite Element Method in Plane Stress Analysis, American Society of Civil Engineers."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Spiliopoulos, J.S.K. (2018). DGM: A deep learning algorithm for solving partial differential equations. J. Comput. Phys., 1339\u20131364.","DOI":"10.1016\/j.jcp.2018.08.029"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1109\/72.712178","article-title":"Artificial neural networks for solving ordinary and partial differential equations","volume":"9","author":"Lagaris","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yadav, N., Yadav, A., and Kumar, M. (2015). An Introduction to Neural Network Methods for Differential Equations, Springer.","DOI":"10.1007\/978-94-017-9816-7"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Coons, S.A. (1967). Surfaces for Computer-Aided Design of Space Forms, Massachusetts Institute of Technology. Technical report.","DOI":"10.21236\/AD0663504"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mortari, D. (2017). The Theory of Connections: Connecting Points. Mathematics, 5.","DOI":"10.3390\/math5040057"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mortari, D., and Leake, C. (2019). The Multivariate Theory of Connections. Mathematics, 7.","DOI":"10.3390\/math7030296"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1058","DOI":"10.3390\/make1040060","article-title":"Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections","volume":"1","author":"Leake","year":"2019","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mortari, D. (2017). Least-squares Solutions of Linear Differential Equations. Mathematics, 5.","DOI":"10.3390\/math5040048"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mortari, D., Johnston, H., and Smith, L. (2018, January 8\u201312). Least-squares Solutions of Nonlinear Differential Equations. Proceedings of the 2018 AAS\/AIAA Space Flight Mechanics Meeting Conference, Kissimmee, FL, USA.","DOI":"10.2514\/6.2018-0959"},{"key":"ref_13","unstructured":"Johnston, H., and Mortari, D. (2018, January 19\u201323). Linear Differential Equations Subject to Relative, Integral, and Infinite Constraints. Proceedings of the 2018 AAS\/AIAA Astrodynamics Specialist Conference, Snowbird, UT, USA."},{"key":"ref_14","unstructured":"Leake, C., and Mortari, D. (2019, January 11\u201315). An Explanation and Implementation of Multivariate Theory of Connections via Examples. Proceedings of the 2019 AAS\/AIAA Astrodynamics Specialist Conference, AAS\/AIAA, Portland, MN, USA."},{"key":"ref_15","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, January 30). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: tensorflow.org."},{"key":"ref_16","unstructured":"Baydin, A.G., Pearlmutter, B.A., and Radul, A.A. (2015). Automatic differentiation in machine learning: A survey. arXiv."},{"key":"ref_17","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_18","first-page":"2121","article-title":"Adaptive Subgradient Methods for Online Learning and Stochastic Optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","unstructured":"Tieleman, T., and Hinton, G. (2012). Lecture 6.5\u2014RMSProp, COURSERA: Neural Networks for Machine Learning, University of Toronto. Technical report."},{"key":"ref_20","unstructured":"Fletcher, R. (1987). Practical Methods of Optimization, Wiley. [2nd ed.]."},{"key":"ref_21","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"Volume 9","author":"Teh","year":"2010","journal-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_23","unstructured":"Johnston, H., and Mortari, D. (2019). Least-squares solutions of boundary-value problems in hybrid systems. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:06:28Z","timestamp":1760173588000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,12]]},"references-count":23,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["make2010004"],"URL":"https:\/\/doi.org\/10.3390\/make2010004","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,12]]}}}