{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T19:37:29Z","timestamp":1775849849305,"version":"3.50.1"},"reference-count":27,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,10,9]],"date-time":"2019-10-09T00:00:00Z","timestamp":1570579200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NASA Space Technology Research Fellowships (NSTRF) 2019","award":["80NSSC19K1152"],"award-info":[{"award-number":["80NSSC19K1152"]}]},{"name":"NASA Space Technology Research Fellowships (NSTRF) 2019","award":["80NSSC19K1149"],"award-info":[{"award-number":["80NSSC19K1149"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Differential equations (DEs) are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. While there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Functional Connections (TFC) with one based on least-squares support vector machines (LS-SVM). The TFC method uses a constrained expression, an expression that always satisfies the DE constraints, which transforms the process of solving a DE into solving an unconstrained optimization problem that is ultimately solved via least-squares (LS). In addition to individual analysis, the two methods are merged into a new methodology, called constrained SVMs (CSVM), by incorporating the LS-SVM method into the TFC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: One first order linear ordinary differential equation (ODE), one first order nonlinear ODE, one second order linear ODE, and one two-dimensional linear partial differential equation (PDE). Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean squared error on the training and test sets. In general, TFC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) than the LS-SVM and CSVM approaches.<\/jats:p>","DOI":"10.3390\/make1040060","type":"journal-article","created":{"date-parts":[[2019,10,9]],"date-time":"2019-10-09T11:25:57Z","timestamp":1570620357000},"page":"1058-1083","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections"],"prefix":"10.3390","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3750-5067","authenticated-orcid":false,"given":"Carl","family":"Leake","sequence":"first","affiliation":[{"name":"Department of Aerospace Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8091-2987","authenticated-orcid":false,"given":"Hunter","family":"Johnston","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lidia","family":"Smith","sequence":"additional","affiliation":[{"name":"Mathematics Department, Blinn College, Bryan, TX 77802, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0787-4547","authenticated-orcid":false,"given":"Daniele","family":"Mortari","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/0771-050X(80)90013-3","article-title":"A Family of Embedded Runge-Kutta Formulae","volume":"6","author":"Dormand","year":"1980","journal-title":"J. Comp. Appl. Math."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1007\/BF03546367","article-title":"Implementation of Gauss-Jackson integration for orbit propagation","volume":"52","author":"Berry","year":"2004","journal-title":"J. Astronaut. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1007\/BF03321533","article-title":"Modified Chebyshev-Picard Iteration Methods for Orbit Propagation","volume":"58","author":"Bai","year":"2011","journal-title":"J. Astronaut. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1007\/s40295-015-0061-1","article-title":"Picard Iteration, Chebyshev Polynomials, and Chebyshev Picard Methods: Application in Astrodynamics","volume":"60","author":"Junkins","year":"2015","journal-title":"J. Astronaut. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1007\/s40295-015-0051-3","article-title":"State Transition Matrix for Perturbed Orbital Motion using Modified Chebyshev Picard Iteration","volume":"6","author":"Reed","year":"2015","journal-title":"J. Astronaut. Sci."},{"key":"ref_6","first-page":"108","article-title":"Rectangular spectral collocation","volume":"36","author":"Driscoll","year":"2016","journal-title":"IMA J. Numer. Anal."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mortari, D. (2017). The Theory of Connections: Connecting Points. Mathematics, 5.","DOI":"10.3390\/math5040057"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mortari, D. (2017). Least-squares Solutions of Linear Differential Equations. Mathematics, 5.","DOI":"10.3390\/math5040048"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.cam.2018.12.007","article-title":"High accuracy least-squares solutions of nonlinear differential equations","volume":"352","author":"Mortari","year":"2019","journal-title":"J. Comput. Appl. Math."},{"key":"ref_10","unstructured":"Johnston, H., and Mortari, D. (2018, January 19\u201323). Linear Differential Equations Subject to Relative, Integral, and Infinite Constraints. Proceedings of the 2018 AAS\/AIAA Astrodynamics Specialist Conference, Snowbird, UT, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Johnston, H., Leake, C., Efendiev, Y., and Mortari, D. (2019). Selected Applications of the Theory of Connections: A Technique for Analytical Constraint Embedding. Mathematics, 7.","DOI":"10.3390\/math7060537"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1356","DOI":"10.1109\/TNNLS.2012.2202126","article-title":"Approximate Solutions to Ordinary Differential Equations using Least-squares Support Vector Machines","volume":"23","author":"Mehrkanoon","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Freire, R.Z., Santos, G.H.d., and Coelho, L.d.S. (2017). Hygrothermal Dynamic and Mould Growth Risk Predictions for Concrete Tiles by Using Least Squares Support Vector Machines. Energies, 10.","DOI":"10.3390\/en10081093"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhao, X., Chen, X., Xu, Y., Xi, D., Zhang, Y., and Zheng, X. (2017). An EMD-Based Chaotic Least Squares Support Vector Machine Hybrid Model for Annual Runoff Forecasting. Water, 9.","DOI":"10.3390\/w9030153"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gedik, N. (2018). Least Squares Support Vector Mechanics to Predict the Stability Number of Rubble-Mound Breakwaters. Water, 10.","DOI":"10.3390\/w10101452"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gao, C., Xue, W., Ren, Y., and Zhou, Y. (2017). Numerical Control Machine Tool Fault Diagnosis Using Hybrid Stationary Subspace Analysis and Least Squares Support Vector Machine with a Single Sensor. Appl. Sci., 7.","DOI":"10.3390\/app7040346"},{"key":"ref_17","unstructured":"Vapnik, V.N. (1998). Statistical Learning Theory, Wiley."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kramer, M.A., Thompson, M.L., and Bhagat, P.M. (1992, January 24\u201326). Embedding Theoretical Models in Neural Networks. Proceedings of the 1992 American Control Conference, Chicago, IL, USA.","DOI":"10.23919\/ACC.1992.4792111"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pathak, D., Kr\u00e4henb\u00fchl, P., and Darrell, T. (2015, January 11\u201318). Constrained Convolutional Neural Networks for Weakly Supervised Segmentation. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.209"},{"key":"ref_20","unstructured":"M\u00e1rquez-Neila, P., Salzmann, M., and Fua, P. (2017). Imposing Hard Constraints on Deep Networks: Promises and Limitations. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lanczos, C. (1957). Applied Analysis. Progress in Industrial Mathematics at ECMI 2008, Dover Publications, Inc.. Chapter 7.","DOI":"10.1063\/1.3060402"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1093\/comjnl\/6.4.358","article-title":"Chebyshev Collocation Methods for Ordinary Differential Equations","volume":"6","author":"Wright","year":"1964","journal-title":"Comput. J."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mortari, D., and Leake, C. (2019). The Multivariate Theory of Connections. Mathematics, 7.","DOI":"10.3390\/math7030296"},{"key":"ref_24","unstructured":"Leake, C., and Mortari, D. (2019, January 11\u201315). An Explanation and Implementation of Multivariate Theory of Functional Connections via Examples. Proceedings of the 2019 AAS\/AIAA Astrodynamics Specialist Conference, Portland, ME, USA."},{"key":"ref_25","unstructured":"Theodoridis, S., and Koutroumbas, K. (2008). Pattern Recognition, Academic Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2502","DOI":"10.1016\/j.automatica.2012.06.095","article-title":"LS-SVM Approximate Solution to Linear Time Varying Descriptor Systems","volume":"48","author":"Mehrkanoon","year":"2012","journal-title":"Automatica"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.neucom.2015.02.013","article-title":"Learning Solutions to Partial Differential Equations using LS-SVM","volume":"159","author":"Mehrkanoon","year":"2015","journal-title":"Neurocomputing"}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/4\/60\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:28:43Z","timestamp":1760189323000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/4\/60"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,9]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["make1040060"],"URL":"https:\/\/doi.org\/10.3390\/make1040060","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,9]]}}}