{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T15:30:29Z","timestamp":1786980629862,"version":"build-2736575974"},"reference-count":94,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-25-1-0042"],"award-info":[{"award-number":["FA9550-25-1-0042"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF-22-2-0158"],"award-info":[{"award-number":["W911NF-22-2-0158"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Sci. Comput."],"published-print":{"date-parts":[[2026,8,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>We present polynomial-augmented neural networks (PANNs), a novel machine learning architecture that combines deep neural networks (DNNs) with polynomial expansions. PANNs combine the strengths of DNNs (flexibility and efficiency in higher-dimensional approximation) with those of polynomial approximation (rapid convergence rates for smooth functions). To aid in both stable training and enhanced accuracy over a variety of problems, we present (1) a family of orthogonality constraints that impose mutual orthogonality between the polynomial and the DNN within a PANN; (2) a simple basis pruning approach to combat the curse of dimensionality introduced by the polynomial component; and (3) an adaptation of a polynomial preconditioning strategy to both DNNs and polynomials.\u00a0We test the resulting architecture for its polynomial reproduction properties, ability to approximate both smooth functions and functions of limited smoothness, and as a method for the solution of partial differential equations (PDEs).\u00a0Through these experiments, we demonstrate that PANNs offer superior approximation properties to DNNs for both regression and the numerical solution of PDEs while also offering enhanced accuracy over both polynomial and DNN-based regression (each) when regressing functions with limited smoothness.<\/jats:p>","DOI":"10.1137\/25m1773556","type":"journal-article","created":{"date-parts":[[2026,8,14]],"date-time":"2026-08-14T07:00:50Z","timestamp":1786690850000},"page":"C1086-C1135","source":"Crossref","is-referenced-by-count":0,"title":["Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation"],"prefix":"10.1137","volume":"48","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-6333-1239","authenticated-orcid":true,"given":"Madison","family":"Cooley","sequence":"first","affiliation":[{"name":"University of Utah, Salt Lake City, Utah."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shandian","family":"Zhe","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake City, Utah."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert M.","family":"Kirby","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake City, Utah."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Varun","family":"Shankar","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake City, Utah."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2026,8,14]]},"reference":[{"key":"ref1","first-page":"1","volume":"10","author":"Adcock B.","year":"2020","journal-title":"Inf. Inference"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-015-9276-6"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1017\/fms.2020.23"},{"key":"ref4","unstructured":"G. A. Barnett, A Robust RBF-FD Formulation based on Polyharmonic Splines and Polynomials, Ph.D. thesis, University of Colorado Boulder, 2015, https:\/\/scholar.colorado.edu\/concern\/graduate_thesis_or_dissertations\/8623hx72q."},{"key":"ref5","unstructured":"R. Basri, M. Galun, A. Geifman, D. Jacobs, Y. Kasten, and S. Kritchman, Frequency bias in neural networks for input of non-uniform density, in Proceedings of the 37th International Conference on Machine Learning, Vol. 119, PMLR, 13-18 Jul 2020, pp. 685\u2013694, https:\/\/proceedings.mlr.press\/v119\/basri20a.html."},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2018.12.029"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.12.013"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-021-01590-0"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1137\/110855442"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2015.05.134"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1007\/s00607-003-0016-4"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1137\/140970100"},{"key":"ref13","unstructured":"Y. Cheng, G. G. Chrysos, M. Georgopoulos, and V. Cevher, Multilinear Operator Networks, preprint, arXiv:2401.17992, 2024."},{"key":"ref14","unstructured":"P. Cheridito, A. Jentzen, and F. Rossmannek, Efficient Approximation of High-Dimensional Functions with Deep Neural Networks, https:\/\/arxiv.org\/abs\/1912.04310, 2019."},{"key":"ref15","first-page":"28390","volume":"34","author":"Chrysos G.","year":"2021","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref16","unstructured":"G. Chrysos, S. Moschoglou, Y. Panagakis, and S. Zafeiriou, Polygan:\u00a0High-Order Polynomial Generators, preprint, arXiv:1908.06571, 2019."},{"key":"ref17","first-page":"692","author":"Chrysos G. G.","year":"2022","journal-title":"European Conference on Computer Vision"},{"key":"ref18","first-page":"4021","volume":"44","author":"Chrysos G. G.","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"G. G. Chrysos, S. Moschoglou, G. Bouritsas, Y. Panagakis, J. Deng, and S. Zafeiriou, P-nets:\u00a0Deep polynomial neural networks, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 2180\u20132188, https:\/\/doi.org\/10.1109\/CVPR42600.2020.00735.","DOI":"10.1109\/CVPR42600.2020.00735"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"G. G. Chrysos, B. Wang, J. Deng, and V. Cevher, Regularization of polynomial networks for image recognition, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 16123\u201316132.","DOI":"10.1109\/CVPR52729.2023.01547"},{"key":"ref21","author":"Cooley M.","year":"2025","journal-title":"Trans. Mach. Learn. Res."},{"key":"ref22","first-page":"512","volume-title":"Mathematical and Scientific Machine Learning, PMLR","author":"Cyr E. C.","year":"2020"},{"key":"ref23","volume-title":"Methods of Numerical Integration","author":"Davis P. J.","year":"2007"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1016\/j.acha.2007.03.003"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1007\/s00365-021-09558-x"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.jco.2015.09.006"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"D. Dung, V. Temlyakov, and T. Ullrich, Hyperbolic Cross Approximation, arXiv:\u00a0Numerical Analysis, 2016, https:\/\/doi.org\/10.1007\/978-3-319-92240-9, https:\/\/arxiv.org\/abs\/1604.08246.","DOI":"10.1007\/978-3-319-92240-9"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1108\/02644400110387145"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2021.3062161"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1142\/6437"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2016.05.026"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1063\/5.0130803"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1023\/A:1019129717644"},{"key":"ref34","first-page":"1","author":"Goyal M.","year":"2020","journal-title":"2020 International Joint Conference on Neural Networks (IJCNN)"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/aa9a90"},{"key":"ref36","doi-asserted-by":"crossref","unstructured":"K. He, X. Zhang, S. Ren, and J. Sun, Delving deep into rectifiers:\u00a0Surpassing human-level performance on imagenet classification, in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026\u20131034.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref37","doi-asserted-by":"crossref","unstructured":"Z. Hu, K. Shukla, G. Karniadakis, and K. Kawaguchi, Tackling the curse of dimensionality with physics-informed neural networks, arXiv, abs\/2307.12306, https:\/\/doi.org\/10.48550\/arXiv.2307.12306, 2023.","DOI":"10.2139\/ssrn.4641406"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106369"},{"key":"ref39","unstructured":"S. Ioffe and C. Szegedy, Batch normalization:\u00a0Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research 37, F. Bach and D. Blei, eds. PMLR, Lille, France, 2015, pp. 448\u2013456, https:\/\/doi.org\/10.48550\/arXiv.1502.03167."},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1137\/16M1063885"},{"key":"ref41","doi-asserted-by":"crossref","unstructured":"A. Jentzen, D. Salimova, and T. Welti, A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients, arXiv abs\/1809.07321, 2018, https:\/\/doi.org\/10.4310\/CMS.2021.v19.n5.a1.","DOI":"10.4310\/CMS.2021.v19.n5.a1"},{"key":"ref42","unstructured":"J. Jiang, Y. Chen, and A. Narayan, A Unified, Goal-Oriented, Hybridized Reduced Basis Method and Generalized Polynomial Chaos Algorithm for Partial Differential Equations with Random Inputs, arXiv:\u00a0Numerical Analysis, (2016), https:\/\/arxiv.org\/abs\/1611.01385."},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03819-2"},{"key":"ref44","volume-title":"Adv. Neural Inform. Process. Syst.","volume":"32","author":"Kileel J.","year":"2019"},{"key":"ref45","unstructured":"D. P. Kingma and J. Ba, Adam:\u00a0A method for stochastic optimization, in 3rd International Conference on Learning Representations, ICLR Conference Track Proceedings, Y. Bengio and Y. LeCun, eds. 2015, https:\/\/doi.org\/10.48550\/arXiv.1412.6980, http:\/\/arxiv.org\/abs\/1412.6980."},{"key":"ref46","unstructured":"A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, and M. W. Mahoney, Characterizing possible failure modes in physics-informed neural networks, Adv. Neural Inform. Process. Syst., 34 (2021), pp. 26548\u201326560, https:\/\/doi.org\/10.48550\/arXiv.2011.05526."},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1126\/science.adi2336"},{"key":"ref48","first-page":"1","volume":"10","author":"Larochelle H.","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1016\/j.saa.2013.05.068"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80131-5"},{"key":"ref51","doi-asserted-by":"crossref","unstructured":"B. Li, S. Tang, and H. Yu, Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units, preprint, arXiv:1903.05858, https:\/\/doi.org\/10.48550\/arXiv.1903.05858, 2019.","DOI":"10.4208\/cicp.OA-2019-0168"},{"key":"ref52","unstructured":"B. Li, S. Tang, and H. Yu, Powernet:\u00a0Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units, preprint, arXiv:1909.05136, https:\/\/arxiv.org\/abs\/1909.05136, 2019."},{"key":"ref53","doi-asserted-by":"crossref","unstructured":"C. Li and B. Adcock, Compressed sensing with local structure:\u00a0Uniform recovery guarantees for the sparsity in levels class, arXiv, https:\/\/doi.org\/10.1016\/J.ACHA.2017.05.006, https:\/\/arxiv.org\/abs\/1601.01988, (2016).","DOI":"10.1016\/j.acha.2017.05.006"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2014.10.035"},{"key":"ref55","unstructured":"Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Fourier Neural Operator for Parametric Partial Differential Equations, preprint, arXiv:2010.08895, 2020."},{"key":"ref56","doi-asserted-by":"publisher","DOI":"10.1016\/j.acha.2021.12.009"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1007\/BF01589116"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2023.3264615"},{"key":"ref59","unstructured":"I. Loshchilov and F. Hutter, Sgdr:\u00a0Stochastic gradient descent with warm restarts, in International Conference on Learning Representations, 2016, https:\/\/doi.org\/10.48550\/arXiv.1608.03983, https:\/\/arxiv.org\/abs\/1608.03983."},{"key":"ref60","doi-asserted-by":"crossref","unstructured":"A. Masud and T. J. Hughes, A stabilized mixed finite element method for darcy flow, Comput. Methods Appl. Mech. Engrg., 191 (2002), pp. 4341\u20134370.","DOI":"10.1016\/S0045-7825(02)00371-7"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2020.3035424"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1016\/j.jco.2015.02.001"},{"key":"ref63","unstructured":"V. Nair and G. E. Hinton, Rectified linear units improve restricted boltzmann machines, in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 807\u2013814, https:\/\/icml.cc\/Conferences\/2010\/papers\/432.pdf."},{"key":"ref64","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2005.07.009"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1090\/mcom\/3192"},{"key":"ref66","doi-asserted-by":"publisher","DOI":"10.1142\/S0219530519410136"},{"key":"ref67","unstructured":"J. A. Opschoor and C. Schwab, Deep Relu Networks and High-Order Finite Element Methods II:\u00a0Chebyshev emulation, preprint, arXiv:2310.07261, https:\/\/arxiv.org\/abs\/2310.07261, 2023."},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-7152(96)00140-X"},{"key":"ref69","unstructured":"R. Pascanu, T. Mikolov, and Y. Bengio, Understanding the exploding gradient problem, arXiv, abs\/1211.5063, https:\/\/arxiv.org\/abs\/1211.5063, 2012."},{"key":"ref70","unstructured":"A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et\u00a0al., Pytorch:\u00a0An imperative style, high-performance deep learning library, 2019, https:\/\/pytorch.org."},{"key":"ref71","doi-asserted-by":"publisher","DOI":"10.1007\/s002110200401"},{"key":"ref72","unstructured":"N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, On the spectral bias of neural networks, in International Conference on Machine Learning, PMLR, 2019, pp. 5301\u20135310, https:\/\/proceedings.mlr.press\/v97\/rahaman19a.html."},{"key":"ref73","first-page":"932","volume":"19","author":"Raissi M.","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref74","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.10.045"},{"key":"ref75","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2017.04.037"},{"key":"ref76","doi-asserted-by":"publisher","DOI":"10.1137\/17M114090X"},{"key":"ref77","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110633"},{"key":"ref78","unstructured":"G. Shen, Y. Jiao, Y. Lin, and J. Huang, Differentiable Neural Networks with Repu Activation:\u00a0With Applications to Score Estimation and Isotonic Regression, preprint, arXiv:2305.00608, https:\/\/doi.org\/10.48550\/arXiv.2305.00608, 2023."},{"key":"ref79","doi-asserted-by":"publisher","DOI":"10.1137\/090765547"},{"key":"ref80","doi-asserted-by":"publisher","DOI":"10.1006\/jsco.1996.0052"},{"key":"ref81","first-page":"1929","volume":"15","author":"Srivastava N.","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref82","doi-asserted-by":"publisher","DOI":"10.1137\/16M1055992"},{"key":"ref83","unstructured":"S. Wang, B. Li, Y. Chen, and P. Perdikaris, Piratenets:\u00a0Physics-Informed Deep Learning with Residual Adaptive Networks, preprint, arXiv:2402.00326, 2024."},{"key":"ref84","unstructured":"S. Wang, S. Sankaran, and P. Perdikaris, Respecting causality is all you need for training physics-informed neural networks, https:\/\/arxiv.org\/abs\/2203.07404, 2022."},{"key":"ref85","doi-asserted-by":"crossref","unstructured":"S. Wang, S. Sankaran, H. Wang, and P. Perdikaris, An Expert\u2019s Guide to Training Physics-Informed Neural Networks, preprint, arXiv:2308.08468, 2023.","DOI":"10.1016\/j.cma.2024.116813"},{"key":"ref86","doi-asserted-by":"publisher","DOI":"10.1137\/20M1318043"},{"key":"ref87","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2021.113938"},{"key":"ref88","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA6297"},{"key":"ref89","unstructured":"Y. Wu, G. G. Chrysos, and V. Cevher, Adversarial Audio Synthesis with Complex-Valued Polynomial Networks, preprint, arXiv:2206.06811, 2022."},{"key":"ref90","unstructured":"Z.Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, and Z. Ma, Frequency Principle:\u00a0Fourier Analysis Sheds Light on Deep Neural Networks, preprint, arXiv:1901.06523, https:\/\/arxiv.org\/abs\/1901.06523, 2019."},{"key":"ref91","doi-asserted-by":"publisher","DOI":"10.1109\/TAP.2017.2676724"},{"key":"ref92","doi-asserted-by":"publisher","DOI":"10.52202\/068431-0318"},{"key":"ref93","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.15.240"},{"key":"ref94","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.120.143001"}],"container-title":["SIAM Journal on Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/25M1773556","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T15:15:29Z","timestamp":1786979729000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/25M1773556"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,14]]},"references-count":94,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,8,31]]}},"alternative-id":["10.1137\/25M1773556"],"URL":"https:\/\/doi.org\/10.1137\/25m1773556","relation":{},"ISSN":["1064-8275","1095-7197"],"issn-type":[{"value":"1064-8275","type":"print"},{"value":"1095-7197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,14]]}}}