{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T21:58:18Z","timestamp":1785794298973,"version":"3.56.0"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T00:00:00Z","timestamp":1713312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1909457"],"award-info":[{"award-number":["1909457"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2011140"],"award-info":[{"award-number":["2011140"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2108479"],"award-info":[{"award-number":["2108479"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000893","name":"Simons Foundation","doi-asserted-by":"publisher","award":["fellowship"],"award-info":[{"award-number":["fellowship"]}],"id":[{"id":"10.13039\/100000893","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Appl. Math. Stat."],"abstract":"<jats:p>The learning speed of feed-forward neural networks is notoriously slow and has presented a bottleneck in deep learning applications for several decades. For instance, gradient-based learning algorithms, which are used extensively to train neural networks, tend to work slowly when all of the network parameters must be iteratively tuned. To counter this, both researchers and practitioners have tried introducing randomness to reduce the learning requirement. Based on the original construction of Igelnik and Pao, single layer neural-networks with random input-to-hidden layer weights and biases have seen success in practice, but the necessary theoretical justification is lacking. In this study, we begin to fill this theoretical gap. We then extend this result to the non-asymptotic setting using a concentration inequality for Monte-Carlo integral approximations. We provide a (corrected) rigorous proof that the Igelnik and Pao construction is a universal approximator for continuous functions on compact domains, with approximation error squared decaying asymptotically like <jats:italic>O<\/jats:italic>(1\/<jats:italic>n<\/jats:italic>) for the number <jats:italic>n<\/jats:italic> of network nodes. We then extend this result to the non-asymptotic setting, proving that one can achieve any desired approximation error with high probability provided <jats:italic>n<\/jats:italic> is sufficiently large. We further adapt this randomized neural network architecture to approximate functions on smooth, compact submanifolds of Euclidean space, providing theoretical guarantees in both the asymptotic and non-asymptotic forms. Finally, we illustrate our results on manifolds with numerical experiments.<\/jats:p>","DOI":"10.3389\/fams.2024.1284706","type":"journal-article","created":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T13:20:34Z","timestamp":1713360034000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Random vector functional link networks for function approximation on manifolds"],"prefix":"10.3389","volume":"10","author":[{"given":"Deanna","family":"Needell","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron A.","family":"Nelson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rayan","family":"Saab","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Palina","family":"Salanevich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olov","family":"Schavemaker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,4,17]]},"reference":[{"key":"B1","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/CVPR.2015.7298594","article-title":"Going deeper with convolutions","author":"Szegedy","year":"2015","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B3","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition"},{"key":"B4","doi-asserted-by":"publisher","first-page":"4700","DOI":"10.1109\/CVPR.2017.243","article-title":"Densely connected convolutional networks","author":"Huang","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B5","doi-asserted-by":"publisher","first-page":"2413","DOI":"10.1109\/CVPR.2018.00256","article-title":"Convolutional neural networks with alternately updated clique","author":"Yang","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B6","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1109\/18.256500","article-title":"Universal approximation bounds for superpositions of a sigmoidal function","volume":"39","author":"Barron","year":"1993","journal-title":"IEEE Trans Inf Theory"},{"key":"B7","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1006\/acha.1998.0248","article-title":"Harmonic analysis of neural networks","volume":"6","author":"Cand\u00e9s","year":"1999","journal-title":"Appl Comput Harm Analysis"},{"key":"B8","article-title":"Memory capacity of neural networks with threshold and ReLU activations","author":"Vershynin","year":"2020","journal-title":"arXiv preprint arXiv:200106938"},{"key":"B9","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.neunet.2019.04.009","article-title":"The capacity of feedforward neural networks","volume":"116","author":"Baldi","year":"2019","journal-title":"Neural Netw"},{"key":"B10","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1109\/72.655045","article-title":"Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions","volume":"9","author":"Huang","year":"1998","journal-title":"IEEE Trans Neural Netw"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1078","DOI":"10.1016\/j.asoc.2018.07.013","article-title":"Letter: On non-iterative learning algorithms with closed-form solution","volume":"70","author":"Suganthan","year":"2018","journal-title":"Appl Soft Comput"},{"key":"B12","first-page":"3623","article-title":"Modern Neural Networks Generalize on Small Data Sets","author":"Olson","year":"2018","journal-title":"Proceedings of the 32Nd International Conference on Neural Information Processing Systems. NIPS'18"},{"key":"B13","first-page":"1","article-title":"Feedforward neural networks with random weights","author":"Schmidt","year":"1992","journal-title":"II. Conference B: Pattern Recognition Methodology and Systems"},{"key":"B14","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/0952-1976(94)00056-S","article-title":"Random activation weight neural net (RAWN) for fast non-iterative training","volume":"8","author":"Te Braake","year":"1995","journal-title":"Eng Appl Artif Intell"},{"key":"B15","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1109\/2.144401","article-title":"Functional-link net computing: theory, system architecture, and functionalities","volume":"25","author":"Pao","year":"1992","journal-title":"Computer"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1320","DOI":"10.1109\/72.471375","article-title":"Stochastic choice of basis functions in adaptive function approximation and the functional-link net","volume":"6","author":"Igelnik","year":"1995","journal-title":"IEEE Trans Neur Netw"},{"key":"B17","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"B18","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1016\/0925-2312(94)90053-1","article-title":"Learning and generalization characteristics of the random vector functional-link net","volume":"6","author":"Pao","year":"1994","journal-title":"Neurocomputing"},{"key":"B19","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/0925-2312(95)00066-F","article-title":"The functional link net and learning optimal control","volume":"9","author":"Pao","year":"1995","journal-title":"Neurocomputing"},{"key":"B20","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1109\/3477.740166","article-title":"rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction","volume":"29","author":"Chen","year":"1999","journal-title":"IEEE Trans Syst Man Cybern B"},{"key":"B21","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/S0925-2312(99)00149-6","article-title":"Unconstrained word-based approach for off-line script recognition using density-based random-vector functional-link net","volume":"31","author":"Park","year":"2000","journal-title":"Neurocomputing"},{"key":"B22","doi-asserted-by":"publisher","first-page":"3243","DOI":"10.1109\/TCYB.2016.2588526","article-title":"Visual tracking with convolutional random vector functional link network","volume":"47","author":"Zhang","year":"2017","journal-title":"IEEE Trans Cybern"},{"key":"B23","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1109\/MCI.2017.2742867","article-title":"Benchmarking ensemble classifiers with novel co-trained kernel ridge regression and random vector functional link ensembles [research frontier]","volume":"12","author":"Zhang","year":"2017","journal-title":"IEEE Comput Intell Mag"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1146","DOI":"10.1016\/j.asoc.2017.09.020","article-title":"An ensemble of decision trees with random vector functional link networks for multi-class classification","volume":"70","author":"Katuwal","year":"2018","journal-title":"Appl Soft Comput"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1083","DOI":"10.1016\/j.asoc.2017.10.010","article-title":"A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression","volume":"70","author":"Vukovi\u0107","year":"2018","journal-title":"Appl Soft Comput"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1097","DOI":"10.1016\/j.asoc.2017.02.013","article-title":"non-iterative decomposition-ensemble learning paradigm using RVFL network for crude oil price forecasting","volume":"70","author":"Tang","year":"2018","journal-title":"Appl Soft Comput"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.1016\/j.asoc.2017.08.055","article-title":"Indian summer monsoon rainfall prediction: A comparison of iterative and non-iterative approaches","volume":"70","author":"Dash","year":"2018","journal-title":"Appl Soft Comput"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/IJCNN.2018.8489703","article-title":"Twitter sentiment classification based on deep random vector functional link","author":"Henr\u00edquez","year":"2018"},{"key":"B29","article-title":"Random vector functional link neural network based ensemble deep learning","author":"Katuwal","year":"2019","journal-title":"arXiv preprint arXiv:190700350"},{"key":"B30","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.neunet.2019.01.007","article-title":"An unsupervised parameter learning model for RVFL neural network","volume":"112","author":"Zhang","year":"2019","journal-title":"Neural Netw"},{"key":"B31","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/0893-6080(91)90009-T","article-title":"Approximation capabilities of multilayer feedforward networks","volume":"4","author":"Hornik","year":"1991","journal-title":"Neural Netw"},{"key":"B32","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/S0893-6080(05)80131-5","article-title":"Multilayer feedforward networks with a nonpolynomial activation function can approximate any function","volume":"6","author":"Leshno","year":"1993","journal-title":"Neural Netw"},{"key":"B33","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1109\/72.557702","article-title":"Comments on \u201cStochastic choice of basis functions in adaptive function approximation and the functional-link net\u201d [with reply]","volume":"8","author":"Li","year":"1997","journal-title":"IEEE Trans Neur Netw"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1847","DOI":"10.1158\/2159-8290.CD-21-0282","article-title":"Mapping phenotypic plasticity upon the cancer cell state landscape using manifold learning","volume":"12","author":"Burkhardt","year":"2022","journal-title":"Cancer Discov"},{"key":"B35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10827-022-00839-3","article-title":"Neural manifold analysis of brain circuit dynamics in health and disease","volume":"51","author":"Mitchell-Heggs","year":"2023","journal-title":"J Comput Neurosci"},{"key":"B36","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1017\/S0962492913000044","article-title":"High-dimensional integration: the quasi-Monte Carlo way","volume":"22","author":"Dick","year":"2013","journal-title":"Acta Numerica"},{"key":"B37","article-title":"The Concentration of Measure Phenomenon","author":"Ledoux","year":"2001"},{"key":"B38","author":"Massart","year":"1998","journal-title":"About the constants in Talagrand's deviation inequalities for empirical processes"},{"key":"B39","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1007\/s002220050108","article-title":"New concentration inequalities in product spaces","volume":"126","author":"Talagrand","year":"1996","journal-title":"Invent Mathem"},{"key":"B40","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1016\/j.acha.2016.04.003","article-title":"Provable approximation properties for deep neural networks","volume":"44","author":"Shaham","year":"2018","journal-title":"Appl Comput Harmon Anal"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-7400-6_3","author":"Tu","year":"2010","journal-title":"An Introduction to Manifolds"},{"key":"B42","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1016\/j.acha.2011.08.001","article-title":"Multi-scale geometric methods for data sets II: geometric multi-resolution analysis","volume":"32","author":"Allard","year":"2012","journal-title":"Appl Comput Harmon Anal"},{"key":"B43","doi-asserted-by":"publisher","first-page":"953","DOI":"10.1109\/SAMPTA.2017.8024465","article-title":"On recovery guarantees for one-bit compressed sensing on manifolds","volume":"65","author":"Iwen","year":"2018"},{"key":"B44","first-page":"43","article-title":"Multiscale dictionary learning: non-asymptotic bounds and robustness","volume":"17","author":"Maggioni","year":"2016","journal-title":"J Mach Learn Res"},{"key":"B45","first-page":"1","article-title":"Adaptive Geometric Multiscale Approximations for Intrinsically Low-dimensional Data","volume":"20","author":"Liao","year":"2019","journal-title":"J Mach Learn Res"},{"key":"B46","volume-title":"Introduction to Fourier Analysis on Euclidean Spaces","author":"Stein","year":"1971"},{"key":"B47","volume-title":"Functional Analysis","author":"Rudin","year":"1991"}],"container-title":["Frontiers in Applied Mathematics and Statistics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fams.2024.1284706\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T13:20:48Z","timestamp":1713360048000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fams.2024.1284706\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,17]]},"references-count":47,"alternative-id":["10.3389\/fams.2024.1284706"],"URL":"https:\/\/doi.org\/10.3389\/fams.2024.1284706","relation":{},"ISSN":["2297-4687"],"issn-type":[{"value":"2297-4687","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,17]]},"article-number":"1284706"}}