{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:16:16Z","timestamp":1760217376958,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2015,2,12]],"date-time":"2015-02-12T00:00:00Z","timestamp":1423699200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Distributed data collection and analysis over a network are ubiquitous, especially over a wireless sensor network (WSN). To our knowledge, the data model used in most of the distributed algorithms is linear. However, in real applications, the linearity of systems is not always guaranteed. In nonlinear cases, the single hidden layer feedforward neural network (SLFN) with radial basis function (RBF) hidden neurons has the ability to approximate any continuous functions and, thus, may be used as the nonlinear learning system. However, confined by the communication cost, using the distributed version of the conventional algorithms to train the neural network directly is usually prohibited. Fortunately, based on the theorems provided in the extreme learning machine (ELM) literature, we only need to compute the output weights of the SLFN. Computing the output weights itself is a linear learning problem, although the input-output mapping of the overall SLFN is still nonlinear. Using the distributed algorithmto cooperatively compute the output weights of the SLFN, we obtain a distributed extreme learning machine (dELM) for nonlinear learning in this paper. This dELM is applied to the regression problem and classification problem to demonstrate its effectiveness and advantages.<\/jats:p>","DOI":"10.3390\/e17020818","type":"journal-article","created":{"date-parts":[[2015,2,12]],"date-time":"2015-02-12T12:08:24Z","timestamp":1423742904000},"page":"818-840","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Distributed Extreme Learning Machine for Nonlinear Learning over Network"],"prefix":"10.3390","volume":"17","author":[{"given":"Songyan","family":"Huang","sequence":"first","affiliation":[{"name":"Department of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3147-1553","authenticated-orcid":false,"given":"Chunguang","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,2,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1109\/TSP.2009.2033729","article-title":"Diffusion LMS strategies for distributed estimation","volume":"58","author":"Cattivelli","year":"2010","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1847","DOI":"10.1109\/JPROC.2010.2052531","article-title":"Gossip algorithms for distributed signal processing","volume":"98","author":"Dimakis","year":"2011","journal-title":"Proc. IEEE"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.ins.2012.02.008","article-title":"Distributed estimation over complex networks","volume":"197","author":"Liu","year":"2012","journal-title":"Inf. Sci"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2365","DOI":"10.1109\/TSP.2009.2016226","article-title":"Distributed LMS for consensus-based in-network adaptive processing","volume":"57","author":"Schizas","year":"2009","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4011","DOI":"10.1109\/TSP.2013.2265221","article-title":"Diffusion information theoretic learning for distributed estimation over network","volume":"61","author":"Li","year":"2013","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_6","unstructured":"Lopes, C.G., and Sayed, A.H. (2006, January 6\u20137). Distributed processing over adaptive networks. Lexington, MA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1465","DOI":"10.1109\/TSP.2010.2100386","article-title":"Analysis of spatial and incermental LMS processing for distribued estimation","volume":"59","author":"Cattivelli","year":"2011","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1109\/TSP.2007.907881","article-title":"The kernel least-mean-square algorithm","volume":"56","author":"Liu","year":"2008","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2165","DOI":"10.1109\/TSP.2004.830991","article-title":"Online learning with kernels","volume":"52","author":"Kivinen","year":"2004","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_10","unstructured":"Censor, Y., and Zenios, S.A. (1997). Parallel Optimization: Theory, Algorithms, and Applications, Oxford University Press."},{"key":"ref_11","unstructured":"Predd, J.B., Kulkarni, S.R., and Poor, H.V. (2006, January 13\u201317). Distributed kernel regression: An algorithm for training collaboratively. Punta del Este, Uruguay."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1856","DOI":"10.1109\/TIT.2009.2012992","article-title":"A collaborative training algorithm for distributed learning","volume":"55","author":"Predd","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1109\/LSP.2010.2040926","article-title":"Robust and low complexity distributed kernel least squares learning in sensor networks","volume":"17","author":"Kulkarni","year":"2010","journal-title":"IEEE Signal Process. Lett"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1109\/72.80341","article-title":"Orthogonal least squares learning algorithm for radial basis function networks","volume":"2","author":"Chen","year":"1991","journal-title":"IEEE Trans. Neural Netw"},{"key":"ref_15","unstructured":"Huang, G.B., Zhu, Q.Y., and Siew, C.K. (2004, January 25\u201329). Extreme learning machine: A new learning scheme of feedforward neural networks. Budapest, Hungary."},{"key":"ref_16","unstructured":"Huang, G.B., and Siew, C.K. (2004, January 6\u20139). Extreme learning machine: RBF network case. Automation, Robotics and Vision, Kunming, China."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1109\/TNN.2006.880583","article-title":"A fast and accurate online sequential learning algorithm for feedforward networks","volume":"17","author":"Liang","year":"2006","journal-title":"IEEE Trans. Neural Netw"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1109\/TSMCB.2011.2168604","article-title":"Extreme learning machine for regression and multiclass classification","volume":"42","author":"Huang","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. B"},{"key":"ref_20","unstructured":"Rao, C.R., and Mitra, S.K. (1971). Generalized Inverse of Matrices and its Applications, Wiley."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6217","DOI":"10.1109\/TSP.2012.2217338","article-title":"Diffusion strategies outperform consensus strategies for distributed estiamtion over adaptive networks","volume":"60","author":"Tu","year":"2012","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lopes, C.G., and Sayed, A.H. (2007, January 15\u201320). Diffusion least-mean squares over adaptive networks. Honolulu, HI, USA.","DOI":"10.1109\/ICASSP.2007.366830"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3122","DOI":"10.1109\/TSP.2008.917383","article-title":"Diffusion least-mean squares over adaptive networks: Formulation and performance analysis","volume":"56","author":"Lopes","year":"2008","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4480","DOI":"10.1109\/TSP.2012.2198468","article-title":"Diffusion sparse least-mean squares over networks","volume":"60","author":"Liu","year":"2012","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cattivelli, F.S., Lopes, C.G., and Sayed, A.H. (2007, January 17\u201320). A diffusion RLS scheme for distributed estimation over adaptive networks. In. Helsinki, Finland.","DOI":"10.1109\/SPAWC.2007.4401393"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1865","DOI":"10.1109\/TSP.2007.913164","article-title":"Diffusion recursive least-squares for distributed estimation over adaptive networks","volume":"56","author":"Cattivelli","year":"2008","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1109\/TSP.2014.2302731","article-title":"Distributed sparse recursive least-squares over networks","volume":"62","author":"Liu","year":"2014","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.sysconle.2004.02.022","article-title":"Fast linear iterations for distributed averaging","volume":"53","author":"Xiao","year":"2004","journal-title":"Syst. Control Lett"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Takahashi, N., Yamada, I., and Sayed, A.H. (2009, January 19\u201324). Diffusion least-mean squares with adaptive combiners. Taipei, Taiwan.","DOI":"10.1109\/ICASSP.2009.4960216"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4795","DOI":"10.1109\/TSP.2010.2051429","article-title":"Diffusion least-mean squares with adaptive combiners: formulation and performance analysis","volume":"58","author":"Takahashi","year":"2010","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Scherber, D.S., and Papadopoulos, H.C. (2004, January 26\u201327). Locally constructed algorithms for distributed computations in ad hoc networks. New York, NY, USA.","DOI":"10.1145\/984622.984625"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Haykin, S., and Ray Liu, K.J. (2009). Handbook on Array Processing and Sensor Networks, Wiley.","DOI":"10.1002\/9780470487068"},{"key":"ref_33","unstructured":"Blake, C., and Merz, C. Available Online: http:\/\/archive.ics.uci.edu\/ml\/datasets.html."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/2\/818\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:42:38Z","timestamp":1760215358000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/2\/818"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,2,12]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2015,2]]}},"alternative-id":["e17020818"],"URL":"https:\/\/doi.org\/10.3390\/e17020818","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2015,2,12]]}}}