{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:49:50Z","timestamp":1760143790754,"version":"build-2065373602"},"reference-count":58,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["71501138","71971151"],"award-info":[{"award-number":["71501138","71971151"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The echo state network (ESN) is a recurrent neural network that has yielded state-of-the-art results in many areas owing to its rapid learning ability and the fact that the weights of input neurons and hidden neurons are fixed throughout the learning process. However, the setting procedure for initializing the ESN\u2019s recurrent structure may lead to difficulties in designing a sound reservoir that matches a specific task. This paper proposes an improved pre-training method to adjust the model\u2019s parameters and topology to obtain an adaptive reservoir for a given application. Two strategies, namely global random selection and ensemble training, are introduced to pre-train the randomly initialized ESN model. Specifically, particle swarm optimization is applied to optimize chosen fixed and global weight values within the network, and the reliability and stability of the pre-trained model are enhanced by employing the ensemble training strategy. In addition, we test the feasibility of the model for time series prediction on six benchmarks and two real-life datasets. The experimental results show a clear enhancement in the ESN learning results. Furthermore, the proposed global random selection and ensemble training strategies are also applied to pre-train the extreme learning machine (ELM), which has a similar training process to the ESN model. Numerical experiments are subsequently carried out on the above-mentioned eight datasets. The experimental findings consistently show that the performance of the proposed pre-trained ELM model is also improved significantly. The suggested two strategies can thus enhance the ESN and ELM models\u2019 prediction accuracy and adaptability.<\/jats:p>","DOI":"10.3390\/e26030215","type":"journal-article","created":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T10:37:36Z","timestamp":1709116656000},"page":"215","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Ensemble and Pre-Training Approach for Echo State Network and Extreme Learning Machine Models"],"prefix":"10.3390","volume":"26","author":[{"given":"Lingyu","family":"Tang","sequence":"first","affiliation":[{"name":"School of Science, Civil Aviation Flight University of China, Guanghan 618307, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"Business School, Sichuan Normal University, Chengdu 610101, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mengyao","family":"Wang","sequence":"additional","affiliation":[{"name":"Business School, Sichuan Normal University, Chengdu 610101, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chunyu","family":"Zhao","sequence":"additional","affiliation":[{"name":"Business School, Sichuan Normal University, Chengdu 610101, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/j.energy.2016.12.033","article-title":"Day-ahead natural gas demand forecasting based on the combination of wavelet transform and ANFIS\/genetic algorithm\/neural network model","volume":"118","author":"Panapakidis","year":"2017","journal-title":"Energy"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"116778","DOI":"10.1016\/j.energy.2019.116778","article-title":"Effective energy consumption forecasting using enhanced bagged echo state network","volume":"193","author":"Hu","year":"2020","journal-title":"Energy"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"116552","DOI":"10.1016\/j.energy.2019.116552","article-title":"Nature-inspired metaheuristic ensemble model for forecasting energy consumption in residential buildings","volume":"191","author":"Tran","year":"2020","journal-title":"Energy"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.asoc.2017.01.022","article-title":"A new class of MODWT-SVM-DE hybrid model emphasizing on simplification structure in data pre-processing: A case study of annual electricity consumptions","volume":"54","author":"Sujjaviriyasup","year":"2017","journal-title":"Appl. Soft Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2617","DOI":"10.1016\/j.apm.2014.10.065","article-title":"A hybrid application algorithm based on the support vector machine and artificial intelligence: An example of electric load forecasting","volume":"39","author":"Chen","year":"2015","journal-title":"Appl. Math. Model."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.neucom.2020.01.031","article-title":"Empirical Mode Decomposition based Multi-objective Deep Belief Network for short-term power load forecasting","volume":"388","author":"Fan","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1016\/j.renene.2021.03.020","article-title":"Ultra-short-term wind speed forecasting using an optimized artificial intelligence algorithm","volume":"171","author":"Wang","year":"2021","journal-title":"Renew. Energy"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1016\/j.renene.2020.03.168","article-title":"Negative correlation learning-based RELM ensemble model integrated with OVMD for multi-step ahead wind speed forecasting","volume":"156","author":"Peng","year":"2020","journal-title":"Renew. Energy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"117449","DOI":"10.1016\/j.apenergy.2021.117449","article-title":"Ensemble wind speed forecasting with multi-objective Archimedes optimization algorithm and sub-model selection","volume":"301","author":"Zhang","year":"2021","journal-title":"Appl. Energy."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.atmosenv.2014.11.050","article-title":"Artificial intelligence based approach to forecast PM 2.5 during haze episodes: A case study of Delhi, India","volume":"102","author":"Mishra","year":"2015","journal-title":"Atmos. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.envsci.2021.06.011","article-title":"A Big Data and Artificial Intelligence Framework for Smart and Personalized Air Pollution Monitoring and Health Management in Hong Kong","volume":"124","author":"Li","year":"2021","journal-title":"Environ. Sci. Policy"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1805","DOI":"10.1038\/s41598-021-81455-6","article-title":"Artificial intelligence accuracy assessment in NO2 concentration forecasting of metropolises air","volume":"11","author":"Shams","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_13","first-page":"922","article-title":"Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting","volume":"33","author":"Guo","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"6505","DOI":"10.1109\/ACCESS.2019.2963784","article-title":"PSO-ELM: A Hybrid Learning Model for Short-Term Traffic Flow Forecasting","volume":"8","author":"Cai","year":"2020","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"85","DOI":"10.2991\/ijcis.d.200120.001","article-title":"A Hybrid Method for Traffic Flow Forecasting Using Multimodal Deep Learning","volume":"13","author":"Du","year":"2020","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110059","DOI":"10.1016\/j.chaos.2020.110059","article-title":"Applications of machine learning and artificial intelligence for COVID-19 (SARS-CoV-2) pandemic: A review","volume":"139","author":"Lalmuanawma","year":"2020","journal-title":"Chaos Solitons Fractals"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"110027","DOI":"10.1016\/j.chaos.2020.110027","article-title":"Forecasting Brazilian and American COVID-19 cases based on artificial intelligence coupled with climatic exogenous variables","volume":"139","author":"Ribeiro","year":"2020","journal-title":"Chaos Solitons Fractals"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.dsx.2020.04.012","article-title":"Artificial Intelligence (AI) applications for COVID-19 pandemic","volume":"14","author":"Vaishya","year":"2020","journal-title":"Diabetes Metab. Syndr. Clin. Res. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1609","DOI":"10.1007\/s00521-019-04212-x","article-title":"Stock price prediction based on deep neural networks","volume":"32","author":"Yu","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1007\/s13042-019-01041-1","article-title":"Study on the prediction of stock price based on the associated network model of LSTM","volume":"11","author":"Ding","year":"2020","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"115716","DOI":"10.1016\/j.eswa.2021.115716","article-title":"Prediction of stock price direction using a hybrid GA-XGBoost algorithm with a three-stage feature engineering process","volume":"186","author":"Yun","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1016\/j.jaci.2019.12.897","article-title":"Artificial intelligence approaches using natural language processing to advance EHR-based clinical research","volume":"145","author":"Juhn","year":"2020","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"101027","DOI":"10.1016\/j.aei.2019.101027","article-title":"Intelligent compilation of patent summaries using machine learning and natural language processing techniques","volume":"43","author":"Trappey","year":"2020","journal-title":"Adv. Eng. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"126062","DOI":"10.1016\/j.jhydrol.2021.126062","article-title":"Evolutionary artificial intelligence model via cooperation search algorithm and extreme learning machine for multiple scales nonstationary hydrological time series prediction","volume":"595","author":"Feng","year":"2021","journal-title":"J. Hydrol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1007\/s12145-020-00477-2","article-title":"A deep learning approach for hydrological time-series prediction: A case study of Gilgit river basin","volume":"13","author":"Hussain","year":"2020","journal-title":"Earth Sci. Inform."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1471","DOI":"10.1007\/s11600-019-00330-1","article-title":"Long short-term memory (LSTM) recurrent neural network for low-flow hydrological time series forecasting","volume":"67","author":"Sahoo","year":"2019","journal-title":"Acta Geophys."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.compbiolchem.2018.11.017","article-title":"Chaos enhanced grey wolf optimization wrapped ELM for diagnosis of paraquat-poisoned patients","volume":"78","author":"Zhao","year":"2019","journal-title":"Comput. Biol. Chem."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Muduli, D., Dash, R., and Majhi, B. (2020). Automated breast cancer detection in digital mammograms: A moth flame optimization based ELM approach. Biomed. Signal Process. Control, 59.","DOI":"10.1016\/j.bspc.2020.101912"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"24777","DOI":"10.1007\/s11042-021-10517-8","article-title":"Examination of the ECG signal classification technique DEA-ELM using deep convolutional neural network features","volume":"80","author":"Diker","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Virgeniya, S.C., and Ramaraj, E. (2021). A Novel Deep Learning based Gated Recurrent Unit with Extreme Learning Machine for Electrocardiogram (ECG) Signal Recognition. Biomed. Signal Process. Control, 68.","DOI":"10.1016\/j.bspc.2021.102779"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"110423","DOI":"10.1016\/j.chaos.2020.110423","article-title":"Mode decomposition method integrating mode reconstruction, feature extraction, and ELM for tourist arrival forecasting","volume":"143","author":"Lingyu","year":"2021","journal-title":"Chaos Solitons Fractals"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.tourman.2018.07.010","article-title":"Forecasting tourist arrivals with machine learning and internet search index","volume":"70","author":"Sun","year":"2019","journal-title":"Tour. Manag."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1251","DOI":"10.1007\/s00521-016-2754-1","article-title":"Extreme learning machine for structured output spaces","volume":"30","author":"Maliha","year":"2018","journal-title":"Neural Comput. Appl."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.neucom.2016.09.118","article-title":"Hardware architecture for large parallel array of Random Feature Extractors applied to image recognition","volume":"261","author":"Patil","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"102923","DOI":"10.1016\/j.annals.2020.102923","article-title":"Daily tourism volume forecasting for tourist attractions","volume":"83","author":"Bi","year":"2020","journal-title":"Ann. Tour. Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1016\/j.energy.2018.01.177","article-title":"Hourly day-ahead solar irradiance prediction using weather forecasts by LSTM","volume":"148","author":"Qing","year":"2018","journal-title":"Energy"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"106435","DOI":"10.1016\/j.cie.2020.106435","article-title":"An optimized model using LSTM network for demand forecasting","volume":"143","author":"Abbasimehr","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3286","DOI":"10.1109\/TIP.2019.2895466","article-title":"Hybrid LSTM and Encoder\u2013Decoder Architecture for Detection of Image Forgeries","volume":"28","author":"Bappy","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/ACCESS.2017.2778011","article-title":"Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features","volume":"6","author":"Ullah","year":"2018","journal-title":"IEEE Access"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ins.2020.03.034","article-title":"SiTGRU: Single-Tunnelled Gated Recurrent Unit for Abnormality Detection","volume":"524","author":"Fanta","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1109\/TETCI.2017.2762739","article-title":"Light Gated Recurrent Units for Speech Recognition","volume":"2","author":"Ravanelli","year":"2018","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1109\/TIE.2017.2733438","article-title":"Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit Networks","volume":"65","author":"Zhao","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"103271","DOI":"10.1016\/j.annals.2021.103271","article-title":"Multi-attraction, hourly tourism demand forecasting","volume":"90","author":"Zheng","year":"2021","journal-title":"Ann. Tour. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"114693","DOI":"10.1016\/j.eswa.2021.114693","article-title":"Driver stress detection via multimodal fusion using attention-based CNN-LSTM","volume":"173","author":"Mou","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"104785","DOI":"10.1016\/j.knosys.2019.05.028","article-title":"EA-LSTM: Evolutionary attention-based LSTM for time series prediction","volume":"181","author":"Li","year":"2019","journal-title":"Knowl. Based Syst."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3459","DOI":"10.1109\/TIP.2018.2818328","article-title":"Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection","volume":"27","author":"Song","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.neucom.2019.05.068","article-title":"Effects of singular value spectrum on the performance of echo state network","volume":"358","author":"Li","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.asoc.2018.08.024","article-title":"Stacked autoencoder with echo-state regression for tourism demand forecasting using search query data","volume":"73","author":"Lv","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1016\/j.renene.2020.09.109","article-title":"Wind speed forecasting based on variational mode decomposition and improved echo state network","volume":"164","author":"Hu","year":"2021","journal-title":"Renew. Energy"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ins.2016.08.081","article-title":"Functional echo state network for time series classification","volume":"373","author":"Ma","year":"2016","journal-title":"Inf. Sci."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.asoc.2017.01.049","article-title":"PSO-based analysis of Echo State Network parameters for time series forecasting","volume":"55","author":"Chouikhi","year":"2017","journal-title":"Appl. Soft Comput."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Ferreira, A.A., and Ludermir, T.B. (August, January 31). Comparing evolutionary methods for reservoir computing pre-training. Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.","DOI":"10.1109\/IJCNN.2011.6033233"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Basterrech, S., Alba, E., and Sn\u00e1\u0161el, V. (2015). An Experimental Analysis of the Echo State Network Initialization Using the Particle Swarm Optimization. arXiv.","DOI":"10.1109\/NaBIC.2014.6921880"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1126\/science.1091277","article-title":"Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication","volume":"304","author":"Jaeger","year":"2004","journal-title":"Science"},{"key":"ref_57","unstructured":"Kennedy, J., and Eberhart, R. (December, January 27). Particle swarm optimization. Proceedings of the ICNN\u201995\u2014International Conference on Neural Networks, Perth, WA, Australia."},{"key":"ref_58","unstructured":"Dua, D., and Graff, C. (2019). UCI Machine Learning Repository, University of California, School of Information and Computer Science. Available online: http:\/\/archive.ics.uci.edu\/ml."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/3\/215\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:06:38Z","timestamp":1760105198000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/3\/215"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,28]]},"references-count":58,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["e26030215"],"URL":"https:\/\/doi.org\/10.3390\/e26030215","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,2,28]]}}}