{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T14:06:27Z","timestamp":1779890787631,"version":"3.53.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"20","license":[{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100009042","name":"Universidad de Sevilla","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100009042","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Echo state networks (ESNs) belong to the class of recurrent neural networks and have demonstrated robust performance in time series prediction tasks. In this study, we investigate the capability of different ESN architectures to capture spatial relationships in images without transforming them into temporal sequences. We begin with three pre-existing ESN-based architectures and enhance their design by incorporating multiple output layers, customising them for a classification task. Our investigation involves an examination of the behaviour of these modified networks, coupled with a comprehensive performance comparison against the baseline vanilla ESN architecture. Our experiments on the MNIST data set reveal that a network with multiple independent reservoirs working in parallel outperforms other ESN-based architectures for this task, achieving a classification accuracy of 98.43%. This improvement on the classical ESN architecture is accompanied by reduced training times. While the accuracy of ESN-based architectures lags behind that of convolutional neural network-based architectures, the significantly lower training times of ESNs with multiple reservoirs operating in parallel make them a compelling choice for learning spatial relationships in scenarios prioritising energy efficiency and rapid training. This multi-reservoir ESN architecture overcomes standard ESN limitations regarding memory requirements and training times for large networks, providing more accurate predictions than other ESN-based models. These findings contribute to a deeper understanding of the potential of ESNs as a tool for image classification.<\/jats:p>","DOI":"10.1007\/s00521-024-09656-4","type":"journal-article","created":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T15:01:51Z","timestamp":1713452511000},"page":"11901-11918","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Exploring deep echo state networks for image classification: a multi-reservoir approach"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0943-9032","authenticated-orcid":false,"given":"E. J.","family":"L\u00f3pez-Ortiz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Perea-Trigo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"L. M.","family":"Soria-Morillo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"F.","family":"Sancho-Caparrini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J. J.","family":"Vegas-Olmos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,18]]},"reference":[{"key":"9656_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2023.106271","volume":"123","author":"Y Guo","year":"2023","unstructured":"Guo Y, Yu H, Ma L, Zeng L, Luo X (2023) Thfe: a triple-hierarchy feature enhancement method for tiny boat detection. Eng Appl Artif Intell 123:106271. https:\/\/doi.org\/10.1016\/j.engappai.2023.106271","journal-title":"Eng Appl Artif Intell"},{"key":"9656_CR2","doi-asserted-by":"publisher","unstructured":"Lei Q, Guo Y, Ma L, Luo X (2023) Few-shot object detection via instance-wise and prototypical contrastive learning. In: Chang S (ed) The 35th international conference on software engineering and knowledge engineering, SEKE 2023, KSIR virtual conference center, USA, July 1\u201310. KSI Research Inc., pp 685\u2013690. https:\/\/doi.org\/10.18293\/SEKE2023-129","DOI":"10.18293\/SEKE2023-129"},{"issue":"22","key":"9656_CR3","doi-asserted-by":"publisher","first-page":"20165","DOI":"10.1007\/s00521-022-07577-8","volume":"34","author":"U Bhimavarapu","year":"2022","unstructured":"Bhimavarapu U (2022) Irf-lstm: enhanced regularization function in lstm to predict the rainfall. Neural Comput Appl 34(22):20165\u201320177. https:\/\/doi.org\/10.1007\/s00521-022-07577-8","journal-title":"Neural Comput Appl"},{"issue":"18","key":"9656_CR4","doi-asserted-by":"publisher","first-page":"13597","DOI":"10.1007\/s00521-023-08450-y","volume":"35","author":"ME Sahin","year":"2023","unstructured":"Sahin ME, Ulutas H, Yuce E, Erkoc MF (2023) Detection and classification of Covid-19 by using faster r-cnn and mask r-cnn on ct images. Neural Comput Appl 35(18):13597\u201313611. https:\/\/doi.org\/10.1007\/s00521-023-08450-y","journal-title":"Neural Comput Appl"},{"issue":"28","key":"9656_CR5","doi-asserted-by":"publisher","first-page":"20631","DOI":"10.1007\/s00521-023-08859-5","volume":"35","author":"ASN Raju","year":"2023","unstructured":"Raju ASN, Jayavel K, Rajalakshmi T (2023) An advanced diagnostic colorectalcadx utilises cnn and unsupervised visual explanations to discover malignancies. Neural Comput Appl 35(28):20631\u201320662. https:\/\/doi.org\/10.1007\/s00521-023-08859-5","journal-title":"Neural Comput Appl"},{"key":"9656_CR6","doi-asserted-by":"publisher","unstructured":"Beohar D, Rasool A (2021) Handwritten digit recognition of mnist dataset using deep learning state-of-the-art artificial neural network (ann) and convolutional neural network (cnn). In: 2021 International conference on emerging smart computing and informatics (ESCI), pp 542\u2013548. https:\/\/doi.org\/10.1109\/ESCI50559.2021.9396870","DOI":"10.1109\/ESCI50559.2021.9396870"},{"key":"9656_CR7","doi-asserted-by":"publisher","unstructured":"Kaziha O, Bonny T (2019) A comparison of quantized convolutional and lstm recurrent neural network models using mnist. In: 2019 International conference on electrical and computing technologies and applications (ICECTA), pp 1\u20135. https:\/\/doi.org\/10.1109\/ICECTA48151.2019.8959793","DOI":"10.1109\/ICECTA48151.2019.8959793"},{"issue":"101","key":"9656_CR8","first-page":"102","volume":"100","author":"C Coleman","year":"2017","unstructured":"Coleman C, Narayanan D, Kang D, Zhao T, Zhang J, Nardi L, Bailis P, Olukotun K, R\u00e9 C, Zaharia M (2017) Dawnbench: an end-to-end deep learning benchmark and competition. Training 100(101):102","journal-title":"Training"},{"key":"9656_CR9","unstructured":"Jaeger H (2001) The \u201cecho state\u201d approach to analysing and training recurrent neural networks-with an erratum note\u2019. www.researchgate.net"},{"key":"9656_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-020-04948-x","author":"T Kim","year":"2020","unstructured":"Kim T, King B (2020) Time series prediction using deep echo state networks. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-020-04948-x","journal-title":"Neural Comput Appl"},{"key":"9656_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.mlwa.2022.100300","volume":"8","author":"S Shahi","year":"2022","unstructured":"Shahi S, Fenton FH, Cherry EM (2022) Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: a comparative study. Mach Learn Appl 8:100300. https:\/\/doi.org\/10.1016\/j.mlwa.2022.100300","journal-title":"Mach Learn Appl"},{"key":"9656_CR12","doi-asserted-by":"publisher","unstructured":"Luko\u0161evi\u010dius M (2012) In: Montavon G, Orr GB, M\u00fcller K-R (eds) A practical guide to applying echo state networks. Springer, Berlin, Heidelberg, pp 659\u2013686. https:\/\/doi.org\/10.1007\/978-3-642-35289-8_36","DOI":"10.1007\/978-3-642-35289-8_36"},{"key":"9656_CR13","doi-asserted-by":"publisher","unstructured":"Tian Z, Gao X, Li S, Wang Y (2015) Prediction method for network traffic based on genetic algorithm optimized echo state network. Jisuanji Yanjiu yu Fazhan\/Comput Res Dev 52:1137\u20131145. https:\/\/doi.org\/10.7544\/issn1000-1239.2015.20131757","DOI":"10.7544\/issn1000-1239.2015.20131757"},{"key":"9656_CR14","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1142\/S012906572150057X","volume":"31","author":"Y Xue","year":"2021","unstructured":"Xue Y, Zhang Q, Neri F (2021) Self-adaptive particle swarm optimization-based echo state network for time series prediction. Int J Neural Syst 31:234. https:\/\/doi.org\/10.1142\/S012906572150057X","journal-title":"Int J Neural Syst"},{"key":"9656_CR15","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.neunet.2019.02.001","volume":"115","author":"LA Thiede","year":"2019","unstructured":"Thiede LA, Parlitz U (2019) Gradient based hyperparameter optimization in echo state networks. Neural Netw 115:23\u201329. https:\/\/doi.org\/10.1016\/j.neunet.2019.02.001","journal-title":"Neural Netw"},{"key":"9656_CR16","doi-asserted-by":"publisher","unstructured":"Jaeger H, Luko\u0161evi\u010dius M, Popovici D, Siewert U (2007) Optimization and applications of echo state networks with leaky- integrator neurons. Neural Netw 20(3):335\u2013352. https:\/\/doi.org\/10.1016\/j.neunet.2007.04.016. Echo State Networks and Liquid State Machines","DOI":"10.1016\/j.neunet.2007.04.016"},{"key":"9656_CR17","doi-asserted-by":"publisher","unstructured":"Shrivastava H, Garg A, Cao Y, Zhang Y, Sainath T (2021) Echo state speech recognition.https:\/\/doi.org\/10.48550\/ARXIV.2102.09114","DOI":"10.48550\/ARXIV.2102.09114"},{"key":"9656_CR18","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1016\/j.neunet.2007.04.013","volume":"20","author":"M Tong","year":"2007","unstructured":"Tong M, Bickett A, Christiansen E, Cottrell G (2007) Learning grammatical structure with echo state networks. Neural Netw 20:424\u2013432. https:\/\/doi.org\/10.1016\/j.neunet.2007.04.013","journal-title":"Neural Netw"},{"key":"9656_CR19","doi-asserted-by":"publisher","unstructured":"Cabessa J, Hernault H, Kim H, Lamonato Y, Levy YZ (2021) Efficient text classification with echo state networks. In 2021 International joint conference on neural networks (IJCNN), pp 1\u20138 https:\/\/doi.org\/10.1109\/IJCNN52387.2021.9533958","DOI":"10.1109\/IJCNN52387.2021.9533958"},{"key":"9656_CR20","doi-asserted-by":"publisher","first-page":"1953","DOI":"10.1109\/ROBOT.2005.1570399","volume":"18","author":"M Salmen","year":"2005","unstructured":"Salmen M, Pl\u00f6ger P (2005) Echo state networks used for motor control. Robot Autom 18:1953\u20131958. https:\/\/doi.org\/10.1109\/ROBOT.2005.1570399","journal-title":"Robot Autom"},{"key":"9656_CR21","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-018-3747-z","author":"Q Chen","year":"2020","unstructured":"Chen Q, Zhang A, Huang T, He Q, Song Y (2020) Imbalanced dataset-based echo state networks for anomaly detection. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-018-3747-z","journal-title":"Neural Comput Appl"},{"key":"9656_CR22","doi-asserted-by":"publisher","unstructured":"Sun J, Li L, Peng H (2021) An image classification method based on echo state network. In: 2021 International conference on neuromorphic computing (ICNC), pp 165\u2013170 https:\/\/doi.org\/10.1109\/ICNC52316.2021.9607999","DOI":"10.1109\/ICNC52316.2021.9607999"},{"key":"9656_CR23","unstructured":"Krusna Lukosevicius M (2018) Predicting Mozart\u2019s next note via echo state networks. www.piano-midi.de"},{"key":"9656_CR24","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1016\/j.neunet.2015.08.010","volume":"71","author":"FM Bianchi","year":"2015","unstructured":"Bianchi FM, Scardapane S, Uncini A, Rizzi A, Sadeghian A (2015) Prediction of telephone calls load using echo state network with exogenous variables. Neural Netw 71:204\u2013213. https:\/\/doi.org\/10.1016\/j.neunet.2015.08.010","journal-title":"Neural Netw"},{"key":"9656_CR25","doi-asserted-by":"publisher","unstructured":"Guo X, Qian Y, Tiwari P, Zou Q, Ding Y (2022) Kernel risk sensitive loss-based echo state networks for predicting therapeutic peptides with sparse learning. In: 2022 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 6\u201311 . https:\/\/doi.org\/10.1109\/BIBM55620.2022.9994902","DOI":"10.1109\/BIBM55620.2022.9994902"},{"key":"9656_CR26","unstructured":"Sun C, Song M, Hong S, Li H (2020) A review of designs and applications of echo state networks. http:\/\/arxiv.org\/abs\/2012.02974"},{"key":"9656_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.12.089","author":"C Gallicchio","year":"2017","unstructured":"Gallicchio C, Micheli A, Pedrelli L (2017) Deep reservoir computing: a critical experimental analysis. Neurocomputing. https:\/\/doi.org\/10.1016\/j.neucom.2016.12.089","journal-title":"Neurocomputing"},{"key":"9656_CR28","doi-asserted-by":"publisher","first-page":"5719","DOI":"10.1002\/cpe.5719","volume":"32","author":"A Souahlia","year":"2020","unstructured":"Souahlia A, Belatreche A, Benyettou A, Foitih Z, Benkhelifa E, Curran K (2020) Echo state network-based feature extraction for efficient color image segmentation. Concurr Comput Pract Exp 32:5719. https:\/\/doi.org\/10.1002\/cpe.5719","journal-title":"Concurr Comput Pract Exp"},{"key":"9656_CR29","doi-asserted-by":"publisher","unstructured":"Yang Y, Zhao X, Liu X (2020) A novel exhaust gas temperature prediction method of hot blast stove. In: 2020 39th Chinese control conference (CCC), pp 5916\u20135921. https:\/\/doi.org\/10.23919\/CCC50068.2020.9189443","DOI":"10.23919\/CCC50068.2020.9189443"},{"key":"9656_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.seta.2022.102275","volume":"52","author":"M Mustaqeem Ishaq","year":"2022","unstructured":"Mustaqeem Ishaq M, Kwon S (2022) A cnn-assisted deep echo state network using multiple time-scale dynamic learning reservoirs for generating short-term solar energy forecasting. Sustain Energy Technol Assessm 52:102275. https:\/\/doi.org\/10.1016\/j.seta.2022.102275","journal-title":"Sustain Energy Technol Assessm"},{"key":"9656_CR31","doi-asserted-by":"publisher","unstructured":"Schaetti N, Salomon M, Couturier R (2016) Echo state networks-based reservoir computing for mnist handwritten digits recognition. In: 2016 IEEE intl conference on computational science and engineering (CSE) and IEEE intl conference on embedded and ubiquitous computing (EUC) and 15th intl symposium on distributed computing and applications for business engineering (DCABES), pp 484\u2013491. https:\/\/doi.org\/10.1109\/CSE-EUC-DCABES.2016.229","DOI":"10.1109\/CSE-EUC-DCABES.2016.229"},{"key":"9656_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-021-06359-y","author":"A Barredo Arrieta","year":"2022","unstructured":"Barredo Arrieta A, Gil-Lopez S, La\u00f1a I, Bilbao N, Del Ser J (2022) On the post-hoc explainability of deep echo state networks for time series forecasting, image and video classification. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-021-06359-y","journal-title":"Neural Comput Appl"},{"key":"9656_CR33","doi-asserted-by":"publisher","unstructured":"Tong Z, Tanaka G (2018) Reservoir computing with untrained convolutional neural networks for image recognition. In: 2018 24th international conference on pattern recognition (ICPR), pp 1289\u20131294. https:\/\/doi.org\/10.1109\/ICPR.2018.8545471","DOI":"10.1109\/ICPR.2018.8545471"},{"key":"9656_CR34","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-020-05286-8","author":"R Hu","year":"2021","unstructured":"Hu R, Tang Z, Song X, Luo J, Wu E, Chang S (2021) Ensemble echo network with deep architecture for time-series modeling. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-020-05286-8","journal-title":"Neural Comput Appl"},{"key":"9656_CR35","doi-asserted-by":"publisher","unstructured":"Gardner SD, Haider MR, Moradi L, Vantsevich V (2021) A modified echo state network for time independent image classification. Institute of Electrical and Electronics Engineers Inc. https:\/\/doi.org\/10.1109\/MWSCAS47672.2021.9531776","DOI":"10.1109\/MWSCAS47672.2021.9531776"},{"key":"9656_CR36","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-021-05984-x","author":"S Dettori","year":"2022","unstructured":"Dettori S, Matino I, Colla V, Speets R (2022) A deep learning-based approach for forecasting off-gas production and consumption in the blast furnace. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-021-05984-x","journal-title":"Neural Comput Appl"},{"key":"9656_CR37","unstructured":"An S, Lee M, Park S, Yang H, So J (2020) An ensemble of simple convolutional neural network models for MNIST digit recognition"},{"key":"9656_CR38","unstructured":"Byerly A, Kalganova T, Dear I (2020) A branching and merging convolutional network with homogeneous filter capsules. arXiv:2001.09136v4"},{"key":"9656_CR39","doi-asserted-by":"publisher","unstructured":"Pishchik E (2023) Trainable activations for image classification. https:\/\/doi.org\/10.20944\/preprints202301.0463.v1","DOI":"10.20944\/preprints202301.0463.v1"},{"key":"9656_CR40","doi-asserted-by":"publisher","unstructured":"Saini D, Malik R (2021) Image data augmentation techniques for deep learning-a mirror review. In: 2021 9th International conference on reliability, infocom technologies and optimization (trends and future directions) (ICRITO), pp 1\u20135. https:\/\/doi.org\/10.1109\/ICRITO51393.2021.9596262","DOI":"10.1109\/ICRITO51393.2021.9596262"},{"issue":"5\/6","key":"9656_CR41","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1108\/IJWIS-04-2023-0062","volume":"19","author":"X Cao","year":"2023","unstructured":"Cao X, Guo Y, Yang W, Luo X, Xie S (2023) Intrinsic feature extraction for unsupervised domain adaptation. Inte J Web Inf Syst 19(5\/6):173\u2013189. https:\/\/doi.org\/10.1108\/IJWIS-04-2023-0062","journal-title":"Inte J Web Inf Syst"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09656-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-09656-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09656-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T14:07:30Z","timestamp":1720447650000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-09656-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,18]]},"references-count":41,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["9656"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-09656-4","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,18]]},"assertion":[{"value":"1 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 March 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}