{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T06:02:58Z","timestamp":1772517778197,"version":"3.50.1"},"reference-count":42,"publisher":"IOP Publishing","issue":"1","license":[{"start":{"date-parts":[[2021,8,25]],"date-time":"2021-08-25T00:00:00Z","timestamp":1629849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,8,25]],"date-time":"2021-08-25T00:00:00Z","timestamp":1629849600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"National Science Foundation","award":["CCF-1900675"],"award-info":[{"award-number":["CCF-1900675"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2021,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Reservoir computing (RC) offers efficient temporal data processing with a low training cost by separating recurrent neural networks into a fixed network with recurrent connections and a trainable linear network. The quality of the fixed network, called reservoir, is the most important factor that determines the performance of the RC system. In this paper, we investigate the influence of the hierarchical reservoir structure on the properties of the reservoir and the performance of the RC system. Analogous to deep neural networks, stacking sub-reservoirs in series is an efficient way to enhance the nonlinearity of data transformation to high-dimensional space and expand the diversity of temporal information captured by the reservoir. These deep reservoir systems offer better performance when compared to simply increasing the size of the reservoir or the number of sub-reservoirs. Low frequency components are mainly captured by the sub-reservoirs in later stage of the deep reservoir structure, similar to observations that more abstract information can be extracted by layers in the late stage of deep neural networks. When the total size of the reservoir is fixed, tradeoff between the number of sub-reservoirs and the size of each sub-reservoir needs to be carefully considered, due to the degraded ability of individual sub-reservoirs at small sizes. Improved performance of the deep reservoir structure alleviates the difficulty of implementing the RC system on hardware systems.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac1b75","type":"journal-article","created":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T22:35:20Z","timestamp":1628289320000},"page":"014006","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":41,"title":["Hierarchical architectures in reservoir computing systems"],"prefix":"10.1088","volume":"1","author":[{"given":"John","family":"Moon","sequence":"first","affiliation":[]},{"given":"Yuting","family":"Wu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4731-1976","authenticated-orcid":false,"given":"Wei D","family":"Lu","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2021,8,25]]},"reference":[{"key":"nceac1b75bib1","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"nceac1b75bib2","first-page":"p 1097","volume":"vol 25","author":"Krizhevsky","year":"2012"},{"key":"nceac1b75bib3","author":"He","year":"2016"},{"key":"nceac1b75bib4","author":"Tan","year":"2019"},{"key":"nceac1b75bib5","author":"Mandic","year":"2001"},{"key":"nceac1b75bib6","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"nceac1b75bib7","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1126\/science.1091277","volume":"304","author":"Jaeger","year":"2004","journal-title":"Science"},{"key":"nceac1b75bib8","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1109\/72.279181","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"nceac1b75bib9","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.cosrev.2009.03.005","volume":"3","author":"Luko\u0161evi\u010dius","year":"2009","journal-title":"Comput. Sci. Rev."},{"key":"nceac1b75bib10","author":"Verstraeten","year":"2006"},{"key":"nceac1b75bib11","doi-asserted-by":"publisher","DOI":"10.1103\/physrevlett.120.024102","volume":"120","author":"Pathak","year":"2018","journal-title":"Phys. Rev. Lett."},{"key":"nceac1b75bib12","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1038\/s41928-019-0313-3","volume":"2","author":"Moon","year":"2019","journal-title":"Nat. Electron."},{"key":"nceac1b75bib13","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.neunet.2019.03.005","volume":"115","author":"Tanaka","year":"2019","journal-title":"Neural Netw."},{"key":"nceac1b75bib14","author":"Luko\u0161evi\u010dius","year":"2012"},{"key":"nceac1b75bib15","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.neunet.2007.04.014","volume":"20","author":"Xue","year":"2007","journal-title":"Neural Netw."},{"key":"nceac1b75bib16","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1109\/TNN.2010.2089641","volume":"22","author":"Rodan","year":"2010","journal-title":"IEEE Trans. Neural Netw."},{"key":"nceac1b75bib17","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1038\/ncomms1476","volume":"2","author":"Appeltant","year":"2011","journal-title":"Nat. Commun."},{"key":"nceac1b75bib18","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1016\/j.neunet.2011.02.002","volume":"24","author":"Gallicchio","year":"2011","journal-title":"Neural Netw."},{"key":"nceac1b75bib19","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1016\/j.neunet.2012.11.011","volume":"38","author":"Butcher","year":"2013","journal-title":"Neural Netw."},{"key":"nceac1b75bib20","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1101\/855999","volume":"47","author":"Malik","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"nceac1b75bib21","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.neucom.2016.12.089","volume":"268","author":"Gallicchio","year":"2017","journal-title":"Neurocomputing"},{"key":"nceac1b75bib22","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.neunet.2018.08.002","volume":"108","author":"Gallicchio","year":"2018","journal-title":"Neural Netw."},{"key":"nceac1b75bib23","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1016\/j.ins.2019.09.049","volume":"511","author":"Ma","year":"2020","journal-title":"Inf. Sci."},{"key":"nceac1b75bib24","author":"Mitchell","year":"1998"},{"key":"nceac1b75bib25","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1109\/72.846741","volume":"11","author":"Atiya","year":"2000","journal-title":"IEEE Trans. Neural Netw."},{"key":"nceac1b75bib26","author":"Weigend","year":"2018"},{"key":"nceac1b75bib27","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1126\/science.267326","volume":"197","author":"Mackey","year":"1977","journal-title":"Science"},{"key":"nceac1b75bib28","author":"Gallicchio","year":"2017"},{"key":"nceac1b75bib29","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.neucom.2016.01.088","volume":"192","author":"Otte","year":"2016","journal-title":"Neurocomputing"},{"key":"nceac1b75bib30","doi-asserted-by":"publisher","first-page":"601","DOI":"10.14311\/NNW.2014.24.034","volume":"24","author":"Lazz\u00fas","year":"2014","journal-title":"Neural Netw. World"},{"key":"nceac1b75bib31","author":"Gers","year":"2002"},{"key":"nceac1b75bib32","volume":"vol 5","author":"Jaeger","year":"2001"},{"key":"nceac1b75bib33","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.neunet.2016.07.012","volume":"83","author":"Farka\u0161","year":"2016","journal-title":"Neural Netw."},{"key":"nceac1b75bib34","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1038\/srep00514","volume":"2","author":"Dambre","year":"2012","journal-title":"Sci. Rep."},{"key":"nceac1b75bib35","doi-asserted-by":"publisher","first-page":"10199","DOI":"10.1038\/s41598-017-10257-6","volume":"7","author":"Inubushi","year":"2017","journal-title":"Sci. Rep."},{"key":"nceac1b75bib36","doi-asserted-by":"publisher","first-page":"1900084","DOI":"10.1002\/aisy.201900084","article-title":"Reservoir computing using diffusive memristors","volume":"1","author":"Midya","year":"2019","journal-title":"Adv. Intell. Syst."},{"key":"nceac1b75bib37","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.neunet.2019.01.002","article-title":"A small-world topology enhances the echo state property and signal propagation in reservoir computing","volume":"112","author":"Kawai","year":"2019","journal-title":"Neural Netw."},{"key":"nceac1b75bib38","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1162\/netn_a_00082","article-title":"Optimal modularity and memory capacity of neural reservoirs","volume":"3","author":"Rodriguez","year":"2019","journal-title":"Netw. Neurosci."},{"key":"nceac1b75bib39","first-page":"62","article-title":"Reservoir topology in deep echo state networks","author":"Gallicchio","year":"2019"},{"key":"nceac1b75bib40","author":"Jaeger","year":"2007"},{"key":"nceac1b75bib41","first-page":"19.01.20 - 21.01.20,","article-title":"Short-term load forecasting with missing data using dilated recurrent attention networks","author":"Choi","year":"2020","journal-title":"UiT The Arctic University of Norway, Troms\u00f8"},{"key":"nceac1b75bib42","article-title":"Dynamical anatomy of NARMA10 benchmark task","author":"Kubota","year":"2019"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,13]],"date-time":"2021-12-13T15:27:58Z","timestamp":1639409278000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac1b75"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,25]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,8,25]]},"published-print":{"date-parts":[[2021,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac1b75","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,25]]},"assertion":[{"value":"Hierarchical architectures in reservoir computing systems","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2021-02-15","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-08-03","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-08-25","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}