{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T14:15:52Z","timestamp":1785420952638,"version":"3.56.0"},"reference-count":55,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T00:00:00Z","timestamp":1686182400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>A variety of advanced machine learning and deep learning algorithms achieve state-of-the-art performance on various temporal processing tasks. However, these methods are heavily energy inefficient\u2014they run mainly on the power hungry CPUs and GPUs. Computing with Spiking Networks, on the other hand, has shown to be energy efficient on specialized neuromorphic hardware, e.g., Loihi, TrueNorth, SpiNNaker, etc. In this work, we present two architectures of spiking models, inspired from the theory of Reservoir Computing and Legendre Memory Units, for the Time Series Classification (TSC) task. Our first spiking architecture is closer to the general Reservoir Computing architecture and we successfully deploy it on Loihi; the second spiking architecture differs from the first by the inclusion of non-linearity in the readout layer. Our second model (trained with Surrogate Gradient Descent method) shows that non-linear decoding of the linearly extracted temporal features through spiking neurons not only achieves promising results, but also offers low computation-overhead by significantly reducing the number of neurons compared to the popular LSM based models\u2014more than 40x reduction with respect to the recent spiking model we compare with. We experiment on five TSC datasets and achieve new SoTA spiking results (\u2014as much as 28.607% accuracy improvement on one of the datasets), thereby showing the potential of our models to address the TSC tasks in a green energy-efficient manner. In addition, we also do energy profiling and comparison on Loihi and CPU to support our claims.<\/jats:p>","DOI":"10.3389\/fncom.2023.1148284","type":"journal-article","created":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T05:24:45Z","timestamp":1686201885000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Reservoir based spiking models for univariate Time Series Classification"],"prefix":"10.3389","volume":"17","author":[{"given":"Ramashish","family":"Gaurav","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Terrence C.","family":"Stewart","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Yi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,8]]},"reference":[{"key":"B1","first-page":"125","article-title":"\u201cReservoir computing using a delayed feedback system: towards photonic implementations,\u201d","volume-title":"16th Annual Symposium of the IEEE Photonics Benelux Chapter","author":"Appeltant","year":"2011"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3264659","article-title":"DFR: an energy-efficient analog delay feedback reservoir computing system for brain-inspired computing","volume":"14","author":"Bai","year":"2018","journal-title":"ACM J. Emerg. Technol. Comput. Syst."},{"key":"B3","doi-asserted-by":"publisher","first-page":"48","DOI":"10.3389\/fninf.2013.00048","article-title":"Nengo: a Python tool for building large-scale functional brain models","volume":"7","author":"Bekolay","year":"2014","journal-title":"Front. Neuroinform."},{"key":"B4","doi-asserted-by":"publisher","first-page":"2169","DOI":"10.1109\/TNNLS.2020.3001377","article-title":"Reservoir computing approaches for representation and classification of multivariate time series","volume":"32","author":"Bianchi","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B5","doi-asserted-by":"crossref","DOI":"10.3233\/SHTI220388","article-title":"\u201cECG classification using combination of linear and non-linear features with neural network,\u201d","author":"Biloborodova","year":"2022","journal-title":"Challenges of Trustable AI and Added-Value on Health"},{"key":"B6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3320288.3320304","article-title":"\u201cBenchmarking keyword spotting efficiency on neuromorphic hardware,\u201d","volume-title":"Proceedings of the 7th Annual Neuro-Inspired Computational Elements Workshop","author":"Blouw","year":"2019"},{"key":"B7","article-title":"Hardware aware training for efficient keyword spotting on general purpose and specialized hardware","author":"Blouw","year":"2020","journal-title":"arXiv preprint arXiv:2009.04465"},{"key":"B8","author":"Brogan","year":"1991","journal-title":"Modern Control Theory"},{"key":"B9","article-title":"\u201cOptimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks,\u201d","volume-title":"International Conference on Learning Representations","author":"Bu","year":"2021"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00422-006-0068-6","article-title":"A review of the integrate-and-fire neuron model: I. homogeneous synaptic input","volume":"95","author":"Burkitt","year":"2006","journal-title":"Biol. Cybernet."},{"key":"B11","article-title":"Language modeling using LMUS: 10x better data efficiency or improved scaling compared to transformers","author":"Chilkuri","year":"2021","journal-title":"arXiv preprint arXiv:2110.02402"},{"key":"B12","first-page":"1898","article-title":"\u201cParallelizing legendre memory unit training,\u201d","volume-title":"International Conference on Machine Learning","author":"Chilkuri","year":"2021"},{"key":"B13","doi-asserted-by":"crossref","first-page":"718","DOI":"10.23919\/DATE54114.2022.9774704","article-title":"\u201cCan deep neural networks be converted to ultra low-latency spiking neural networks?\u201d","volume-title":"2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)","author":"Datta","year":"2022"},{"key":"B14","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MM.2018.112130359","article-title":"Loihi: a neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"B15","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1109\/JPROC.2021.3067593","article-title":"Advancing neuromorphic computing with loihi: a survey of results and outlook","volume":"109","author":"Davies","year":"2021","journal-title":"Proc. IEEE"},{"key":"B16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN55064.2022.9892728","article-title":"\u201cEfficient time series classification using spiking reservoir,\u201d","volume-title":"2022 International Joint Conference on Neural Networks (IJCNN)","author":"Dey","year":"2022"},{"key":"B17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN.2018.8489381","article-title":"\u201cDeep spiking neural network model for time-variant signals classification: a real-time speech recognition approach,\u201d","volume-title":"2018 International Joint Conference on Neural Networks (IJCNN)","author":"Dominguez-Morales","year":"2018"},{"key":"B18","author":"Eliasmith","year":"2003","journal-title":"Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems"},{"key":"B19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN48605.2020.9206751","article-title":"\u201cMultivariate time series classification using spiking neural networks,\u201d","volume-title":"2020 International Joint Conference on Neural Networks (IJCNN)","author":"Fang","year":"2020"},{"key":"B20","first-page":"526","article-title":"\u201cSpiking reservoir computing for temporal edge intelligence on loihi,\u201d","volume-title":"2022 IEEE\/ACM 7th Symposium on Edge Computing (SEC)","author":"Gaurav","year":""},{"key":"B21","first-page":"1","article-title":"\u201cSpiking approximations of the maxpooling operation in deep SNNs,\u201d","author":"Gaurav","year":"","journal-title":"2022 International Joint Conference on Neural Networks (Padua: IJCNN)"},{"key":"B22","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1007\/s10489-019-01552-y","article-title":"CLR-based deep convolutional spiking neural network with validation based stopping for time series classification","volume":"50","author":"Gautam","year":"2020","journal-title":"Appl. Intell."},{"key":"B23","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"B24","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"B25","year":"2021","journal-title":"Taking Neuromorphic Computing with Loihi 2 to the Next Level. Technology Brief"},{"key":"B26","volume-title":"The \u201cEcho State\u201d Approach to Analysing and Training Recurrent Neural Networks-With an Erratum Note","author":"Jaeger","year":"2001"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1662","DOI":"10.1109\/ACCESS.2017.2779939","article-title":"Lstm fully convolutional networks for time series classification","volume":"6","author":"Karim","year":"2017","journal-title":"IEEE Access"},{"key":"B28","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"B29","doi-asserted-by":"publisher","first-page":"508","DOI":"10.3389\/fnins.2016.00508","article-title":"Training deep spiking neural networks using backpropagation","volume":"10","author":"Lee","year":"2016","journal-title":"Front. Neurosci."},{"key":"B30","first-page":"6316","article-title":"\u201cA free lunch from ANN: towards efficient, accurate spiking neural networks calibration,\u201d","volume-title":"International Conference on Machine Learning","author":"Li","year":"2021"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1145\/3182382","article-title":"Time series classification with hive-cote: the hierarchical vote collective of transformation-based ensembles","author":"Lines","year":"2018","journal-title":"ACM Trans. Knowledge Discov. Data"},{"key":"B32","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.cosrev.2009.03.005","article-title":"Reservoir computing approaches to recurrent neural network training","volume":"3","author":"Luko\u0161evi\u010dius","year":"2009","journal-title":"Comput. Sci. Rev."},{"key":"B33","doi-asserted-by":"publisher","first-page":"2531","DOI":"10.1162\/089976602760407955","article-title":"Real-time computing without stable states: a new framework for neural computation based on perturbations","volume":"14","author":"Maass","year":"2002","journal-title":"Neural Comput."},{"key":"B34","first-page":"91","article-title":"\u201cRobust anomaly detection in time series through variational autoencoders and a local similarity score,\u201d","volume-title":"International Conference on Bio-inspired Systems and Signal Processing 2021","author":"Matias","year":"2021"},{"key":"B35","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/MSP.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"B36","doi-asserted-by":"crossref","DOI":"10.1109\/MWSCAS48704.2020.9184677","article-title":"\u201cEnergy efficient and adaptive analog ic design for delay-based reservoir computing,\u201d","volume-title":"2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)","author":"Nowshin","year":"2020"},{"key":"B37","doi-asserted-by":"publisher","first-page":"123","DOI":"10.3390\/s22010123","article-title":"Attention autoencoder for generative latent representational learning in anomaly detection","volume":"22","author":"Oluwasanmi","year":"2021","journal-title":"Sensors"},{"key":"B38","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1109\/SiPS52927.2021.00053","article-title":"\u201cEfficient neuromorphic signal processing with loihi 2,\u201d","volume-title":"2021 IEEE Workshop on Signal Processing Systems (SiPS)","author":"Orchard","year":"2021"},{"key":"B39","doi-asserted-by":"publisher","first-page":"8160","DOI":"10.1007\/s10489-022-03859-9","article-title":"Fractional-order multiscale attention feature pyramid network for time series classification","volume":"53","author":"Pan","year":"2022","journal-title":"Appl. Intell."},{"key":"B40","article-title":"\u201cPyTorch: an imperative style, high-performance deep learning library,\u201d","volume-title":"Advances in Neural Information Processing Systems 32","author":"Paszke","year":"2019"},{"key":"B41","article-title":"A spiking neural network for image segmentation","author":"Patel","year":"2021","journal-title":"arXiv preprint arXiv:2106.08921"},{"key":"B42","doi-asserted-by":"publisher","first-page":"819063","DOI":"10.3389\/fnins.2022.819063","article-title":"Liquid state machine on spinnaker for spatio-temporal classification tasks","volume":"16","author":"Pati\u00f1o-Saucedo","year":"2022","journal-title":"Front. Neurosci."},{"key":"B43","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1504\/IJDMB.2019.101395","article-title":"Unsupervised representation learning and anomaly detection in ECG sequences","volume":"22","author":"Pereira","year":"2019","journal-title":"Int. J Data Mining Bioinform."},{"key":"B44","doi-asserted-by":"publisher","first-page":"774","DOI":"10.3389\/fnins.2018.00774","article-title":"Deep learning with spiking neurons: opportunities and challenges","volume":"12","author":"Pfeiffer","year":"2018","journal-title":"Front. Neurosci."},{"key":"B45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ISCAS.2018.8351295","article-title":"\u201cConversion of analog to spiking neural networks using sparse temporal coding,\u201d","volume-title":"2018 IEEE International Symposium on Circuits and Systems (ISCAS)","author":"Rueckauer","year":"2018"},{"key":"B46","author":"Shenoy Renjal","year":"2019","journal-title":"Liquid state machine model with homeostasis and supervised stdp on neuromorphic loihi processor"},{"key":"B47","article-title":"A technical overview of the neural engineering framework","author":"Stewart","year":"2012","journal-title":"Univ. Waterloo"},{"key":"B48","article-title":"\u201cLegendre memory units: continuous-time representation in recurrent neural networks,\u201d","author":"Voelker","year":"2019","journal-title":"Advances in Neural Information Processing Systems 32"},{"key":"B49","volume-title":"Dynamical systems in spiking neuromorphic hardware","author":"Voelker","year":"2019"},{"key":"B50","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1162\/neco_a_01046","article-title":"Improving spiking dynamical networks: accurate delays, higher-order synapses, and time cells","volume":"30","author":"Voelker","year":"2018","journal-title":"Neural Comput."},{"key":"B51","doi-asserted-by":"publisher","first-page":"331","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Front. Neurosci."},{"key":"B52","first-page":"1311","article-title":"\u201cDirect training for spiking neural networks: Faster, larger, better,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Wu","year":"2019"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1088\/2634-4386\/abf150","article-title":"Comparing loihi with a spinnaker 2 prototype on low-latency keyword spotting and adaptive robotic control","author":"Yan","year":"2021","journal-title":"Neuromorph. Comput. Eng"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1514","DOI":"10.1162\/neco_a_01086","article-title":"Superspike: supervised learning in multilayer spiking neural networks","volume":"30","author":"Zenke","year":"2018","journal-title":"Neural Comput."},{"key":"B55","first-page":"11062","article-title":"\u201cGoing deeper with directly-trained larger spiking neural networks,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Zheng","year":"2021"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2023.1148284\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T05:25:14Z","timestamp":1686201914000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2023.1148284\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,8]]},"references-count":55,"alternative-id":["10.3389\/fncom.2023.1148284"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2023.1148284","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,8]]},"article-number":"1148284"}}