{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T16:32:57Z","timestamp":1781886777834,"version":"3.54.5"},"reference-count":40,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2022,9,30]],"date-time":"2022-09-30T00:00:00Z","timestamp":1664496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,9,30]],"date-time":"2022-09-30T00:00:00Z","timestamp":1664496000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"crossref","award":["FA8655-20-1-7037"],"award-info":[{"award-number":["FA8655-20-1-7037"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and energy efficient implementations in neuromorphic hardware. However, the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network architecture and provide an in-depth study on the possible implementations of spiking residual connections. This study shows how, depending on the use case, the optimal residual connection implementation may vary. Additionally, we empirically compare different techniques in image classification datasets taken from the best performing networks. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.14%) and CIFAR-100 (74.65%) datasets and matches the state of the art in DVS-CIFAR10 (72.98%), with less parameters than the previous state of the art and without the need for ANN\u2013SNN conversion. Code available at: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/VicenteAlex\/Spiking_ResNet\" xlink:type=\"simple\">https:\/\/github.com\/VicenteAlex\/Spiking_ResNet<\/jats:ext-link>.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac8bef","type":"journal-article","created":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T22:17:00Z","timestamp":1661293020000},"page":"044001","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Keys to accurate feature extraction using residual spiking neural networks"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2370-6562","authenticated-orcid":true,"given":"Alex","family":"Vicente-Sola","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8963-5050","authenticated-orcid":true,"given":"Davide L","family":"Manna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5905-6816","authenticated-orcid":false,"given":"Paul","family":"Kirkland","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7256-0897","authenticated-orcid":false,"given":"Gaetano","family":"Di Caterina","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2431-2749","authenticated-orcid":false,"given":"Trevor","family":"Bihl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2022,9,30]]},"reference":[{"key":"nceac8befbib1","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/mm.2018.112130359","article-title":"Loihi: a neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"nceac8befbib2","first-page":"1","article-title":"Cognitive computing building block: a versatile and efficient digital neuron model for neurosynaptic cores","author":"Cassidy","year":"2013"},{"key":"nceac8befbib3","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1109\/jproc.2021.3067593","article-title":"Advancing neuromorphic computing with Loihi: a survey of results and outlook","volume":"109","author":"Davies","year":"2021","journal-title":"Proc. IEEE"},{"key":"nceac8befbib4","article-title":"Carbontracker: tracking and predicting the carbon footprint of training deep learning models","author":"Anthony","year":"2020"},{"key":"nceac8befbib5","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"nceac8befbib6","doi-asserted-by":"publisher","first-page":"11441","DOI":"10.1073\/pnas.1604850113","article-title":"Convolutional networks for fast, energy-efficient neuromorphic computing","volume":"113","author":"Esser","year":"2016","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"nceac8befbib7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnins.2020.00439","article-title":"Efficient processing of spatio-temporal data streams with spiking neural networks","volume":"14","author":"Kugele","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac8befbib8","article-title":"Recognizing images with at most one spike per neuron","author":"St\u00f6ckl","year":"2019"},{"key":"nceac8befbib9","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/msp.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"nceac8befbib10","first-page":"13558","article-title":"RMP-SNN: residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network","author":"Han","year":"2020"},{"key":"nceac8befbib11","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1109\/tcds.2020.2971655","article-title":"Deep spiking neural networks with binary weights for object recognition","volume":"13","author":"Wang","year":"2020","journal-title":"IEEE Trans. Cogn. Develop. Syst."},{"key":"nceac8befbib12","first-page":"388","article-title":"Deep spiking neural network: energy efficiency through time based coding","author":"Han","year":"2020"},{"key":"nceac8befbib13","first-page":"2328","article-title":"Optimal conversion of conventional artificial neural networks to spiking neural networks","author":"Deng","year":"2021"},{"key":"nceac8befbib14","article-title":"Progressive tandem learning for pattern recognition with deep spiking neural networks","author":"Wu","year":"2021"},{"key":"nceac8befbib15","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1038\/s42256-018-0015-y","article-title":"Training deep neural networks for binary communication with the whetstone method","volume":"1","author":"Severa","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"nceac8befbib16","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1038\/s42256-021-00311-4","article-title":"Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes","volume":"3","author":"St\u00f6ckl","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"nceac8befbib17","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neunet.2019.09.005","article-title":"Rethinking the performance comparison between SNNS and ANNS","volume":"121","author":"Deng","year":"2020","journal-title":"Neural Netw."},{"key":"nceac8befbib18","doi-asserted-by":"publisher","first-page":"331","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceac8befbib19","doi-asserted-by":"publisher","first-page":"773954","DOI":"10.3389\/fnins.2021.773954","article-title":"Revisiting batch normalization for training low-latency deep spiking neural networks from scratch","volume":"15","author":"Kim","year":"2021","journal-title":"Front. Neurosci."},{"key":"nceac8befbib20","article-title":"Gradient descent for spiking neural networks","volume":"vol 31","author":"Huh","year":"2018"},{"key":"nceac8befbib21","doi-asserted-by":"publisher","first-page":"1542","DOI":"10.1162\/neco_a_01080","article-title":"A learning framework for winner-take-all networks with stochastic synapses","volume":"30","author":"Mostafa","year":"2018","journal-title":"Neural Comput."},{"key":"nceac8befbib22","doi-asserted-by":"publisher","first-page":"3227","DOI":"10.1109\/tnnls.2017.2726060","article-title":"Supervised learning based on temporal coding in spiking neural networks","volume":"29","author":"Mostafa","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceac8befbib23","doi-asserted-by":"publisher","first-page":"424","DOI":"10.3389\/fnins.2020.00424","article-title":"Synaptic plasticity dynamics for deep continuous local learning (decolle)","volume":"14","author":"Kaiser","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac8befbib24","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1016\/j.conb.2017.08.020","article-title":"Learning with three factors: modulating Hebbian plasticity with errors","volume":"46","author":"Ku\u015bmierz","year":"2017","journal-title":"Curr. Opin. Neurobiol."},{"key":"nceac8befbib25","doi-asserted-by":"publisher","first-page":"119","DOI":"10.3389\/fnins.2020.00119","article-title":"Enabling spike-based backpropagation for training deep neural network architectures","volume":"14","author":"Lee","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac8befbib26","article-title":"Slayer: spike layer error reassignment in time","volume":"vol 31","author":"Shrestha","year":"2018"},{"key":"nceac8befbib27","first-page":"11062","article-title":"Going deeper with directly-trained larger spiking neural networks","volume":"vol 35","author":"Zheng","year":"2021"},{"key":"nceac8befbib28","first-page":"2661","article-title":"Incorporating learnable membrane time constant to enhance learning of spiking neural networks","author":"Fang","year":"2021"},{"key":"nceac8befbib29","article-title":"Deep residual learning in spiking neural networks","volume":"vol 34","author":"Fang","year":"2021"},{"key":"nceac8befbib30","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2015"},{"key":"nceac8befbib31","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"nceac8befbib32","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1016\/s0361-9230(99)00161-6","article-title":"Lapicque\u2019s introduction of the integrate-and-fire model neuron (1907)","volume":"50","author":"Abbott","year":"1999","journal-title":"Brain Res. Bull."},{"key":"nceac8befbib33","first-page":"254","article-title":"Efficient neuromorphic signal processing with Loihi 2","author":"Orchard","year":"2021"},{"key":"nceac8befbib34","article-title":"Recurrent batch normalization","author":"Cooijmans","year":"2016"},{"key":"nceac8befbib35","first-page":"8026","article-title":"Pytorch: an imperative style, high-performance deep learning library","volume":"vol 32","author":"Adam","year":"2019"},{"key":"nceac8befbib36","article-title":"SpikingJelly","author":"Fang","year":"2020"},{"key":"nceac8befbib37","first-page":"1437","article-title":"Bohb: robust and efficient hyperparameter optimization at scale","author":"Falkner","year":"2018"},{"key":"nceac8befbib38","doi-asserted-by":"publisher","first-page":"309","DOI":"10.3389\/fnins.2017.00309","article-title":"CIFAR10-DVS: an event-stream dataset for object classification","volume":"11","author":"Li","year":"2017","journal-title":"Front. Neurosci."},{"key":"nceac8befbib39","doi-asserted-by":"publisher","first-page":"95","DOI":"10.3389\/fnins.2019.00095","article-title":"Going deeper in spiking neural networks: VGG and residual architectures","volume":"13","author":"Sengupta","year":"2019","journal-title":"Front. Neurosci."},{"key":"nceac8befbib40","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.20712535.v2","article-title":"Pretrained models - Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks","author":"Vicente-Sola","year":"2022"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,5]],"date-time":"2022-10-05T08:24:13Z","timestamp":1664958253000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8bef"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,30]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,9,30]]},"published-print":{"date-parts":[[2022,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac8bef","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,30]]},"assertion":[{"value":"Keys to accurate feature extraction using residual spiking neural networks","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-06-22","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-08-23","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-09-30","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}