{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T07:44:24Z","timestamp":1769154264757,"version":"3.49.0"},"reference-count":32,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T00:00:00Z","timestamp":1722211200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T00:00:00Z","timestamp":1722211200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>A primary objective of Spiking Neural Networks is a very energy-efficient computation. To achieve this target, a small spike rate is of course very beneficial given the event-driven nature of such a computation. A network that processes information encoded in spike timing can, by its nature, have such a sparse event rate, but, as the network becomes deeper and larger, the spike rate tends to increase without any improvements in the final accuracy. If, on the other hand, a penalty on the excess of spikes is used during the training, the network may shift to a configuration where many neurons are silent, thus affecting the effectiveness of the training itself. In this paper, we present a learning strategy to keep the final spike rate under control by changing the loss function to penalize the spikes generated by neurons after the first ones. Moreover, we also propose a 2-phase training strategy to avoid silent neurons during the training, intended for benchmarks where such an issue can cause the switch off of the network.<\/jats:p>","DOI":"10.1088\/2634-4386\/ad64fd","type":"journal-article","created":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T22:53:08Z","timestamp":1721343188000},"page":"034004","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Reducing the spike rate of deep spiking neural networks based on time-encoding"],"prefix":"10.1088","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4246-946X","authenticated-orcid":false,"given":"Riccardo","family":"Fontanini","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2860-3714","authenticated-orcid":true,"given":"Alessandro","family":"Pilotto","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3468-5197","authenticated-orcid":false,"given":"David","family":"Esseni","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7876-3612","authenticated-orcid":true,"given":"Mirko","family":"Loghi","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2024,7,29]]},"reference":[{"key":"ncead64fdbib1","first-page":"pp 3104","article-title":"Sequence to sequence learning with neural networks","author":"Sutskever","year":"2014"},{"key":"ncead64fdbib2","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ncead64fdbib3","article-title":"Imagenet classification with deep convolutional neural networks","volume":"vol 25","author":"Krizhevsky","year":"2012"},{"key":"ncead64fdbib4","doi-asserted-by":"crossref","DOI":"10.3390\/robotics10010022","article-title":"Deep reinforcement learning for the control of robotic manipulation: a focussed mini-review","author":"Liu","year":"2021"},{"key":"ncead64fdbib5","author":"Goodfellow","year":"2016"},{"key":"ncead64fdbib6","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1016\/S0893-6080(97)00011-7","article-title":"Networks of spiking neurons: the third generation of neural network models","volume":"10","author":"Maass","year":"1997","journal-title":"Neural Netw."},{"key":"ncead64fdbib7","doi-asserted-by":"publisher","first-page":"1629","DOI":"10.1109\/5.58356","article-title":"Neuromorphic electronic systems","volume":"78","author":"Mead","year":"1990"},{"key":"ncead64fdbib8","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.611300","article-title":"Adaptive extreme edge computing for wearable devices","volume":"15","author":"Covi","year":"2021","journal-title":"Front. Neurosci."},{"key":"ncead64fdbib9","doi-asserted-by":"publisher","first-page":"98","DOI":"10.5213\/inj.2013.17.3.98","article-title":"Development of implantable medical devices: from an engineering perspective","volume":"17","author":"Joung","year":"2013","journal-title":"Int. Neurourol. J."},{"key":"ncead64fdbib10","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Front. Neurosci."},{"key":"ncead64fdbib11","first-page":"pp 565","article-title":"Skipper: enabling efficient snn training through activation-checkpointing and time-skipping","author":"Singh","year":"2022"},{"key":"ncead64fdbib12","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2023.1047008","article-title":"Efficient training of spiking neural networks with temporally-truncated local backpropagation through time","volume":"17","author":"Guo","year":"2023","journal-title":"Front. Neurosci."},{"key":"ncead64fdbib13","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2017.00682","article-title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","volume":"11","author":"Rueckauer","year":"2017","journal-title":"Front. Neurosci."},{"key":"ncead64fdbib14","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/MSP.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"ncead64fdbib15","first-page":"419","article-title":"Spikeprop: backpropagation for networks of spiking neurons","author":"Boht\u00e9","year":"2000","journal-title":"ESANN"},{"key":"ncead64fdbib16","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1016\/j.ipl.2005.05.023","article-title":"A gradient descent rule for spiking neurons emitting multiple spikes","volume":"95","author":"Booij","year":"2005","journal-title":"Inf. Process. Lett."},{"key":"ncead64fdbib17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/neco.1996.8.1.1","article-title":"Lower bounds for the computational power of networks of spiking neurons","volume":"8","author":"Maass","year":"1996","journal-title":"Neural Comput."},{"key":"ncead64fdbib18","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1113\/jphysiol.1952.sp004764","article-title":"A quantitative description of membrane current and its application to conduction and excitation in nerve","volume":"117","author":"Hodgkin","year":"1952","journal-title":"J. Physiol."},{"key":"ncead64fdbib19","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-91786-z","article-title":"Event-based backpropagation can compute exact gradients for spiking neural networks","volume":"11","author":"Wunderlich","year":"2021","journal-title":"Sci. Rep."},{"key":"ncead64fdbib20","doi-asserted-by":"crossref","DOI":"10.1145\/3546790.3546798","article-title":"Reducing the spike rate in deep spiking neural networks","author":"Fontanini","year":"2022"},{"key":"ncead64fdbib21","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.neunet.2013.02.003","article-title":"A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks","volume":"43","author":"Xu","year":"2013","journal-title":"Neural Netw."},{"key":"ncead64fdbib22","first-page":"95","article-title":"Learning methods of recurrent spiking neural networks","volume":"13","author":"Selvaratnam","year":"2000","journal-title":"Trans. Inst. Syst. Control Inf. Eng."},{"key":"ncead64fdbib23","first-page":"pp 1","article-title":"Learning methods of recurrent spiking neural networks based on adjoint equations approach","author":"Kuroe","year":"2010"},{"key":"ncead64fdbib24","article-title":"Gradient descent for spiking neural networks","volume":"vol 31","author":"Huh","year":"2018"},{"key":"ncead64fdbib25","author":"Gerstner","year":"2002"},{"key":"ncead64fdbib26","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2011.00073","article-title":"Neuromorphic silicon neuron circuits","volume":"5","author":"Indiveri","year":"2011","journal-title":"Front. Neurosci."},{"key":"ncead64fdbib27","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ncead64fdbib28","first-page":"pp 8024","article-title":"Pytorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019"},{"key":"ncead64fdbib29","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2015"},{"key":"ncead64fdbib30","first-page":"pp 157","article-title":"Improving generalization by data categorization","author":"Li","year":"2005"},{"key":"ncead64fdbib31","article-title":"The yin-yang dataset","author":"Kriener","year":"2021"},{"key":"ncead64fdbib32","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T12:27:17Z","timestamp":1722256037000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad64fd"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,29]]},"references-count":32,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,7,29]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ad64fd","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,29]]},"assertion":[{"value":"Reducing the spike rate of deep spiking neural networks based on time-encoding","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-04-10","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-07-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-07-29","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}