{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T16:02:28Z","timestamp":1781193748140,"version":"3.54.1"},"reference-count":52,"publisher":"MIT Press","issue":"10","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,9,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Spiking neural networks (SNNs) have emerged as a promising alternative to traditional deep neural networks for low-power computing. However, the effectiveness of SNNs is not solely determined by their performance but also by their energy consumption, prediction speed, and robustness to noise. The recent method Fast &amp; Deep, along with others, achieves fast and energy-efficient computation by constraining neurons to fire at most once. Known as time-to-first-spike (TTFS), this constraint, however, restricts the capabilities of SNNs in many aspects. In this work, we explore the relationships of performance, energy consumption, speed, and stability when using this constraint. More precisely, we highlight the existence of trade-offs where performance and robustness are gained at the cost of sparsity and prediction latency. To improve these trade-offs, we propose a relaxed version of Fast &amp; Deep that allows for multiple spikes per neuron. Our experiments show that relaxing the spike constraint provides higher performance while also benefiting from faster convergence, similar sparsity, comparable prediction latency, and better robustness to noise compared to TTFS SNNs. By highlighting the limitations of TTFS and demonstrating the advantages of unconstrained SNNs, we provide valuable insight for the development of effective learning strategies for neuromorphic computing.<\/jats:p>","DOI":"10.1162\/neco_a_01609","type":"journal-article","created":{"date-parts":[[2023,7,31]],"date-time":"2023-07-31T18:06:30Z","timestamp":1690826790000},"page":"1627-1656","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":3,"title":["Exploring Trade-Offs in Spiking Neural Networks"],"prefix":"10.1162","volume":"35","author":[{"given":"Florian","family":"Bacho","sequence":"first","affiliation":[{"name":"CEMS, School of Computing, University of Kent, Canterbury CT2 7NF, U.K. fb320@kent.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dominique","family":"Chu","sequence":"additional","affiliation":[{"name":"CEMS, School of Computing, University of Kent, Canterbury CT2 7NF, U.K. d.f.chu@kent.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,9,8]]},"reference":[{"issue":"10","key":"2023091116080363100_bib1","doi-asserted-by":"publisher","first-page":"1537","DOI":"10.1109\/TCAD.2015.2474396","article-title":"TrueNorth: Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip","volume":"34","author":"Akopyan","year":"2015","journal-title":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems"},{"key":"2023091116080363100_bib2","doi-asserted-by":"crossref","DOI":"10.1145\/3320288.3320304","article-title":"Benchmarking keyword spotting efficiency on neuromorphic hardware","volume-title":"Proceedings of the 7th Annual Neuro-Inspired Computational Elements Workshop","author":"Blouw","year":"2019"},{"key":"2023091116080363100_bib3","article-title":"Spikeprop: Backpropagation for networks of spiking neurons","author":"Boht\u00e9","year":"2000","journal-title":"Proceedings of ESANN"},{"key":"2023091116080363100_bib4","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Advances in neural information processing systems","author":"Brown","year":"2020"},{"key":"2023091116080363100_bib5","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.neucom.2021.10.080","article-title":"An adaptive threshold mechanism for accurate and efficient deep spiking convolutional neural networks","volume":"469","author":"Chen","year":"2022","journal-title":"Neurocomputing"},{"key":"2023091116080363100_bib6","first-page":"2921","article-title":"Emnist: Extending MNIST to handwritten letters","volume-title":"Proceedings of the 2017 International Joint Conference on Neural Networks","author":"Cohen","year":"2017"},{"key":"2023091116080363100_bib7","article-title":"Temporal coding in spiking neural networks with alpha synaptic function: Learning with backpropagation","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing.","author":"Comsa","year":"2020"},{"issue":"7","key":"2023091116080363100_bib8","doi-asserted-by":"publisher","first-page":"2744","DOI":"10.1109\/TNNLS.2020.3044364","article-title":"The Heidelberg spiking data sets for the systematic evaluation of spiking neural networks","volume":"33","author":"Cramer","year":"2022","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"issue":"1","key":"2023091116080363100_bib9","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MM.2018.112130359","article-title":"Loihi: A neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"2023091116080363100_bib10","author":"Devlin","year":"2018","journal-title":"Bert: Pre-training of deep bidirectional transformers for language understanding."},{"key":"2023091116080363100_bib11","first-page":"1","article-title":"Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing","volume-title":"Proceedings of the 2015 International Joint Conference on Neural Networks","author":"Diehl","year":"2015"},{"key":"2023091116080363100_bib12","first-page":"2661","article-title":"Incorporating learnable membrane time constant to enhance learning of spiking neural networks","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Fang","year":"2021"},{"issue":"5","key":"2023091116080363100_bib13","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/13\/5\/051001","article-title":"Large-scale neuromorphic computing systems","volume":"13","author":"Furber","year":"2016","journal-title":"Journal of Neural Engineering"},{"key":"2023091116080363100_bib14","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511815706","volume-title":"Spiking neuron models: Single neurons, populations, plasticity","author":"Gerstner","year":"2002"},{"issue":"9","key":"2023091116080363100_bib15","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1038\/s42256-021-00388-x","article-title":"Fast and energy-efficient neuromorphic deep learning with first-spike times","volume":"3","author":"G\u00f6ltz","year":"2021","journal-title":"Nature Machine Intelligence"},{"issue":"7","key":"2023091116080363100_bib16","doi-asserted-by":"crossref","DOI":"10.3390\/electronics9071059","article-title":"Towards efficient neuromorphic hardware: Unsupervised adaptive neuron pruning","volume":"9","author":"Guo","year":"2020","journal-title":"Electronics"},{"issue":"1","key":"2023091116080363100_bib17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1117\/1.JEI.31.1.010901","article-title":"Review of spike-based neuromorphic computing for brain-inspired vision: Biology, algorithms, and hardware","volume":"31","author":"Hendy","year":"2022","journal-title":"Journal of Electronic Imaging"},{"key":"2023091116080363100_bib18","article-title":"Hybrid macro\/micro level backpropagation for training deep spiking neural networks","volume-title":"Advances in neural information processing systems","author":"Jin","year":"2018"},{"key":"2023091116080363100_bib19","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065720500276","article-title":"Temporal backpropagation for spiking neural networks with one spike per neuron","volume":"30","author":"Kheradpisheh","year":"2020","journal-title":"International Journal of Neural Systems"},{"issue":"2","key":"2023091116080363100_bib20","doi-asserted-by":"publisher","first-page":"1255","DOI":"10.1007\/s11063-021-10680-x","article-title":"BS4NN: Binarized spiking neural networks with temporal coding and learning","volume":"54","author":"Kheradpisheh","year":"2021","journal-title":"Neural Processing Letters"},{"key":"2023091116080363100_bib21","first-page":"1","article-title":"Efficient hardware implementation of STDP for AER-based large-scale SNN neuromorphic system","volume-title":"Proceedings of the 35th International Technical Conference on Circuits\/Systems, Computers and Communications","author":"Kim","year":"2020"},{"key":"2023091116080363100_bib22","article-title":"Adam: A method for stochastic optimization","volume-title":"Proceedings of the 3rd International Conference on Learning Representations","author":"Kingma","year":"2015"},{"key":"2023091116080363100_bib23","article-title":"ImageNet classification with deep convolutional neural networks","volume-title":"Advances in neural information processing systems","author":"Krizhevsky","year":"2012"},{"key":"2023091116080363100_bib24","article-title":"MNIST handwritten digit database. ATT Labs","author":"LeCun","year":"2010"},{"key":"2023091116080363100_bib25","doi-asserted-by":"crossref","DOI":"10.3389\/fnins.2016.00508","article-title":"Training deep spiking neural networks using backpropagation","volume":"10","author":"Lee","year":"2016","journal-title":"Frontiers in Neuroscience"},{"key":"2023091116080363100_bib26","first-page":"23426","article-title":"Differentiable spike: Rethinking gradient-descent for training spiking neural networks","volume-title":"Advances in neural information processing systems","author":"Li","year":"2021"},{"issue":"9","key":"2023091116080363100_bib27","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1016\/S0893-6080(97)00011-7","article-title":"Networks of spiking neurons: The third generation of neural network models","volume":"10","author":"Maass","year":"1997","journal-title":"Neural Networks"},{"key":"2023091116080363100_bib28","first-page":"1928","article-title":"Asynchronous methods for deep reinforcement learning","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Mnih","year":"2016"},{"key":"2023091116080363100_bib29","author":"Mnih","year":"2013","journal-title":"Playing Atari with deep reinforcement learning."},{"issue":"7","key":"2023091116080363100_bib30","article-title":"Supervised learning based on temporal coding in spiking neural networks","volume":"29","author":"Mostafa","year":"2016","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2023091116080363100_bib31","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2017.00324","article-title":"Event-driven random back-propagation: Enabling neuromorphic deep learning machines","volume":"11","author":"Neftci","year":"2017","journal-title":"Frontiers in Neuroscience"},{"key":"2023091116080363100_bib32","article-title":"CuPy: A numPy-compatible library for NVIDIA GPU calculations","author":"Okuta","year":"2017","journal-title":"Proceedings of Workshop on Machine Learning Systems in the Thirty-First Annual Conference on Neural Information Processing Systems"},{"issue":"8","key":"2023091116080363100_bib33","doi-asserted-by":"publisher","first-page":"1943","DOI":"10.1109\/JSSC.2013.2259038","article-title":"SpiNNaker: A 1-W 18-core system-on-chip for massively-parallel neural network simulation","volume":"48","author":"Painkras","year":"2013","journal-title":"IEEE Journal of Solid-State Circuits"},{"key":"2023091116080363100_bib34","first-page":"373","article-title":"Noise-robust deep spiking neural networks with temporal information","volume-title":"Proceedings of the 58th ACM\/IEEE Design Automation Conference","author":"Park","year":"2021"},{"key":"2023091116080363100_bib35","author":"Schmitt","year":"2017","journal-title":"Neuromorphic hardware in the loop: Training a deep spiking network on the brainscales wafer-scale system"},{"key":"2023091116080363100_bib36","article-title":"SLAYER: Spike layer error reassignment in time","volume-title":"Advances in neural information processing systems","author":"Shrestha","year":"2018"},{"key":"2023091116080363100_bib37","doi-asserted-by":"crossref","first-page":"958","DOI":"10.1109\/ICDAR.2003.1227801","article-title":"Best practices for convolutional neural networks applied to visual document analysis","volume-title":"Proceedings of the Seventh International Conference on Document Analysis and Recognition","author":"Simard","year":"2003"},{"key":"2023091116080363100_bib38","article-title":"Deep neural networks for object detection","volume-title":"Advances in neural information processing systems","author":"Szegedy","year":"2013"},{"key":"2023091116080363100_bib39","author":"Taunyazov","year":"2020","journal-title":"Event-driven visual-tactile sensing and learning for robots."},{"key":"2023091116080363100_bib40","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1038\/381520a0","article-title":"Speed of processing in the human visual system","volume":"381","author":"Thorpe","year":"1996","journal-title":"Nature"},{"key":"2023091116080363100_bib41","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"2023091116080363100_bib42","doi-asserted-by":"crossref","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Frontiers in Neuroscience"},{"issue":"1","key":"2023091116080363100_bib43","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-91786-z","article-title":"Event-based backpropagation can compute exact gradients for spiking neural networks","volume":"11","author":"Wunderlich","year":"2021","journal-title":"Scientific Reports"},{"key":"2023091116080363100_bib44","article-title":"Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms","author":"Xiao","year":"2017"},{"key":"2023091116080363100_bib45","doi-asserted-by":"crossref","DOI":"10.3389\/fnins.2022.760298","article-title":"Backpropagation with sparsity regularization for spiking neural network learning","volume":"16","author":"Yan","year":"2022","journal-title":"Frontiers in Neuroscience"},{"issue":"10","key":"2023091116080363100_bib46","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1038\/s42256-021-00397-w","article-title":"Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks","volume":"3","author":"Yin","year":"2021","journal-title":"Nature Machine Intelligence"},{"key":"2023091116080363100_bib47","author":"Yin","year":"2023","journal-title":"Workload-balanced pruning for sparse spiking neural networks"},{"key":"2023091116080363100_bib48","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2018.00987","article-title":"Sparse computation in adaptive spiking neural networks","volume":"12","author":"Zambrano","year":"2019","journal-title":"Frontiers in Neuroscience"},{"issue":"5","key":"2023091116080363100_bib49","doi-asserted-by":"publisher","first-page":"1947","DOI":"10.1109\/TNNLS.2021.3110991","article-title":"Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks","volume":"33","author":"Zhang","year":"2022","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2023091116080363100_bib50","article-title":"Spike-train level backpropagation for training deep recurrent spiking neural networks","volume-title":"Advances in neural information processing systems","author":"Zhang","year":"2019"},{"key":"2023091116080363100_bib51","first-page":"12022","article-title":"Temporal spike sequence learning via backpropagation for deep spiking neural networks","volume-title":"Advances in neural information processing systems","author":"Zhang","year":"2020"},{"issue":"12","key":"2023091116080363100_bib52","doi-asserted-by":"publisher","first-page":"11143","DOI":"10.1609\/aaai.v35i12.17329","article-title":"Temporal-coded deep spiking neural network with easy training and robust performance","volume":"35","author":"Zhou","year":"2021","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/10\/1627\/2157828\/neco_a_01609.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/10\/1627\/2157828\/neco_a_01609.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,11]],"date-time":"2023-09-11T16:08:26Z","timestamp":1694448506000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/10\/1627\/117019\/Exploring-Trade-Offs-in-Spiking-Neural-Networks"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,8]]},"references-count":52,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,9,8]]},"published-print":{"date-parts":[[2023,9,8]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01609","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,10]]},"published":{"date-parts":[[2023,9,8]]}}}