{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T22:38:52Z","timestamp":1776465532754,"version":"3.51.2"},"reference-count":58,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2022,12,8]],"date-time":"2022-12-08T00:00:00Z","timestamp":1670457600000},"content-version":"vor","delay-in-days":7,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,12,8]],"date-time":"2022-12-08T00:00:00Z","timestamp":1670457600000},"content-version":"tdm","delay-in-days":7,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"Novartis Stiftung f\u00fcr Medizinisch-Biologische Forschung"},{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"crossref","award":["PCEFP3_202981"],"award-info":[{"award-number":["PCEFP3_202981"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Spiking neural networks (SNNs) underlie low-power, fault-tolerant information processing in the brain and could constitute a power-efficient alternative to conventional deep neural networks when implemented on suitable neuromorphic hardware accelerators. However, instantiating SNNs that solve complex computational tasks\n                    <jats:italic>in-silico<\/jats:italic>\n                    remains a significant challenge. Surrogate gradient (SG) techniques have emerged as a standard solution for training SNNs end-to-end. Still, their success depends on synaptic weight initialization, similar to conventional artificial neural networks (ANNs). Yet, unlike in the case of ANNs, it remains elusive what constitutes a good initial state for an SNN. Here, we develop a general initialization strategy for SNNs inspired by the fluctuation-driven regime commonly observed in the brain. Specifically, we derive practical solutions for data-dependent weight initialization that ensure fluctuation-driven firing in the widely used leaky integrate-and-fire neurons. We empirically show that SNNs initialized following our strategy exhibit superior learning performance when trained with SGs. These findings generalize across several datasets and SNN architectures, including fully connected, deep convolutional, recurrent, and more biologically plausible SNNs obeying Dale\u2019s law. Thus fluctuation-driven initialization provides a practical, versatile, and easy-to-implement strategy for improving SNN training performance on diverse tasks in neuromorphic engineering and computational neuroscience.\n                  <\/jats:p>","DOI":"10.1088\/2634-4386\/ac97bb","type":"journal-article","created":{"date-parts":[[2022,10,5]],"date-time":"2022-10-05T18:17:48Z","timestamp":1664993868000},"page":"044016","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":30,"title":["Fluctuation-driven initialization for spiking neural network training"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1927-8198","authenticated-orcid":true,"given":"Julian","family":"Rossbroich","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7880-6694","authenticated-orcid":true,"given":"Julia","family":"Gygax","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1883-644X","authenticated-orcid":true,"given":"Friedemann","family":"Zenke","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2022,12,8]]},"reference":[{"key":"nceac97bbbib1","author":"Sterling","year":"2017"},{"key":"nceac97bbbib2","doi-asserted-by":"publisher","first-page":"73","DOI":"10.3389\/fnins.2011.00073","article-title":"Neuromorphic silicon neural circuits","volume":"5","author":"Indiveri","year":"2011","journal-title":"Front. Neurosci."},{"key":"nceac97bbbib3","article-title":"Exponential expressivity in deep neural networks through transient chaos","volume":"vol 29","author":"Poole","year":"2016"},{"key":"nceac97bbbib4","article-title":"Spiking deep networks with LIF neurons","author":"Hunsberger","year":"2015"},{"key":"nceac97bbbib5","doi-asserted-by":"publisher","first-page":"899","DOI":"10.1162\/neco_a_01367","article-title":"The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks","volume":"33","author":"Zenke","year":"2021","journal-title":"Neural Comput."},{"key":"nceac97bbbib6","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/msp.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"nceac97bbbib7","article-title":"Untersuchungen zu dynamischen neuronalen Netzen","author":"Hochreiter","year":"1991"},{"key":"nceac97bbbib8","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"nceac97bbbib9","article-title":"On the difficulty of training recurrent neural networks","author":"Pascanu","year":"2013"},{"key":"nceac97bbbib10","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","author":"Glorot","year":"2010"},{"key":"nceac97bbbib11","first-page":"1026","article-title":"Delving deep into rectifiers: surpassing humanlevel performance on ImageNet classification","author":"He","year":"2015"},{"key":"nceac97bbbib12","article-title":"All you need is a good init","author":"Mishkin","year":"2015"},{"key":"nceac97bbbib13","article-title":"Training very deep networks","author":"Srivastava","year":"2015"},{"key":"nceac97bbbib14","doi-asserted-by":"publisher","first-page":"508","DOI":"10.3389\/fnins.2016.00508","article-title":"Training deep spiking neural networks using backpropagation","volume":"10","author":"Lee","year":"2016","journal-title":"Front. Neurosci."},{"key":"nceac97bbbib15","article-title":"Training deep spiking neural networks","author":"Ledinauskas","year":"2020"},{"key":"nceac97bbbib16","doi-asserted-by":"publisher","first-page":"8413","DOI":"10.1103\/physreve.62.8413","article-title":"Comparison of current-driven and conductance-driven neocortical model neurons with Hodgkin\u2013Huxley voltage-gated channels","volume":"62","author":"Tiesinga","year":"2000","journal-title":"Phys. Rev. E"},{"key":"nceac97bbbib17","doi-asserted-by":"publisher","first-page":"2345","DOI":"10.1523\/JNEUROSCI.3349-03.2004","article-title":"Neuronal integration of synaptic input in the fluctuation-driven regime","volume":"24","author":"Kuhn","year":"2004","journal-title":"J. Neurosci."},{"key":"nceac97bbbib18","doi-asserted-by":"publisher","DOI":"10.7554\/elife.18805","article-title":"Lognormal firing rate distribution reveals prominent fluctuation-driven regime in spinal motor networks","volume":"5","author":"Petersen","year":"2016","journal-title":"eLife"},{"key":"nceac97bbbib19","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1146\/annurev.neuro.28.061604.135637","article-title":"Neural network dynamics","volume":"28","author":"Vogels","year":"2005","journal-title":"Annu. Rev. Neurosci."},{"key":"nceac97bbbib20","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1023\/A:1008925309027","article-title":"Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons","volume":"8","author":"Brunel","year":"2000","journal-title":"J. Comput. Neurosci."},{"key":"nceac97bbbib21","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1093\/cercor\/7.3.237","article-title":"Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex","volume":"7","author":"Amit","year":"1997","journal-title":"Cerebral Cortex"},{"key":"nceac97bbbib22","author":"Gerstner","year":"2014"},{"key":"nceac97bbbib23","doi-asserted-by":"publisher","first-page":"2744","DOI":"10.1109\/TNNLS.2020.3044364","article-title":"The Heidelberg spiking data sets for the systematic evaluation of spiking neural networks","volume":"33","author":"Cramer","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceac97bbbib24","doi-asserted-by":"crossref","DOI":"10.1038\/s42256-021-00397-w","article-title":"Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks","author":"Yin","year":"2021"},{"key":"nceac97bbbib25","article-title":"RMSprop loses to SMORMS3\u2014beware the epsilon!","author":"Funk","year":"2022"},{"key":"nceac97bbbib26","article-title":"ADAM: a method for stochastic optimization","author":"Kingma","year":"2017"},{"key":"nceac97bbbib27","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1038\/nrn1327","article-title":"Homeostatic plasticity in the developing nervous system","volume":"5","author":"Turrigiano","year":"2004","journal-title":"Nat. Rev. Neurosci."},{"key":"nceac97bbbib28","doi-asserted-by":"publisher","first-page":"3722","DOI":"10.1523\/jneurosci.2511-15.2016","article-title":"Homeostatic activity-dependent tuning of recurrent networks for robust propagation of activity","volume":"36","author":"Gjorgjieva","year":"2016","journal-title":"J. Neurosci."},{"key":"nceac97bbbib29","doi-asserted-by":"publisher","first-page":"20160259","DOI":"10.1098\/rstb.2016.0259","article-title":"Hebbian plasticity requires compensatory processes on multiple timescales","volume":"372","author":"Zenke","year":"2017","journal-title":"Phil. Trans. R. Soc. B"},{"key":"nceac97bbbib30","article-title":"Highway networks","author":"Srivastava","year":"2015"},{"key":"nceac97bbbib31","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"nceac97bbbib32","first-page":"7388","article-title":"A low power, fully event-based gesture recognition system","author":"Amir","year":"2017"},{"key":"nceac97bbbib33","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1113\/jphysiol.1954.sp005226","article-title":"Cholinergic and inhibitory synapses in a pathway from motor-axon collaterals to motoneurones","volume":"126","author":"Eccles","year":"1954","journal-title":"J. Physiol."},{"key":"nceac97bbbib34","doi-asserted-by":"publisher","first-page":"669","DOI":"10.1016\/j.neuron.2018.09.013","article-title":"Precise synaptic balance in the zebrafish homolog of olfactory cortex","volume":"100","author":"Rupprecht","year":"2018","journal-title":"Neuron"},{"key":"nceac97bbbib35","doi-asserted-by":"publisher","first-page":"1216","DOI":"10.1016\/j.cell.2014.03.058","article-title":"Npas4 regulates excitatory\u2013inhibitory balance within neural circuits through cell-type-specific gene programs","volume":"157","author":"Spiegel","year":"2014","journal-title":"Cell"},{"key":"nceac97bbbib36","article-title":"Backpropagation for energy-efficient neuromorphic computing","volume":"vol 28","author":"Esser","year":"2015"},{"key":"nceac97bbbib37","article-title":"Training spiking deep networks for neuromorphic hardware","author":"Hunsberger","year":"2016"},{"key":"nceac97bbbib38","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1007\/s11263-014-0788-3","article-title":"Spiking deep convolutional neural networks for energy-efficient object recognition","volume":"113","author":"Cao","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"nceac97bbbib39","doi-asserted-by":"publisher","first-page":"178","DOI":"10.3389\/fnins.2013.00178","article-title":"Real-time classification and sensor fusion with a spiking deep belief network","volume":"7","author":"O\u2019Connor","year":"2013","journal-title":"Front. Neurosci."},{"key":"nceac97bbbib40","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v36i1.19874","article-title":"Optimized potential initialization for low-latency spiking neural networks","author":"Bu","year":"2022"},{"key":"nceac97bbbib41","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/s0925-2312(01)00658-0","article-title":"Error-backpropagation in temporally encoded networks of spiking neurons","volume":"48","author":"Bohte","year":"2002","journal-title":"Neurocomputing"},{"key":"nceac97bbbib42","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1016\/j.ipl.2005.05.023","article-title":"A gradient descent rule for spiking neurons emitting multiple spikes","volume":"95","author":"Booij","year":"2005","journal-title":"Inf. Process. Lett."},{"key":"nceac97bbbib43","doi-asserted-by":"publisher","first-page":"3227","DOI":"10.1109\/TNNLS.2017.2726060","article-title":"Supervised learning based on temporal coding in spiking neural networks","volume":"29","author":"Mostafa","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceac97bbbib44","doi-asserted-by":"publisher","first-page":"2050027","DOI":"10.1142\/s0129065720500276","article-title":"Temporal backpropagation for spiking neural networks with one spike per neuron","volume":"30","author":"Kheradpisheh","year":"2020","journal-title":"Int. J. Neural Syst."},{"key":"nceac97bbbib45","first-page":"8529","article-title":"Temporal coding in spiking neural networks with alpha synaptic function","author":"Comsa","year":"2020"},{"key":"nceac97bbbib46","doi-asserted-by":"publisher","first-page":"1514","DOI":"10.1162\/neco_a_01086","article-title":"SuperSpike: supervised learning in multilayer spiking neural networks","volume":"30","author":"Zenke","year":"2018","journal-title":"Neural Comput."},{"key":"nceac97bbbib47","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.651141","article-title":"Comparison of artificial and spiking neural networks on digital hardware","volume":"15","author":"Davidson","year":"2021","journal-title":"Front. Neurosci."},{"key":"nceac97bbbib48","doi-asserted-by":"publisher","first-page":"437","DOI":"10.3389\/fnins.2015.00437","article-title":"Converting static image datasets to spiking neuromorphic datasets using saccades","volume":"9","author":"Orchard","year":"2015","journal-title":"Front. Neurosci."},{"key":"nceac97bbbib49","first-page":"795","article-title":"Long short-term memory and learning-to-learn in networks of spiking neurons","author":"Bellec","year":"2018"},{"key":"nceac97bbbib50","article-title":"Surrogate gradients design","author":"Herranz-Celotti","year":"2022"},{"key":"nceac97bbbib51","article-title":"Accelerating training of deep spiking neural networks with parameter initialization","author":"Ding","year":"2022"},{"key":"nceac97bbbib52","article-title":"AutoSNN: towards energy efficient spiking neural networks","author":"Na","year":"2022"},{"key":"nceac97bbbib53","doi-asserted-by":"publisher","first-page":"94","DOI":"10.3389\/fncom.2016.00094","article-title":"Toward an integration of deep learning and neuroscience","volume":"10","author":"Marblestone","year":"2016","journal-title":"Front. Comput. Neurosci."},{"key":"nceac97bbbib54","doi-asserted-by":"publisher","first-page":"1761","DOI":"10.1038\/s41593-019-0520-2","article-title":"A deep learning framework for neuroscience","volume":"22","author":"Richards","year":"2019","journal-title":"Nat. Neurosci."},{"key":"nceac97bbbib55","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1038\/s41583-020-0277-3","article-title":"Backpropagation and the brain","volume":"21","author":"Lillicrap","year":"2020","journal-title":"Nat. Rev. Neurosci."},{"key":"nceac97bbbib56","article-title":"Learning multiple layers of features from tiny images","author":"Krizhevsky","year":"2009"},{"key":"nceac97bbbib57","first-page":"8024","article-title":"PyTorch: an imperative style, high-performance deep learning library","volume":"vol 32","author":"Paszke","year":"2019"},{"key":"nceac97bbbib58","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1038\/nrn3687","article-title":"The log-dynamic brain: how skewed distributions affect network operations","volume":"15","author":"Buzs\u00e1ki","year":"2014","journal-title":"Nat. Rev. Neurosci."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T11:03:29Z","timestamp":1670411009000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac97bb"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,1]]},"references-count":58,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,12,8]]},"published-print":{"date-parts":[[2022,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac97bb","relation":{"has-review":[{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v2\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/AC97BB\/v2\/decision1","asserted-by":"object"}]},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,1]]},"assertion":[{"value":"Fluctuation-driven initialization for spiking neural network training","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-06-21","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-10-05","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-12-08","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}