{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T20:58:46Z","timestamp":1778446726473,"version":"3.51.4"},"reference-count":65,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2022,9,30]],"date-time":"2022-09-30T00:00:00Z","timestamp":1664496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,9,30]],"date-time":"2022-09-30T00:00:00Z","timestamp":1664496000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100002418","name":"Intel Corporation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100002418","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000089","name":"Office of International Science and Engineering","doi-asserted-by":"crossref","award":["2020624"],"award-info":[{"award-number":["2020624"]}],"id":[{"id":"10.13039\/100000089","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Science Foundation","award":["1652159"],"award-info":[{"award-number":["1652159"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Adaptive \u2018life-long\u2019 learning at the edge and during online task performance is an aspirational goal of artificial intelligence research. Neuromorphic hardware implementing spiking neural networks (SNNs) are particularly attractive in this regard, as their real-time, event-based, local computing paradigm makes them suitable for edge implementations and fast learning. However, the long and iterative learning that characterizes state-of-the-art SNN training is incompatible with the physical nature and real-time operation of neuromorphic hardware. Bi-level learning, such as meta-learning is increasingly used in deep learning to overcome these limitations. In this work, we demonstrate gradient-based meta-learning in SNNs using the surrogate gradient method that approximates the spiking threshold function for gradient estimations. Because surrogate gradients can be made twice differentiable, well-established, and effective second-order gradient meta-learning methods such as model agnostic meta learning (MAML) can be used. We show that SNNs meta-trained using MAML perform comparably to conventional artificial neural networks meta-trained with MAML on event-based meta-datasets. Furthermore, we demonstrate the specific advantages that accrue from meta-learning: fast learning without the requirement of high precision weights or gradients, training-to-learn with quantization and mitigating the effects of approximate synaptic plasticity rules. Our results emphasize how meta-learning techniques can become instrumental for deploying neuromorphic learning technologies on real-world problems.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac8828","type":"journal-article","created":{"date-parts":[[2022,8,9]],"date-time":"2022-08-09T22:18:35Z","timestamp":1660083515000},"page":"044002","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["Meta-learning spiking neural networks with surrogate gradient descent"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7719-5796","authenticated-orcid":true,"given":"Kenneth M","family":"Stewart","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emre O","family":"Neftci","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2022,9,30]]},"reference":[{"key":"nceac8828bib1","doi-asserted-by":"publisher","first-page":"1629","DOI":"10.1109\/5.58356","article-title":"Neuromorphic electronic systems","volume":"78","author":"Mead","year":"1990","journal-title":"Proc. IEEE"},{"key":"nceac8828bib2","doi-asserted-by":"publisher","first-page":"123","DOI":"10.3389\/fnins.2011.00073","article-title":"Neuromorphic silicon neuron circuits","volume":"5","author":"Indiveri","year":"2011","journal-title":"Front. Neurosci."},{"key":"nceac8828bib3","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1038\/s42256-019-0097-1","article-title":"Benchmarks for progress in neuromorphic computing","volume":"1","author":"Davies","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"nceac8828bib4","doi-asserted-by":"publisher","first-page":"1367","DOI":"10.1109\/JPROC.2014.2313954","article-title":"Neuromorphic electronic circuits for building autonomous cognitive systems","volume":"102","author":"Chicca","year":"2013","journal-title":"Proc. IEEE"},{"key":"nceac8828bib5","doi-asserted-by":"publisher","first-page":"90","DOI":"10.3389\/fnins.2012.00090","article-title":"Is a 4-bit synaptic weight resolution enough? constraints on enabling spike timing dependent plasticity in neuromorphic hardware","volume":"6","author":"Pfeil","year":"2012","journal-title":"Front. Neurosci."},{"key":"nceac8828bib6","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1109\/mm.2018.112130359","article-title":"Loihi: a neuromorphic manycore processor with on-chip learning","volume":"99","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"nceac8828bib7","doi-asserted-by":"publisher","first-page":"5268","DOI":"10.1016\/j.isci.2018.06.010","article-title":"Data and power efficient intelligence with neuromorphic learning machines","volume":"5","author":"Neftci","year":"2018","journal-title":"iScience"},{"key":"nceac8828bib8","doi-asserted-by":"publisher","first-page":"516","DOI":"10.1109\/msp.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"nceac8828bib9","doi-asserted-by":"publisher","first-page":"424","DOI":"10.3389\/fnins.2020.00424","article-title":"Synaptic plasticity for deep continuous local learning","volume":"14","author":"Kaiser","year":"2019","journal-title":"Front. Neurosci."},{"key":"nceac8828bib10","article-title":"Training spiking multilayer networks with surrogate gradients on an analog neuromorphic substrate","author":"Cramer","year":"2020"},{"key":"nceac8828bib11","article-title":"Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets","author":"Bellec","year":"2019"},{"key":"nceac8828bib12","article-title":"Online spatiotemporal learning in deep neural networks","author":"Bohnstingl","year":"2020"},{"key":"nceac8828bib13","doi-asserted-by":"publisher","first-page":"1514","DOI":"10.1162\/neco_a_01086","article-title":"SuperSpike: supervised learning in multilayer spiking neural networks","volume":"30","author":"Zenke","year":"2018","journal-title":"Neural Comput."},{"key":"nceac8828bib14","doi-asserted-by":"publisher","first-page":"324","DOI":"10.3389\/fnins.2017.00324","article-title":"Event-driven random backpropagation: enabling neuromorphic deep learning machines","volume":"11","author":"Neftci","year":"2017","journal-title":"Front. Neurosci."},{"key":"nceac8828bib15","author":"Vapnik","year":"2013"},{"key":"nceac8828bib16","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1037\/0033-295x.102.3.419","article-title":"Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory","volume":"102","author":"McClelland","year":"1995","journal-title":"Psychol. Rev."},{"key":"nceac8828bib17","first-page":"p 217","article-title":"Large scale online learning","volume":"vol 16","author":"LeCun","year":"2004"},{"key":"nceac8828bib18","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1038\/s41467-021-27653-2","article-title":"Brain-inspired global-local learning incorporated with neuromorphic computing","volume":"13","author":"Wu","year":"2022","journal-title":"Nat. Commun."},{"key":"nceac8828bib19","doi-asserted-by":"publisher","first-page":"53","DOI":"10.3389\/fncir.2018.00053","article-title":"Eligibility traces and plasticity on behavioral time scales: experimental support of neoHebbian three-factor learning rules","volume":"12","author":"Gerstner","year":"2018","journal-title":"Front. Neural Circuits"},{"key":"nceac8828bib20","doi-asserted-by":"publisher","first-page":"424","DOI":"10.3389\/fnins.2020.00424","article-title":"Synaptic plasticity dynamics for deep continuous local learning (DECOLLE)","volume":"14","author":"Kaiser","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac8828bib21","doi-asserted-by":"crossref","DOI":"10.1109\/AICAS48895.2020.9073998","article-title":"Error-triggered three-factor learning dynamics for crossbar arrays","author":"Payvand","year":"2020"},{"key":"nceac8828bib22","doi-asserted-by":"publisher","DOI":"10.1017\/s0140525x16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2017","journal-title":"Behav. Brain Sci."},{"key":"nceac8828bib23","article-title":"Matching networks for one shot learning","author":"Vinyals","year":"2016"},{"key":"nceac8828bib24","article-title":"N-Omniglot: a large-scale neuromorphic dataset for spatiotemporal sparse few-shot learning","author":"Li","year":"2021"},{"key":"nceac8828bib25","article-title":"Multi-digit MNIST for few-shot learning","author":"Sun","year":"2019"},{"key":"nceac8828bib26","author":"Griewank","year":"2008"},{"key":"nceac8828bib27","first-page":"1126","article-title":"Model-agnostic meta-learning for fast adaptation of deep networks","volume":"vol 70","author":"Finn","year":"2017"},{"key":"nceac8828bib28","article-title":"On first-order meta-learning algorithms","author":"Nichol","year":"2018"},{"key":"nceac8828bib29","author":"Gerstner","year":"2002"},{"key":"nceac8828bib30","doi-asserted-by":"crossref","DOI":"10.1101\/2020.06.29.176925","article-title":"The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks","author":"Zenke","year":"2020"},{"key":"nceac8828bib31","doi-asserted-by":"publisher","first-page":"5595","DOI":"10.5555\/3122009.3242010","article-title":"Automatic differentiation in machine learning: a survey","volume":"18","author":"Baydin","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"nceac8828bib32","article-title":"Automatic differentiation in PyTorch","author":"Paszke","year":"2017"},{"key":"nceac8828bib33","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1109\/jssc.2007.914337","article-title":"A 128 \u00d7 128 120 dB 15 \u03bcs latency asynchronous temporal contrast vision sensor","volume":"43","author":"Lichtsteiner","year":"2008","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceac8828bib34","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1109\/jssc.2010.2085952","article-title":"A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS","volume":"46","author":"Posch","year":"2011","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceac8828bib35","doi-asserted-by":"publisher","first-page":"437","DOI":"10.3389\/fnins.2015.00437","article-title":"Converting static image datasets to spiking neuromorphic datasets using saccades","volume":"9","author":"Orchard","year":"2015","journal-title":"Front. Neurosci."},{"key":"nceac8828bib36","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2019.00058","article-title":"Graph-based object classification for neuromorphic vision sensing","author":"Bi","year":"2019"},{"key":"nceac8828bib37","doi-asserted-by":"publisher","first-page":"2333","DOI":"10.1109\/jssc.2014.2342715","article-title":"A 240 \u00d7 180 130 dB 3 \u03bcs latency global shutter spatiotemporal vision sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceac8828bib38","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.1126\/science.aab3050","article-title":"Human-level concept learning through probabilistic program induction","volume":"350","author":"Lake","year":"2015","journal-title":"Science"},{"key":"nceac8828bib39","first-page":"1412","article-title":"Slayer: spike layer error reassignment in time","author":"Shrestha","year":"2018"},{"key":"nceac8828bib40","article-title":"Rapid learning or feature reuse? Towards understanding the effectiveness of MAML","author":"Raghu","year":"2020"},{"key":"nceac8828bib41","article-title":"Meta-learning and universality: deep representations and gradient descent can approximate any learning algorithm","author":"Finn","year":"2018"},{"key":"nceac8828bib42","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1109\/jetcas.2020.3032058","article-title":"Online few-shot gesture learning on a neuromorphic processor","volume":"10","author":"Stewart","year":"2020","journal-title":"IEEE J. Emerg. Sel. Top. Circuits Syst."},{"key":"nceac8828bib43","first-page":"3981","article-title":"Learning to learn by gradient descent by gradient descent","author":"Andrychowicz","year":"2016"},{"key":"nceac8828bib44","first-page":"35","author":"Kotthoff","year":"2019"},{"key":"nceac8828bib45","article-title":"Rounding methods for neural networks with low resolution synaptic weights","author":"Muller","year":"2015"},{"key":"nceac8828bib46","doi-asserted-by":"publisher","first-page":"222","DOI":"10.3389\/fnins.2015.00222","article-title":"Robustness of spiking deep belief networks to noise and reduced bit precision of neuro-inspired hardware platforms","volume":"9","author":"Stromatias","year":"2015","journal-title":"Front. Neurosci."},{"key":"nceac8828bib47","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41586-018-0180-5","article-title":"Equivalent-accuracy accelerated neural-network training using analogue memory","volume":"558","author":"Ambrogio","year":"2018","journal-title":"Nature"},{"key":"nceac8828bib48","doi-asserted-by":"publisher","first-page":"935","DOI":"10.1109\/jproc.2020.3045625","article-title":"Brain-inspired learning on neuromorphic substrates","volume":"116","author":"Zenke","year":"2021","journal-title":"Proc. IEEE"},{"key":"nceac8828bib49","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41467-018-07757-y","article-title":"Spike-timing-dependent plasticity learning of coincidence detection with passively integrated memristive circuits","volume":"9","author":"Prezioso","year":"2018","journal-title":"Nat. Commun."},{"key":"nceac8828bib50","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.773954","article-title":"Revisiting batch normalization for training low-latency deep spiking neural networks from scratch","volume":"15","author":"Kim","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac8828bib51","doi-asserted-by":"crossref","DOI":"10.24963\/ijcai.2021\/321","article-title":"Optimal ANN-SNN conversion for fast and accurate inference in deep spiking neural networks","author":"Ding","year":"2021"},{"key":"nceac8828bib52","doi-asserted-by":"publisher","first-page":"682","DOI":"10.3389\/fnins.2017.00682","article-title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","volume":"11","author":"Rueckauer","year":"2017","journal-title":"Front. Neurosci."},{"key":"nceac8828bib53","doi-asserted-by":"publisher","first-page":"1997","DOI":"10.5555\/3322706.3361996","article-title":"Neural architecture search: a survey","volume":"20","author":"Elsken","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"nceac8828bib54","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-20053-3_3","article-title":"Neural architecture search for spiking neural networks","author":"Kim","year":"2022"},{"key":"nceac8828bib55","doi-asserted-by":"crossref","DOI":"10.1101\/2020.06.17.156513","article-title":"One-shot learning with spiking neural networks","author":"Scherr","year":"2020"},{"key":"nceac8828bib56","article-title":"Signature verification using a \u2018Siamese\u2019 time delay neural network","volume":"vol 6","author":"Bromley","year":"1993"},{"key":"nceac8828bib57","article-title":"Siamese neural networks for one-shot image recognition","volume":"vol 2","author":"Koch","year":"2015"},{"key":"nceac8828bib58","doi-asserted-by":"publisher","first-page":"2439","DOI":"10.1162\/neco_a_01423","article-title":"Few-shot learning in spiking neural networks by multi-timescale optimization","volume":"33","author":"Jiang","year":"2021","journal-title":"Neural Comput."},{"key":"nceac8828bib59","first-page":"7243","article-title":"A low power, fully event-based gesture recognition system","author":"Amir","year":"2017"},{"key":"nceac8828bib60","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2022.850932","article-title":"Heterogeneous ensemble-based spike-driven few-shot online learning","volume":"16","author":"Yang","year":"2022","journal-title":"Front. Neurosci."},{"key":"nceac8828bib61","doi-asserted-by":"crossref","DOI":"10.1109\/DSLW51110.2021.9523405","article-title":"Fast on-device adaptation for spiking neural networks via online-within-online meta-learning","author":"Rosenfeld","year":"2021"},{"key":"nceac8828bib62","first-page":"3559","article-title":"Differentiable plasticity: training plastic neural networks with backpropagation","author":"Miconi","year":"2018"},{"key":"nceac8828bib63","article-title":"Brain-inspired global-local hybrid learning towards human-like intelligence","author":"Wu","year":"2020"},{"key":"nceac8828bib64","article-title":"Event-based vision: a survey","author":"Gallego","year":"2019"},{"key":"nceac8828bib65","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.conb.2010.03.007","article-title":"Neuromorphic sensory systems","volume":"20","author":"Liu","year":"2010","journal-title":"Curr. Opin. Neurobiol."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T04:01:35Z","timestamp":1676347295000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac8828"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,30]]},"references-count":65,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,9,30]]},"published-print":{"date-parts":[[2022,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac8828","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,30]]},"assertion":[{"value":"Meta-learning spiking neural networks with surrogate gradient descent","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-01-26","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-08-09","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-09-30","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}