{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T18:49:49Z","timestamp":1785350989164,"version":"3.55.0"},"reference-count":75,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2022,12,8]],"date-time":"2022-12-08T00:00:00Z","timestamp":1670457600000},"content-version":"vor","delay-in-days":7,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,12,8]],"date-time":"2022-12-08T00:00:00Z","timestamp":1670457600000},"content-version":"tdm","delay-in-days":7,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Spiking neural networks (SNNs) have recently emerged as the low-power alternative to artificial neural networks (ANNs) because of their sparse, asynchronous, and binary event-driven processing. Due to their energy efficiency, SNNs have a high possibility of being deployed for real-world, resource-constrained systems such as autonomous vehicles and drones. However, owing to their non-differentiable and complex neuronal dynamics, most previous SNN optimization methods have been limited to image recognition. In this paper, we explore the SNN applications beyond classification and present semantic segmentation networks configured with spiking neurons. Specifically, we first investigate two representative SNN optimization techniques for recognition tasks (i.e., ANN-SNN conversion and surrogate gradient learning) on semantic segmentation datasets. We observe that, when converted from ANNs, SNNs suffer from high latency and low performance due to the spatial variance of features. Therefore, we directly train networks with surrogate gradient learning, resulting in lower latency and higher performance than ANN-SNN conversion. Moreover, we redesign two fundamental ANN segmentation architectures (i.e., Fully Convolutional Networks and DeepLab) for the SNN domain. We conduct experiments on three semantic segmentation benchmarks including PASCAL VOC2012 dataset, DDD17 event-based dataset, and synthetic segmentation dataset combined CIFAR10 and MNIST datasets. In addition to showing the feasibility of SNNs for semantic segmentation, we show that SNNs can be more robust and energy-efficient compared to their ANN counterparts in this domain.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac9b86","type":"journal-article","created":{"date-parts":[[2022,10,19]],"date-time":"2022-10-19T22:22:06Z","timestamp":1666218126000},"page":"044015","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":70,"title":["Beyond classification: directly training spiking neural networks for semantic segmentation"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3542-7720","authenticated-orcid":true,"given":"Youngeun","family":"Kim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua","family":"Chough","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Priyadarshini","family":"Panda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2022,12,8]]},"reference":[{"key":"nceac9b86bib1","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"nceac9b86bib2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2015"},{"key":"nceac9b86bib3","first-page":"1440","article-title":"Fast R-CNN","author":"Girshick","year":"2015"},{"key":"nceac9b86bib4","doi-asserted-by":"publisher","first-page":"2295","DOI":"10.1109\/jproc.2017.2761740","article-title":"Efficient processing of deep neural networks: a tutorial and survey","volume":"105","author":"Sze","year":"2017","journal-title":"Proc. IEEE"},{"key":"nceac9b86bib5","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1038\/s41586-019-1677-2","article-title":"Towards spike-based machine intelligence with neuromorphic computing","volume":"575","author":"Roy","year":"2019","journal-title":"Nature"},{"key":"nceac9b86bib6","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neunet.2019.09.005","article-title":"Rethinking the performance comparison between SNNs and ANNs","volume":"121","author":"Deng","year":"2020","journal-title":"Neural Netw."},{"key":"nceac9b86bib7","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1109\/jproc.2014.2304638","article-title":"The spinnaker project","volume":"102","author":"Furber","year":"2014","journal-title":"Proc. IEEE"},{"key":"nceac9b86bib8","doi-asserted-by":"publisher","first-page":"1537","DOI":"10.1109\/tcad.2015.2474396","article-title":"Truenorth: design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip","volume":"34","author":"Akopyan","year":"2015","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"nceac9b86bib9","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/mm.2018.112130359","article-title":"LOIHI: a neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"nceac9b86bib10","doi-asserted-by":"publisher","first-page":"95","DOI":"10.3389\/fnins.2019.00095","article-title":"Going deeper in spiking neural networks: VGG and residual architectures","volume":"13","author":"Sengupta","year":"2019","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib11","first-page":"13558","article-title":"RMP-SNN: residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network","author":"Han","year":"2020"},{"key":"nceac9b86bib12","first-page":"1","article-title":"Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing","author":"Diehl","year":"2015"},{"key":"nceac9b86bib13","doi-asserted-by":"publisher","first-page":"682","DOI":"10.3389\/fnins.2017.00682","article-title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","volume":"11","author":"Rueckauer","year":"2017","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib14","doi-asserted-by":"publisher","first-page":"58443","DOI":"10.1109\/access.2020.2983149","article-title":"A survey of autonomous driving: common practices and emerging technologies","volume":"8","author":"Yurtsever","year":"2020","journal-title":"IEEE Access"},{"key":"nceac9b86bib15","article-title":"Speeding up semantic segmentation for autonomous driving","volume":"vol 2","author":"Treml","year":"2016"},{"key":"nceac9b86bib16","first-page":"1752","article-title":"CNN-based semantic segmentation using level set loss","author":"Kim","year":"2019"},{"key":"nceac9b86bib17","first-page":"7151","article-title":"Context encoding for semantic segmentation","author":"Zhang","year":"2018"},{"key":"nceac9b86bib18","first-page":"801","article-title":"Encoder-decoder with ATROUS separable convolution for semantic image segmentation","author":"Chen","year":"2018"},{"key":"nceac9b86bib19","first-page":"3431","article-title":"Fully convolutional networks for semantic segmentation","author":"Long","year":"2015"},{"key":"nceac9b86bib20","doi-asserted-by":"crossref","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128 \u00d7 128 120 db 15\u03bc s latency asynchronous temporal contrast vision sensor","volume":"43","author":"Patrick","year":"2008","journal-title":"IEEE J. Solid-State Circ."},{"key":"nceac9b86bib21","first-page":"2060","article-title":"A 128 \u00d7 128 120 db 30 mw asynchronous vision sensor that responds to relative intensity change","author":"Lichtsteiner","year":"2006"},{"key":"nceac9b86bib22","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.1109\/jproc.2014.2346153","article-title":"Retinomorphic event-based vision sensors: bioinspired cameras with spiking output","volume":"102","author":"Posch","year":"2014","journal-title":"Proc. IEEE"},{"key":"nceac9b86bib23","first-page":"2426","article-title":"Activity-driven, event-based vision sensors","author":"Delbr\u00fcck","year":"2010"},{"key":"nceac9b86bib24","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1109\/msp.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"nceac9b86bib25","doi-asserted-by":"publisher","first-page":"508","DOI":"10.3389\/fnins.2016.00508","article-title":"Training deep spiking neural networks using backpropagation","volume":"10","author":"Lee","year":"2016","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib26","doi-asserted-by":"publisher","first-page":"331","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib27","article-title":"Revisiting batch normalization for training low-latency deep spiking neural networks from scratch","author":"Kim","year":"2020"},{"key":"nceac9b86bib28","doi-asserted-by":"publisher","first-page":"653","DOI":"10.3389\/fnins.2020.00653","article-title":"Toward scalable, efficient, and accurate deep spiking neural networks with backward residual connections, stochastic softmax, and hybridization","volume":"14","author":"Panda","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib29","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1007\/s11263-014-0788-3","article-title":"Spiking deep convolutional neural networks for energy-efficient object recognition","volume":"113","author":"Cao","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"nceac9b86bib30","doi-asserted-by":"publisher","first-page":"99","DOI":"10.3389\/fncom.2015.00099","article-title":"Unsupervised learning of digit recognition using spike-timing-dependent plasticity","volume":"9","author":"Diehl","year":"2015","journal-title":"Front. Comput. Neurosci."},{"key":"nceac9b86bib31","first-page":"8529","article-title":"Temporal coding in spiking neural networks with alpha synaptic function","author":"Comsa","year":"2020"},{"key":"nceac9b86bib32","doi-asserted-by":"crossref","DOI":"10.1109\/TSP.2021.3121632","article-title":"Federated learning with spiking neural networks","author":"Venkatesha","year":"2021"},{"key":"nceac9b86bib33","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.neunet.2021.09.022","article-title":"Optimizing deeper spiking neural networks for dynamic vision sensing","volume":"144","author":"Kim","year":"2021","journal-title":"Neural Netw."},{"key":"nceac9b86bib34","article-title":"Privatesnn: fully privacy-preserving spiking neural networks","author":"Kim","year":"2021"},{"key":"nceac9b86bib35","doi-asserted-by":"publisher","first-page":"3227","DOI":"10.1109\/tnnls.2017.2726060","article-title":"Supervised learning based on temporal coding in spiking neural networks","volume":"29","author":"Mostafa","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learning Syst."},{"key":"nceac9b86bib36","doi-asserted-by":"crossref","DOI":"10.1109\/DAC18072.2020.9218689","article-title":"T2FSNN: deep spiking neural networks with time-to-first-spike coding","author":"Park","year":"2020"},{"key":"nceac9b86bib37","author":"Gerstner","year":"2002"},{"key":"nceac9b86bib38","doi-asserted-by":"publisher","first-page":"10464","DOI":"10.1523\/jneurosci.18-24-10464.1998","article-title":"Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type","volume":"18","author":"Bi","year":"1998","journal-title":"J. Neurosci."},{"key":"nceac9b86bib39","author":"Hebb","year":"2005"},{"key":"nceac9b86bib40","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1038\/361031a0","article-title":"A synaptic model of memory: long-term potentiation in the hippocampus","volume":"361","author":"Bliss","year":"1993","journal-title":"Nature"},{"key":"nceac9b86bib41","doi-asserted-by":"publisher","first-page":"665","DOI":"10.3389\/fnins.2018.00665","article-title":"On practical issues for stochastic STDP hardware with 1-bit synaptic weights","volume":"12","author":"Yousefzadeh","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib42","first-page":"1","article-title":"Implementing spike-timing-dependent plasticity on spinnaker neuromorphic hardware","author":"Jin","year":"2010"},{"key":"nceac9b86bib43","article-title":"Visual explanations from spiking neural networks using interspike intervals","author":"Kim","year":"2021"},{"key":"nceac9b86bib44","doi-asserted-by":"publisher","first-page":"855","DOI":"10.3389\/fnins.2019.00855","article-title":"A swarm optimization solver based on ferroelectric spiking neural networks","volume":"13","author":"Fang","year":"2019","journal-title":"Front. Neurosci."},{"key":"nceac9b86bib45","first-page":"1","article-title":"Neuromorphic nearest neighbor search using Intel\u2019s Pohoiki springs","author":"Frady","year":"2020"},{"key":"nceac9b86bib46","first-page":"11270","article-title":"Spiking-Yolo: spiking neural network for energy-efficient object detection","volume":"vol 34","author":"Kim","year":"2020"},{"key":"nceac9b86bib47","first-page":"234","article-title":"U-net: convolutional networks for biomedical image segmentation","author":"Ronneberger","year":"2015"},{"key":"nceac9b86bib48","first-page":"424","article-title":"3d u-net: learning dense volumetric segmentation from sparse annotation","author":"\u00c7i\u00e7ek","year":"2016"},{"key":"nceac9b86bib49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/0600000079","article-title":"Computer vision for autonomous vehicles: problems, datasets and state of the art","volume":"12","author":"Janai","year":"2020","journal-title":"FNT Comput. Graphics Vision"},{"key":"nceac9b86bib50","article-title":"Image segmentation using deep learning: a survey","author":"Minaee","year":"2020"},{"key":"nceac9b86bib51","first-page":"1520","article-title":"Learning deconvolution network for semantic segmentation","author":"Noh","year":"2015"},{"key":"nceac9b86bib52","first-page":"1925","article-title":"Refinenet: multi-path refinement networks for high-resolution semantic segmentation","author":"Lin","year":"2017"},{"key":"nceac9b86bib53","article-title":"TensorFlow: large-scale machine learning on heterogeneous systems","author":"Abadi","year":"2015"},{"key":"nceac9b86bib54","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s11063-010-9149-6","article-title":"Segmentation and edge detection based on spiking neural network model","volume":"32","author":"Meftah","year":"2010","journal-title":"Neural Process. Lett."},{"key":"nceac9b86bib55","first-page":"1043","article-title":"Image segmentation method based on spiking neural network with adaptive synaptic weights","author":"Zheng","year":"2019"},{"key":"nceac9b86bib56","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207075","article-title":"SpikeSEG: Spiking segmentation via STDP saliency mapping","author":"Kirkland","year":"2020"},{"key":"nceac9b86bib57","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2020.568319","article-title":"Perception Understanding Action: Adding Understanding to the Perception Action Cycle With Spiking Segmentation","volume":"14","author":"Kirkland","year":"2020","journal-title":"Front. Neurorobot."},{"key":"nceac9b86bib58","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1109\/jproc.2021.3067593","article-title":"Advancing neuromorphic computing with LOIHI: a survey of results and outlook","volume":"109","author":"Davies","year":"2021","journal-title":"Proc. IEEE"},{"key":"nceac9b86bib59","first-page":"317","article-title":"Parallel time batching: Systolic-array acceleration of sparse spiking neural computation","author":"Lee","year":"2022"},{"key":"nceac9b86bib60","first-page":"349","article-title":"Spinalflow: an architecture and dataflow tailored for spiking neural networks","author":"Narayanan","year":"2020"},{"key":"nceac9b86bib61","first-page":"1","article-title":"Resparc: a reconfigurable and energy-efficient architecture with memristive crossbars for deep spiking neural networks","author":"Ankit","year":"2017"},{"key":"nceac9b86bib62","first-page":"574","article-title":"A 67.5 \u03bcj\/prediction accelerator for spiking neural networks in image segmentation","volume":"69","author":"Chen","year":"2021","journal-title":"IEEE Trans. Circ. Syst. II"},{"key":"nceac9b86bib63","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"DEEPLAB: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS","volume":"40","author":"Chen","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Machine Intelligence"},{"key":"nceac9b86bib64","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal visual object classes (VOC) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"nceac9b86bib65","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/j.neucom.2018.05.087","article-title":"Deep neural networks with weighted spikes","volume":"311","author":"Kim","year":"2018","journal-title":"Neurocomputing"},{"key":"nceac9b86bib66","first-page":"991","article-title":"Semantic contours from inverse detectors","author":"Hariharan","year":"2011"},{"key":"nceac9b86bib67","article-title":"DDD17: end-to-end Davis driving dataset","author":"Binas","year":"2017"},{"key":"nceac9b86bib68","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW.2019.00205","article-title":"EV-SEGNET: semantic segmentation for event-based cameras","author":"Alonso","year":"2019"},{"key":"nceac9b86bib69","article-title":"Mnist-cifar10-segmentation"},{"key":"nceac9b86bib70","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","author":"Ioffe","year":"2015"},{"key":"nceac9b86bib71","article-title":"DIET-SNN: direct input encoding with leakage and threshold optimization in deep spiking neural networks","author":"Rathi","year":"2020"},{"key":"nceac9b86bib72","first-page":"10","article-title":"1.1 computing\u2019s energy problem (and what we can do about it)","author":"Horowitz","year":"2014"},{"key":"nceac9b86bib73","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-021-91786-z","article-title":"Event-based backpropagation can compute exact gradients for spiking neural networks","volume":"11","author":"Wunderlich","year":"2021","journal-title":"Sci. Rep."},{"key":"nceac9b86bib74","first-page":"21","article-title":"Semantic segmentation of uav aerial videos using convolutional neural networks","author":"Girisha","year":"2019"},{"key":"nceac9b86bib75","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.isprsjprs.2020.05.009","article-title":"UAVID: a semantic segmentation dataset for UAV imagery","volume":"165","author":"Lyu","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T16:03:43Z","timestamp":1670429023000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac9b86"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,1]]},"references-count":75,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,12,8]]},"published-print":{"date-parts":[[2022,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac9b86","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,1]]},"assertion":[{"value":"Beyond classification: directly training spiking neural networks for semantic segmentation","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-01-30","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-10-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-12-08","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}