{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T02:10:31Z","timestamp":1771467031001,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,8]],"date-time":"2021-09-08T00:00:00Z","timestamp":1631059200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Congshi","award":["CQYC201905015"],"award-info":[{"award-number":["CQYC201905015"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Neuromorphic hardware systems have been gaining ever-increasing focus in many embedded applications as they use a brain-inspired, energy-efficient spiking neural network (SNN) model that closely mimics the human cortex mechanism by communicating and processing sensory information via spatiotemporally sparse spikes. In this paper, we fully leverage the characteristics of spiking convolution neural network (SCNN), and propose a scalable, cost-efficient, and high-speed VLSI architecture to accelerate deep SCNN inference for real-time low-cost embedded scenarios. We leverage the snapshot of binary spike maps at each time-step, to decompose the SCNN operations into a series of regular and simple time-step CNN-like processing to reduce hardware resource consumption. Moreover, our hardware architecture achieves high throughput by employing a pixel stream processing mechanism and fine-grained data pipelines. Our Zynq-7045 FPGA prototype reached a high processing speed of 1250 frames\/s and high recognition accuracies on the MNIST and Fashion-MNIST image datasets, demonstrating the plausibility of our SCNN hardware architecture for many embedded applications.<\/jats:p>","DOI":"10.3390\/s21186006","type":"journal-article","created":{"date-parts":[[2021,9,8]],"date-time":"2021-09-08T21:28:45Z","timestamp":1631136525000},"page":"6006","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps"],"prefix":"10.3390","volume":"21","author":[{"given":"Ling","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"}]},{"given":"Jing","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"}]},{"given":"Cong","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"},{"name":"Key Laboratory of Dependable Service Computing in Cyber Physical Society, Ministry of Education, Chongqing University, Chongqing 400044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7478-3103","authenticated-orcid":false,"given":"Yingcheng","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"}]},{"given":"Wei","family":"He","sequence":"additional","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"}]},{"given":"Xichuan","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China"},{"name":"Key Laboratory of Dependable Service Computing in Cyber Physical Society, Ministry of Education, Chongqing University, Chongqing 400044, China"}]},{"given":"Xu","family":"Yang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Liyuan","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Nanjian","family":"Wu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.1109\/JPROC.2015.2444094","article-title":"Memory and Information Processing in Neuromorphic Systems","volume":"103","author":"Indiveri","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Seo, J.S., Brezzo, B., Liu, Y., Parker, B.D., Esser, S.K., Montoye, R.K., Rajendran, B., Tierno, J.A., Chang, L., and Modha, D.S. (2011, January 19\u201321). A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons. Proceedings of the 2011 IEEE Custom Integrated Circuits Conference (CICC), San Jose, CA, USA.","DOI":"10.1109\/CICC.2011.6055293"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1943","DOI":"10.1109\/JSSC.2013.2259038","article-title":"SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation","volume":"48","author":"Painkras","year":"2013","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1537","DOI":"10.1109\/TCAD.2015.2474396","article-title":"TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip","volume":"34","author":"Akopyan","year":"2015","journal-title":"IEEE Trans. Comput. Des. Integr. Circuits Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1109\/JPROC.2014.2313565","article-title":"Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations","volume":"102","author":"Benjamin","year":"2014","journal-title":"Proc. IEEE"},{"key":"ref_6","first-page":"141","article-title":"A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128 K synapses","volume":"9","author":"Qiao","year":"2020","journal-title":"Front. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MM.2018.112130359","article-title":"Loihi: A Neuromorphic Manycore Processor with On-Chip Learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"ref_8","first-page":"145","article-title":"A 0.086-mm2 12.7-pJ\/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS","volume":"13","author":"Frenkel","year":"2019","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"992","DOI":"10.1109\/JSSC.2018.2884901","article-title":"A 4096-Neuron 1M-Synapse 3.8-pJ\/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS","volume":"54","author":"Chen","year":"2018","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.sysarc.2017.01.003","article-title":"Darwin: A neuromorphic hardware co-processor based on spiking neural networks","volume":"77","author":"Ma","year":"2017","journal-title":"J. Syst. Arch."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1109\/TBCAS.2019.2928793","article-title":"MorphIC: A 65-nm 738k-Synapse\/mm2 Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning","volume":"13","author":"Frenkel","year":"2019","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1109\/TCSI.2021.3052885","article-title":"A Fast and Energy-Efficient SNN Processor with Adaptive Clock\/Event-Driven Computation Scheme and Online Learning","volume":"68","author":"Li","year":"2021","journal-title":"IEEE Trans. Circuits Syst."},{"key":"ref_13","first-page":"2655","article-title":"A 64K-Neuron 64M-1b-Synapse 2.64pJ\/SOP Neuromorphic Chip With All Memory on Chip for Spike-Based Models in 65 nm CMOS","volume":"68","author":"Kuang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"583","DOI":"10.3389\/fnins.2018.00583","article-title":"Neural and synaptic array transceiver: A brain-inspired computing framework for embedded learning","volume":"12","author":"Detorakis","year":"2018","journal-title":"Front. Neurosci."},{"key":"ref_15","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_16","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the IEEE conference on computer vision and pattern recognition 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1007\/s11263-014-0788-3","article-title":"Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition","volume":"113","author":"Cao","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1109\/TNNLS.2014.2362542","article-title":"Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network","volume":"26","author":"Zhao","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.neunet.2017.12.005","article-title":"STDP-based spiking deep convolutional neural networks for object recognition","volume":"99","author":"Kheradpisheh","year":"2018","journal-title":"Neural Netw."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1109\/TCDS.2018.2833071","article-title":"Deep Spiking Convolutional Neural Network Trained With Unsupervised Spike-Timing-Dependent Plasticity","volume":"11","author":"Lee","year":"2019","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"189","DOI":"10.3389\/fnins.2019.00189","article-title":"ReStoCNet: Residual Stochastic Binary Convolutional Spiking Neural Network for Memory-Efficient Neuromorphic Computing","volume":"13","author":"Srinivasan","year":"2019","journal-title":"Front. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/j.neunet.2019.08.034","article-title":"Deep CovDenseSNN: A hierarchical event-driven dynamic framework with spiking neurons in noisy environment","volume":"121","author":"Xu","year":"2019","journal-title":"Neural Netw."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"122402","DOI":"10.1007\/s11432-019-1468-0","article-title":"Deterministic conversion rule for CNNs to efficient spiking convolutional neural networks","volume":"63","author":"Yang","year":"2020","journal-title":"Sci. China Inf. Sci."},{"key":"ref_25","first-page":"504","article-title":"An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors","volume":"47","author":"Gotarredona","year":"2011","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"63","DOI":"10.3389\/fnins.2018.00063","article-title":"A configurable event-driven convolutional node with rate saturation mechanism for modular ConvNet systems implementation","volume":"12","year":"2018","journal-title":"Front. Neurosci."},{"key":"ref_27","first-page":"159","article-title":"Neuromorphic LIF row-by-row multiconvolution processor for FPGA","volume":"13","year":"2018","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Frenkel, C., Legat, J.D., and Bol, D. (2020, January 12\u201314). A 28-nm convolutional neuromorphic processor enabling online learning with spike-based retinas. Proceedings of the 2020 IEEE International Symposium on Circuits and Systems (ISCAS), Seville, Spain.","DOI":"10.1109\/ISCAS45731.2020.9180440"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kang, Z., Wang, L., Guo, S., Gong, R., Deng, Y., and Dou, Q. (2019, January 12\u201315). ASIE: An Asynchronous SNN Inference Engine for AER Events Processing. Proceedings of the 2019 25th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC), Hirosaki, Japan.","DOI":"10.1109\/ASYNC.2019.00015"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1007\/s11390-020-9686-z","article-title":"SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array","volume":"35","author":"Wang","year":"2020","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2651","DOI":"10.1109\/TCSI.2019.2899356","article-title":"CORDIC-SNN: On-FPGA STDP Learning with Izhikevich Neurons","volume":"66","author":"Heidarpur","year":"2019","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"682","DOI":"10.3389\/fnins.2017.00682","article-title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","volume":"11","author":"Rueckauer","year":"2017","journal-title":"Front. Neurosci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1162\/neco_a_01245","article-title":"An FPGA implementation of deep spiking neural networks for low-power and fast classification","volume":"32","author":"Ju","year":"2020","journal-title":"Neural Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"99","DOI":"10.3389\/fncom.2015.00099","article-title":"Unsupervised learning of digit recognition using spike-timing-dependent plasticity","volume":"9","author":"Diehl","year":"2015","journal-title":"Front. Comput. Neurosci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.neucom.2020.10.100","article-title":"CompSNN: A lightweight spiking neural network based on spatiotemporally compressive spike features","volume":"425","author":"Wang","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_36","first-page":"1581","article-title":"DeepTempo: A Hardware-Friendly Direct Feedback Alignment Multi-Layer Tempotron Learning Rule for Deep Spiking Neural Networks","volume":"68","author":"Shi","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_37","unstructured":"Lazzaro, J., and Wawrzynek, J. (1995, January 27\u201329). A multi-sender asynchronous extension to the AER protocol. Proceedings of the Sixteenth Conference on Advanced Research in VLSI 1995, Chapel Hill, NC, USA."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_39","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1109\/TNNLS.2013.2245677","article-title":"Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons","volume":"24","author":"Yu","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Diehl, P.U., Neil, D., Binas, J., Cook, M., Liu, S.C., and Pfeiffer, M. (2015, January 12\u201317). Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.","DOI":"10.1109\/IJCNN.2015.7280696"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2621","DOI":"10.1109\/TVLSI.2013.2294916","article-title":"Minitaur, an event-driven FPGA-based spiking network accelerator","volume":"22","author":"Neil","year":"2014","journal-title":"IEEE Trans. Very Large Scale Integr. (VLSI) Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6006\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:58:37Z","timestamp":1760165917000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6006"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,8]]},"references-count":42,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186006"],"URL":"https:\/\/doi.org\/10.3390\/s21186006","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,8]]}}}