{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T17:11:42Z","timestamp":1774631502747,"version":"3.50.1"},"reference-count":45,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Ultra-high-speed object detection and tracking are crucial in fields such as fault detection and scientific observation. Existing solutions to this task have deficiencies in processing speeds. To deal with this difficulty, we propose a neural-inspired ultra-high-speed moving object filtering, detection, and tracking scheme, as well as a corresponding accelerator based on a high-speed spike camera. We parallelize the filtering module and divide the detection module to accelerate the algorithm and balance latency among modules for the benefit of the task-level pipeline. To be specific, a block-based parallel computation model is proposed to accelerate the filtering module, and the detection module is accelerated by a parallel connected component labeling algorithm modeling spike sparsity and spatial connectivity of moving objects with a searching tree. The hardware optimizations include processing the LIF layer with a group of multiplexers to reduce ADD operations and replacing expensive exponential operations with multiplications of preprocessed fixed-point values to increase processing speed and minimize resource consumption. We design an accelerator with the above techniques, achieving 19 times acceleration over the serial version after 25-way parallelization. A processing system for the accelerator is also implemented on the Xilinx ZCU-102 board to validate its functionality and performance. Our accelerator can process more than 20,000 spike images with 250 \u00d7 400 resolution per second with 1.618 W dynamic power consumption.<\/jats:p>","DOI":"10.1162\/neco_a_01507","type":"journal-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T23:46:09Z","timestamp":1657237569000},"page":"1812-1839","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":10,"title":["An FPGA Accelerator for High-Speed Moving Objects Detection and Tracking With a Spike Camera"],"prefix":"10.1162","volume":"34","author":[{"given":"Yaoyu","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, China yaoyu.zhu@pku.edu.cn"}]},{"given":"Yu","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, China"},{"name":"Beijing Academy of Artificial Intelligence, Beijing 10084, China yuzhang2.liu@gmail.com"}]},{"given":"Xiaodong","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, China donxie@pku.edu.cn"}]},{"given":"Tiejun","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, China tjhuang@pku.edu.cn"}]}],"member":"281","published-online":{"date-parts":[[2022,7,14]]},"reference":[{"issue":"7010","key":"2022071522332893500_B1","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1038\/nature03010","article-title":"Synaptic computation","volume":"431","author":"Abbott","year":"2004","journal-title":"Nature"},{"issue":"5297","key":"2022071522332893500_B2","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1126\/science.275.5297.221","article-title":"Synaptic depression and cortical gain control","author":"Abbott","year":"1997","journal-title":"Science"},{"issue":"3","key":"2022071522332893500_B3","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1162\/NECO_a_00703","article-title":"What can neuromorphic event-driven precise timing add to spike-based pattern recognition?","volume":"27","author":"Akolkar","year":"2015","journal-title":"Neural Computation"},{"key":"2022071522332893500_B4","first-page":"483","article-title":"Validity of the single processor approach to achieving large scale computing capabilities","volume-title":"Proceedings of the 1967 Spring Joint Computer Conference","author":"Amdahl","year":"1967"},{"issue":"11","key":"2022071522332893500_B5","doi-asserted-by":"publisher","first-page":"1282","DOI":"10.1016\/j.cviu.2010.03.021","article-title":"Accelerated hardware video object segmentation: From foreground detection to connected components labelling","volume":"114","author":"Appiah","year":"2010","journal-title":"Computer Vision and Image Understanding"},{"issue":"1","key":"2022071522332893500_B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2008\/246309","article-title":"Evaluating multiple object tracking performance: The CLEAR MOT metrics","volume":"2008","author":"Bernardin","year":"2008","journal-title":"EURASIP Journal on Image and Video Processing"},{"key":"2022071522332893500_B7","first-page":"214","article-title":"Computer vision face tracking for use in a perceptual user interface","volume-title":"Proceedings of the IEEE Workshop on Applications of Computer Vision","author":"Bradski","year":"1998"},{"issue":"10","key":"2022071522332893500_B8","doi-asserted-by":"publisher","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","article-title":"A 240 \u00d7 180 130 dB 3 \u03bcs latency global shutter spatiotemporal vision sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE Journal of Solid-State Circuits"},{"key":"2022071522332893500_B9","doi-asserted-by":"publisher","DOI":"10.3389\/fncom.2013.00075","article-title":"Probabilistic inference of short-term synaptic plasticity in neocortical microcircuits","volume":"7","author":"Costa","year":"2013","journal-title":"Frontiers in Computational Neuroscience"},{"key":"2022071522332893500_B10","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1109\/DCC.2017.69","article-title":"Spike camera and its coding methods","volume-title":"Proceedings of the 2017 Data Compression Conference","author":"Dong","year":"2017"},{"issue":"7","key":"2022071522332893500_B11","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1016\/S0166-2236(00)01835-X","article-title":"Short-term synaptic plasticity as a temporal filter","volume":"24","author":"Fortune","year":"2001","journal-title":"Trends in Neurosciences"},{"issue":"1","key":"2022071522332893500_B12","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1109\/TASE.2013.2292583","article-title":"Simultaneous vision-based shape and motion analysis of cells fast-flowing in a microchannel","volume":"12","author":"Gu","year":"2015","journal-title":"IEEE Transactions on Automation Science and Engineering"},{"issue":"4","key":"2022071522332893500_B13","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1007\/s11633-016-1024-0","article-title":"Review of some advances and applications in real-time high-speed vision: Our views and experiences","volume":"13","author":"Gu","year":"2016","journal-title":"International Journal of Automation and Computing"},{"issue":"4","key":"2022071522332893500_B14","doi-asserted-by":"publisher","first-page":"1346","DOI":"10.1109\/TASE.2015.2462118","article-title":"LOC-based high-throughput cell morphology analysis system","volume":"12","author":"Gu","year":"2015","journal-title":"IEEE Transactions on Automation Science and Engineering"},{"issue":"1","key":"2022071522332893500_B15","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/TCSVT.2012.2202195","article-title":"Fast FPGA-based multiobject feature extraction","volume":"23","author":"Gu","year":"2013","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"2022071522332893500_B16","first-page":"1","article-title":"Live demonstration: A 768 \u00d7 640 pixels 200 Meps dynamic vision sensor","volume-title":"Proceedings of the 2017 IEEE International Symposium on Circuits and Systems","author":"Guo","year":"2017"},{"key":"2022071522332893500_B17","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1109\/CVPR42600.2020.00180","article-title":"Neuromorphic camera guided high dynamic range imaging","volume-title":"Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Han","year":"2020"},{"issue":"9","key":"2022071522332893500_B18","doi-asserted-by":"publisher","first-page":"1977","DOI":"10.1016\/j.patcog.2008.10.013","article-title":"Fast connected-component labeling","volume":"42","author":"He","year":"2009","journal-title":"Pattern Recognition"},{"key":"2022071522332893500_B19","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.patcog.2017.04.018","article-title":"The connected-component labeling problem: A review of state-of-the-art algorithms","volume":"70","author":"He","year":"2017","journal-title":"Pattern Recognition"},{"key":"2022071522332893500_B20","article-title":"A detection and tracking method, device and readable storage medium","author":"Huang","year":"2020"},{"issue":"1","key":"2022071522332893500_B21","doi-asserted-by":"publisher","DOI":"10.1117\/1.JEI.21.1.013010","article-title":"Color-histogram-based tracking at 2000 FPS","volume":"21","author":"Ishii","year":"2012","journal-title":"Journal of Electronic Imaging"},{"issue":"1","key":"2022071522332893500_B22","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"Journal of Basic Engineering"},{"issue":"7","key":"2022071522332893500_B23","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.0040207","article-title":"Excitatory and feed-forward inhibitory hippocampal synapses work synergistically as an adaptive filter of natural spike trains","volume":"4","author":"Klyachko","year":"2006","journal-title":"PLOS Biol."},{"issue":"10","key":"2022071522332893500_B24","doi-asserted-by":"publisher","first-page":"3818","DOI":"10.1109\/JSEN.2019.2895294","article-title":"A hardware-oriented algorithm for ultra-high-speed object detection","volume":"19","author":"Li","year":"2019","journal-title":"IEEE Sensors Journal"},{"issue":"5","key":"2022071522332893500_B25","doi-asserted-by":"publisher","first-page":"1703","DOI":"10.1007\/s11554-019-00931-5","article-title":"A novel hardware-oriented ultra-high-speed object detection algorithm based on convolutional neural network","volume":"17","author":"Li","year":"2020","journal-title":"Journal of Real-Time Image Processing"},{"issue":"5","key":"2022071522332893500_B26","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1109\/TIP.2008.919369","article-title":"A run-based two-scan labeling algorithm","volume":"17","author":"Lifeng","year":"2008","journal-title":"IEEE Transactions on Image Processing"},{"key":"2022071522332893500_B27","doi-asserted-by":"publisher","first-page":"134926","DOI":"10.1109\/ACCESS.2019.2941282","article-title":"Low latency event-based filtering and feature extraction for dynamic vision sensors in real-time FPGA applications","volume":"7","author":"Linares-Barranco","year":"2019","journal-title":"IEEE Access"},{"issue":"17","key":"2022071522332893500_B28","doi-asserted-by":"crossref","DOI":"10.3390\/s19173707","article-title":"An FPGA-based ultra-high- speed object detection algorithm with multi-frame information fusion","volume":"91","author":"Long","year":"2019","journal-title":"Sensors"},{"issue":"9","key":"2022071522332893500_B29","doi-asserted-by":"publisher","first-page":"2384","DOI":"10.1162\/neco_a_01112","article-title":"Spiking elementary motion detector in neuromorphic systems","volume":"30","author":"Milde","year":"2018","journal-title":"Neural Computation"},{"issue":"1","key":"2022071522332893500_B30","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1137\/0105003","article-title":"Algorithms for the assignment and transportation problems","volume":"5","author":"Munkres","year":"1957","journal-title":"Journal of the Society for Industrial and Applied Mathematics"},{"issue":"4","key":"2022071522332893500_B31","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1162\/NECO_a_00720","article-title":"Visual tracking using neuromorphic asynchronous event-based cameras","volume":"27","author":"Ni","year":"2015","journal-title":"Neural Computation"},{"issue":"6","key":"2022071522332893500_B32","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1109\/4.678644","article-title":"Space-variant nonorthogonal structure CMOS image sensor design","volume":"33","author":"Pardo","year":"1998","journal-title":"IEEE Journal of Solid-State Circuits"},{"key":"2022071522332893500_B33","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128 \u00d7 128 120 dB 15 \u03bcs latency asynchronous temporal contrast vision sensor","volume":"43","author":"Patrick","year":"2008","journal-title":"IEEE Journal of Solid-State Circuits"},{"issue":"1","key":"2022071522332893500_B34","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1109\/JSSC.2010.2085952","article-title":"A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS","volume":"46","author":"Posch","year":"2010","journal-title":"IEEE Journal of Solid-State Circuits"},{"issue":"4","key":"2022071522332893500_B35","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1109\/34.3918","article-title":"Efficient component labeling of images of arbitrary dimension represented by linear bintrees","volume":"10","author":"Samet","year":"1988","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"3","key":"2022071522332893500_B36","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1109\/JSSC.2012.2230553","article-title":"A 128 \u00d7 128 1.5% contrast sensitivity 0.9% FPN 3 s latency 4 mW asynchronous frame-free dynamic visionsensor using transimpedance preamplifiers","volume":"48","author":"Serrano-Gotarredona","year":"2013","journal-title":"IEEE Journal of Solid-State Circuits"},{"key":"2022071522332893500_B37","first-page":"544","article-title":"Super high-speed vision platform for processing 1024 \u00d7 1024 images in real time at 12500 FPS","volume-title":"Proceedings of the 2016 IEEE\/SICE International Symposium on System Integration","author":"Sharma","year":"2016"},{"issue":"4","key":"2022071522332893500_B38","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1162\/089976698300017502","article-title":"Neural networks with dynamic synapses","volume":"10","author":"Tsodyks","year":"1998","journal-title":"Neural Computation"},{"issue":"1","key":"2022071522332893500_B39","doi-asserted-by":"publisher","DOI":"10.1186\/1687-5281-2013-21","article-title":"Real-time single-pass connected components analysis algorithm","volume":"2013","author":"Zhao","year":"2013","journal-title":"EURASIP Journal on Image and Video Processing"},{"key":"2022071522332893500_B40","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/CVPR46437.2021.01182","article-title":"Spk2ImgNet: Learning to reconstruct dynamic scene from continuous spike stream","volume-title":"Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhao","year":"2021"},{"key":"2022071522332893500_B41","first-page":"371","article-title":"Motion estimation for spike camera data sequence via spike interval analysis","volume-title":"Proceedings of the 2020 IEEE International Conference on Visual Communications and Image Processing","author":"Zhao","year":"2020"},{"key":"2022071522332893500_B42","first-page":"10","article-title":"High-speed image reconstruction through short-term plasticity for spiking cameras","volume-title":"Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zheng","year":"2021"},{"key":"2022071522332893500_B43","author":"Zhu","year":"2019","journal-title":"A retina-inspired sampling method for visual texture reconstruction."},{"key":"2022071522332893500_B44","doi-asserted-by":"crossref","first-page":"1435","DOI":"10.1109\/CVPR42600.2020.00151","article-title":"Retina-like visual image reconstruction via spiking neural model","author":"Zhu","year":"2020","journal-title":"Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"issue":"1","key":"2022071522332893500_B45","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1146\/annurev.physiol.64.092501.114547","article-title":"Short-term synaptic plasticity","volume":"64","author":"Zucker","year":"2002","journal-title":"Annual Review of Physiology"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/34\/8\/1812\/2034912\/neco_a_01507.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/34\/8\/1812\/2034912\/neco_a_01507.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T22:33:55Z","timestamp":1657924435000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/34\/8\/1812\/111782\/An-FPGA-Accelerator-for-High-Speed-Moving-Objects"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,14]]},"references-count":45,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,7,14]]},"published-print":{"date-parts":[[2022,7,14]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01507","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,8]]},"published":{"date-parts":[[2022,7,14]]}}}