{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,29]],"date-time":"2025-11-29T07:56:42Z","timestamp":1764403002102,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,6,16]],"date-time":"2020-06-16T00:00:00Z","timestamp":1592265600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["TEC2016-77785-P"],"award-info":[{"award-number":["TEC2016-77785-P"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1    \u03bc   s to 6.7    \u03bc   s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.<\/jats:p>","DOI":"10.3390\/s20123404","type":"journal-article","created":{"date-parts":[[2020,6,16]],"date-time":"2020-06-16T13:20:43Z","timestamp":1592313643000},"page":"3404","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7268-2915","authenticated-orcid":false,"given":"Ricardo","family":"Tapiador-Morales","sequence":"first","affiliation":[{"name":"Robotics and Technology of Computers Lab (ETSII-EPS), University of Seville, 41089 Sevilla, Spain"},{"name":"aiCTX AG, 8092 Zurich, Switzerland"}]},{"given":"Jean-Matthieu","family":"Maro","sequence":"additional","affiliation":[{"name":"Neuromorphic Vision and Natural Computation, Sorbonne Universit\u00e9, 75006 Paris, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3061-5922","authenticated-orcid":false,"given":"Angel","family":"Jimenez-Fernandez","sequence":"additional","affiliation":[{"name":"Robotics and Technology of Computers Lab (ETSII-EPS), University of Seville, 41089 Sevilla, Spain"},{"name":"SCORE Lab, Research Institute of Computer Engineering (I3US), University of Seville, 41089 Seville, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4512-6750","authenticated-orcid":false,"given":"Gabriel","family":"Jimenez-Moreno","sequence":"additional","affiliation":[{"name":"Robotics and Technology of Computers Lab (ETSII-EPS), University of Seville, 41089 Sevilla, Spain"},{"name":"SCORE Lab, Research Institute of Computer Engineering (I3US), University of Seville, 41089 Seville, Spain"}]},{"given":"Ryad","family":"Benosman","sequence":"additional","affiliation":[{"name":"Neuromorphic Vision and Natural Computation, Sorbonne Universit\u00e9, 75006 Paris, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6056-740X","authenticated-orcid":false,"given":"Alejandro","family":"Linares-Barranco","sequence":"additional","affiliation":[{"name":"Robotics and Technology of Computers Lab (ETSII-EPS), University of Seville, 41089 Sevilla, Spain"},{"name":"SCORE Lab, Research Institute of Computer Engineering (I3US), University of Seville, 41089 Seville, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,16]]},"reference":[{"key":"ref_1","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 2012 Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_2","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tallent, N.R., Gawande, N.A., Siegel, C., Vishnu, A., and Hoisie, A. (2018). Evaluating On-Node GPU Interconnects for Deep Learning Workloads, Springer.","DOI":"10.1007\/978-3-319-72971-8_1"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Saeed, A., Al-Hamadi, A., Niese, R., and Elzobi, M. (2014). Frame-Based Facial Expression Recognition Using Geometrical Features. Adv. Hum. Comput. Interact., 2014.","DOI":"10.1155\/2014\/408953"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zanchettin, C., Bezerra, B.L.D., and Azevedo, W.W. (2012, January 10\u201315). A KNN-SVM hybrid model for cursive handwriting recognition. Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia.","DOI":"10.1109\/IJCNN.2012.6252719"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"32","DOI":"10.3389\/fnins.2012.00032","article-title":"Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel ConvNets for visual processing","volume":"6","author":"Farabet","year":"2012","journal-title":"Front. Neurosci."},{"key":"ref_8","unstructured":"Mead, C. (1989). Analog VLSI and Neutral Systems, Addison-Wesley."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sterling, P., and Laughlin, S. (2015). Principles of Neural Design, MIT Press.","DOI":"10.7551\/mitpress\/9780262028707.001.0001"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1109\/JSSC.2016.2604285","article-title":"A 0.5 V 55 \u03bcW 64 \u00d7 2 Channel Binaural Silicon Cochlea for Event-Driven Stereo-Audio Sensing","volume":"51","author":"Yang","year":"2016","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1109\/TNNLS.2016.2583223","article-title":"A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach","volume":"28","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128 \u00d7 128 120 dB 15 us Latency Asynchronous Temporal Contrast Vision Sensor","volume":"43","author":"Lichtsteiner","year":"2008","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/TVLSI.2007.893624","article-title":"Arbitrated Time-to-First Spike CMOS Image Sensor With On-Chip Histogram Equalization","volume":"15","author":"Shoushun","year":"2007","journal-title":"IEEE Trans. Very Large Scale Integr. VLSI Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1109\/JSSC.2010.2085952","article-title":"A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS","volume":"46","author":"Posch","year":"2011","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1109\/JSSC.2011.2118490","article-title":"A 3.6 \u03bc s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor","volume":"46","year":"2011","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","article-title":"A 240 \u00d7 180 130 dB 3 \u03bcs Latency Global Shutter Spatiotemporal Vision Sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1109\/JSSC.2014.2386899","article-title":"Selective Change Driven Vision Sensor With Continuous-Time Logarithmic Photoreceptor and Winner-Take-All Circuit for Pixel Selection","volume":"50","author":"Pardo","year":"2015","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Son, B., Suh, Y., Kim, S., Jung, H., Kim, J., Shin, C., Park, K., Lee, K., Park, J., and Woo, J. (2017, January 5\u20139). 4.1 A 640\u00d7480 dynamic vision sensor with a 9 \u03bcm pixel and 300 Meps address-event representation. Proceedings of the 2017 IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, CA, USA.","DOI":"10.1109\/ISSCC.2017.7870263"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Linares-Barranco, A., G\u00f3mez-Rodr\u00edguez, F., Villanueva, V., Longinotti, L., and Delbr\u00fcck, T. (2015, January 24\u201327). A USB3.0 FPGA event-based filtering and tracking framework for dynamic vision sensors. Proceedings of the 2015 IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal.","DOI":"10.1109\/ISCAS.2015.7169172"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"223","DOI":"10.3389\/fnins.2013.00223","article-title":"Robotic goalie with 3 ms reaction time at 4event-based dynamic vision sensor","volume":"7","author":"Delbruck","year":"2013","journal-title":"Front. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"134926","DOI":"10.1109\/ACCESS.2019.2941282","article-title":"Low Latency Event-Based Filtering and Feature Extraction for Dynamic Vision Sensors in Real-Time FPGA Applications","volume":"7","author":"Moeys","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Linares-Barranco, A., Liu, H., Rios-Navarro, A., Gomez-Rodriguez, F., Moeys, D.P., and Delbruck, T. (2018). Approaching Retinal Ganglion Cell Modeling and FPGA Implementation for Robotics. Entropy, 20.","DOI":"10.3390\/e20060475"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1109\/TNNLS.2014.2362542","article-title":"Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network","volume":"26","author":"Zhao","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_24","first-page":"159","article-title":"Neuromorphic LIF Row-by-Row Multiconvolution Processor for FPGA","volume":"13","year":"2019","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2706","DOI":"10.1109\/TPAMI.2013.71","article-title":"Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing\u2014Application to feedforward convnets","volume":"35","author":"Zhao","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","first-page":"481","article-title":"Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details","volume":"9","year":"2015","journal-title":"Front. Neurosci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"437","DOI":"10.3389\/fnins.2015.00437","article-title":"Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades","volume":"9","author":"Orchard","year":"2015","journal-title":"Front. Neurosci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1109\/TPAMI.2016.2574707","article-title":"Hots: A hierarchy of event-based time-surfaces for pattern recognition","volume":"39","author":"Lagorce","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2454","DOI":"10.1109\/TC.2012.142","article-title":"Overview of the spinnaker system architecture","volume":"62","author":"Furber","year":"2013","journal-title":"IEEE Trans. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Schmitt, S., Kl\u00e4hn, J., Bellec, G., Gr\u00fcbl, A., G\u00fcttler, M., Hartel, A., Hartmann, S., de Oliveira, D.H., Husmann, K., and Jeltsch, S. (2017, January 14\u201319). Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7966125"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1537","DOI":"10.1109\/TCAD.2015.2474396","article-title":"TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip","volume":"34","author":"Akopyan","year":"2015","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circuits Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/TBCAS.2017.2759700","article-title":"A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)","volume":"12","author":"Moradi","year":"2018","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/MC.2018.157113521","article-title":"Programming Spiking Neural Networks on Intel\u2019s Loihi","volume":"51","author":"Lin","year":"2018","journal-title":"Computer"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"051001","DOI":"10.1088\/1741-2560\/13\/5\/051001","article-title":"Large-scale neuromorphic computing systems","volume":"13","author":"Furber","year":"2016","journal-title":"J. Neural Eng."},{"key":"ref_35","unstructured":"Delbr\u00fcck, T. (2020, June 14). jAER Open Source Project (2007). Available online: https:\/\/github.com\/SensorsINI\/jaer."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"275","DOI":"10.3389\/fnins.2020.00275","article-title":"Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities","volume":"14","author":"Maro","year":"2020","journal-title":"Front. Neurosci."},{"key":"ref_37","unstructured":"Piromsopa, K., Arporntewan, C., and Chongstitvatana, P. (2001, January 14\u201316). An FPGA Implementation of a Fixed-Point Square Root Operation. Proceedings of the International Symposium on Communications and Information Technology, (ISCIT 2001), Chiang Mai, Thailand."},{"key":"ref_38","unstructured":"Li, Y., and Chu, W. (1996, January 7\u20139). A new non-restoring square root algorithm and its VLSI implementations. Proceedings of the International Conference on Computer Design, Austin, TX, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1109\/TNNLS.2018.2852335","article-title":"NullHop:A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps","volume":"30","author":"Aimar","year":"2018","journal-title":"Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Berner, R., Delbr\u00fcck, T., Civit-Balcells, A., and Linares-Barranco, A. (2007, January 27\u201330). A 5 Meps $100 USB2.0 address-event monitor-sequencer interface. Proceedings of the 2007 IEEE International Symposium on Circuits and Systems, New Orleans, LA, USA.","DOI":"10.1109\/ISCAS.2007.378616"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/TBCAS.2012.2195725","article-title":"Multicasting Mesh AER: A Scalable Assembly Approach for Reconfigurable Neuromorphic Structured AER Systems. Application to ConvNets","volume":"7","year":"2013","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Baby, S.A., Vinod, B., Chinni, C., and Mitra, K. (2017, January 26\u201329). Dynamic Vision Sensors for Human Activity Recognition. Proceedings of the 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), Nanjing, China.","DOI":"10.1109\/ACPR.2017.136"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Amir, A., Taba, B., Berg, D., Melano, T., McKinstry, J., Nolfo, C.D., Nayak, T., Andreopoulos, A., Garreau, G., and Mendoza, M. (2017, January 21\u201326). A Low Power, Fully Event-Based Gesture Recognition System. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.781"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"63","DOI":"10.3389\/fnins.2018.00063","article-title":"A Configurable Event-Driven Convolutional Node with Rate Saturation Mechanism for Modular ConvNet Systems Implementation","volume":"12","year":"2018","journal-title":"Front. Neurosci."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/12\/3404\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:39:38Z","timestamp":1760175578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/12\/3404"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,16]]},"references-count":44,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["s20123404"],"URL":"https:\/\/doi.org\/10.3390\/s20123404","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,6,16]]}}}