{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T03:53:06Z","timestamp":1769917986189,"version":"3.49.0"},"reference-count":43,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2022,9,15]],"date-time":"2022-09-15T00:00:00Z","timestamp":1663200000000},"content-version":"vor","delay-in-days":14,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,9,15]],"date-time":"2022-09-15T00:00:00Z","timestamp":1663200000000},"content-version":"tdm","delay-in-days":14,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Emerging usages for millimeter wave (mmWave) radar have drawn extensive attention and inspired the exploration of learning mmWave radar data. To be effective, instead of using conventional approaches, recent works have employed modern neural network models to process mmWave radar data. However, due to some inevitable obstacles, e.g., noise and sparsity issues in data, the existing approaches are generally customized for specific scenarios. In this paper, we propose a general neuromorphic framework, termed mm-SNN, to process mmWave radar data with spiking neural networks (SNNs), leveraging the intrinsic advantages of SNNs in processing noisy and sparse data. Specifically, we first present the overall design of mm-SNN, which is adaptive and easily expanded for multi-sensor systems. Second, we introduce general and straightforward attention-based improvements into the mm-SNN to enhance the data representation, helping promote performance. Moreover, we conduct explorative experiments to certify the robustness and effectiveness of the mm-SNN. To the best of our knowledge, mm-SNN is the first SNN-based framework that processes mmWave radar data without using extra modules to alleviate the noise and sparsity issues, and at the same time, achieve considerable performance in the task of trajectory estimation.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac889b","type":"journal-article","created":{"date-parts":[[2022,8,10]],"date-time":"2022-08-10T22:17:35Z","timestamp":1660169855000},"page":"034013","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["General spiking neural network framework for the learning trajectory from a noisy mmWave radar"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2147-3870","authenticated-orcid":true,"given":"Xin","family":"Liu","sequence":"first","affiliation":[]},{"given":"Mingyu","family":"Yan","sequence":"additional","affiliation":[]},{"given":"Lei","family":"Deng","sequence":"additional","affiliation":[]},{"given":"Yujie","family":"Wu","sequence":"additional","affiliation":[]},{"given":"De","family":"Han","sequence":"additional","affiliation":[]},{"given":"Guoqi","family":"Li","sequence":"additional","affiliation":[]},{"given":"Xiaochun","family":"Ye","sequence":"additional","affiliation":[]},{"given":"Dongrui","family":"Fan","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2022,9,15]]},"reference":[{"key":"nceac889bbib1","first-page":"265","article-title":"Tensorflow: a system for large-scale machine learning","author":"Abadi","year":"2016"},{"key":"nceac889bbib2","first-page":"111","article-title":"Object classification technique for mmWave FMCW radars using range-FFT features","author":"Bhatia","year":"2021"},{"key":"nceac889bbib3","doi-asserted-by":"publisher","first-page":"956","DOI":"10.3390\/s20040956","article-title":"Spatial attention fusion for obstacle detection using mmWave radar and vision sensor","volume":"20","author":"Chang","year":"2020","journal-title":"Sensors"},{"key":"nceac889bbib4","first-page":"1519","article-title":"LISNN: improving spiking neural networks with lateral interactions for robust object recognition","author":"Cheng","year":"2020"},{"key":"nceac889bbib5","doi-asserted-by":"publisher","first-page":"2228","DOI":"10.1109\/jssc.2020.2970709","article-title":"Tianjic: a unified and scalable chip bridging spike-based and continuous neural computation","volume":"55","author":"Deng","year":"2020","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceac889bbib6","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neunet.2019.09.005","article-title":"Rethinking the performance comparison between SNNS and ANNS","volume":"121","author":"Deng","year":"2020","journal-title":"Neural Netw."},{"key":"nceac889bbib7","first-page":"2758","article-title":"Flownet: learning optical flow with convolutional networks","author":"Dosovitskiy","year":"2015"},{"key":"nceac889bbib8","first-page":"120","article-title":"SuperRF: enhanced 3D RF representation using stationary low-cost mmWave radar","author":"Fang","year":"2020"},{"key":"nceac889bbib9","article-title":"Adaptive cruise control","author":"","year":"2020"},{"key":"nceac889bbib10","author":"Garcia","year":"2018"},{"key":"nceac889bbib11","article-title":"Generative adversarial nets","volume":"vol 27","author":"Goodfellow","year":"2014"},{"key":"nceac889bbib12","first-page":"464","article-title":"Through fog high-resolution imaging using millimeter wave radar","author":"Guan","year":"2020"},{"key":"nceac889bbib13","first-page":"38","article-title":"Pedestrian detection based on fusion of millimeter wave radar and vision","author":"Guo","year":"2018"},{"key":"nceac889bbib14","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1109\/tbcas.2018.2834558","article-title":"Spiking optical flow for event-based sensors using IBM\u2019s TrueNorth neurosynaptic system","volume":"12","author":"Haessig","year":"2018","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"nceac889bbib15","first-page":"1524","article-title":"A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM","author":"Handa","year":"2014"},{"key":"nceac889bbib16","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.neunet.2020.08.001","article-title":"Comparing SNNS and RNNS on neuromorphic vision datasets: similarities and differences","volume":"132","author":"He","year":"2020","journal-title":"Neural Netw."},{"key":"nceac889bbib17","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"nceac889bbib18","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/0004-3702(81)90024-2","article-title":"Determining optical flow","volume":"17","author":"Horn","year":"1981","journal-title":"Artif. Intell."},{"key":"nceac889bbib19","first-page":"7132","article-title":"Squeeze-and-excitation networks","author":"Hu","year":"2018"},{"key":"nceac889bbib20","first-page":"2462","article-title":"Flownet 2.0: evolution of optical flow estimation with deep networks","author":"Ilg","year":"2017"},{"key":"nceac889bbib21","article-title":"Automotive mmWave radar sensors","author":"","year":"2021"},{"key":"nceac889bbib22","first-page":"1","author":"Iovescu","year":"2017"},{"key":"nceac889bbib23","first-page":"5209","article-title":"Hire-SNN: harnessing the inherent robustness of energy-efficient deep spiking neural networks by training with crafted input noise","author":"Kundu","year":"2021"},{"key":"nceac889bbib24","first-page":"1995","article-title":"Convolutional networks for images, speech, and time series","volume":"vol 3361","author":"LeCun","year":"1995"},{"key":"nceac889bbib25","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2016.XII.042","article-title":"Vehicle detection from 3D Lidar using fully convolutional network","author":"Li","year":"2016"},{"key":"nceac889bbib26","first-page":"14","article-title":"See through smoke: robust indoor mapping with low-cost mmWave radar","author":"Lu","year":"2020"},{"key":"nceac889bbib27","first-page":"109","article-title":"milliego: single-chip mmWave radar aided egomotion estimation via deep sensor fusion","author":"Lu","year":"2020"},{"key":"nceac889bbib28","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1016\/s0893-6080(97)00011-7","article-title":"Networks of spiking neurons: the third generation of neural network models","volume":"10","author":"Maass","year":"1997","journal-title":"Neural Netw."},{"key":"nceac889bbib29","first-page":"8026","article-title":"Pytorch: an imperative style, high-performance deep learning library","volume":"vol 32","author":"Paszke","year":"2019"},{"key":"nceac889bbib30","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1038\/s41586-019-1424-8","article-title":"Towards artificial general intelligence with hybrid Tianjic chip architecture","volume":"572","author":"Pei","year":"2019","journal-title":"Nature"},{"key":"nceac889bbib31","doi-asserted-by":"publisher","first-page":"10032","DOI":"10.1109\/jsen.2020.2991741","article-title":"mm-Pose: real-time human skeletal posture estimation using mmWave radars and CNNS","volume":"20","author":"Sengupta","year":"2020","journal-title":"IEEE Sens. J."},{"key":"nceac889bbib32","article-title":"Towards a benchmark for RGB-D SLAM evaluation","author":"Sturm","year":"2011"},{"key":"nceac889bbib33","doi-asserted-by":"publisher","first-page":"971","DOI":"10.1109\/jlt.1985.1074315","article-title":"Precision time domain reflectometry in optical fiber systems using a frequency modulated continuous wave ranging technique","volume":"3","author":"Uttam","year":"1985","journal-title":"J. Lightwave Technol."},{"key":"nceac889bbib34","first-page":"250","article-title":"Learning parallax attention for stereo image super-resolution","author":"Wang","year":"2019"},{"key":"nceac889bbib35","first-page":"2043","article-title":"Deepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks","author":"Wang","year":"2017"},{"key":"nceac889bbib36","first-page":"1","article-title":"Frequency estimation algorithm for ranging of millimeter wave LFMCW radar","author":"Wu","year":"2016"},{"key":"nceac889bbib37","doi-asserted-by":"publisher","first-page":"331","DOI":"10.3389\/fnins.2018.00331","article-title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","volume":"12","author":"Wu","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceac889bbib38","first-page":"1311","article-title":"Direct training for spiking neural networks: faster, larger, better","volume":"vol 33","author":"Wu","year":"2019"},{"key":"nceac889bbib39","first-page":"269","article-title":"mmMesh: towards 3D real-time dynamic human mesh construction using millimeter-wave","author":"Xue","year":"2021"},{"key":"nceac889bbib40","doi-asserted-by":"publisher","DOI":"10.1088\/2634-4386\/abf150","article-title":"Comparing Loihi with a SpiNNaker 2 prototype on low-latency keyword spotting and adaptive robotic control","volume":"1","author":"Yan","year":"2021","journal-title":"Neuromorphic Comput. Eng."},{"key":"nceac889bbib41","first-page":"221","article-title":"Temporal-wise attention spiking neural networks for event streams classification","author":"Yao","year":"2021"},{"key":"nceac889bbib42","first-page":"2790","article-title":"Pu-net: point cloud upsampling network","author":"Yu","year":"2018"},{"key":"nceac889bbib43","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1038\/s41467-019-13827-6","article-title":"An artificial spiking afferent nerve based on Mott memristors for neurorobotics","volume":"11","author":"Zhang","year":"2020","journal-title":"Nat. Commun."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,15]],"date-time":"2022-09-15T15:34:40Z","timestamp":1663256080000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac889b"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,1]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,9,15]]},"published-print":{"date-parts":[[2022,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac889b","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,1]]},"assertion":[{"value":"General spiking neural network framework for the learning trajectory from a noisy mmWave radar","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-02-26","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-08-10","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-09-15","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}