{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T21:25:18Z","timestamp":1776115518098,"version":"3.50.1"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"China National Key R&D Program","award":["2018YFB2100302"],"award-info":[{"award-number":["2018YFB2100302"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Grant No. 61902066"],"award-info":[{"award-number":["Grant No. 61902066"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["Grant NoBK20190336"],"award-info":[{"award-number":["Grant NoBK20190336"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,9,6]]},"abstract":"<jats:p>Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc. This work studies the problem of cross-modal human re-identification (ReID), in response to the regular human movements across camera-allowed regions (e.g., streets) and camera-restricted regions (e.g., offices) deployed with heterogeneous sensors. By leveraging the emerging low-cost RGB-D cameras and mmWave radars, we propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Firstly, to address the fundamental inter-modality discrepancy, we propose a novel signature synthesis algorithm based on the observed specular reflection model of a human body. Secondly, an effective cross-modal deep metric learning model is introduced to deal with interference caused by unsynchronized data across radars and cameras. Through extensive experiments in both indoor and outdoor environments, we demonstrate that our proposed system is able to achieve ~ 92.5% top-1 accuracy and ~ 97.5% top-5 accuracy out of 56 volunteers. We also show that our proposed system is able to robustly reidentify subjects even when multiple subjects are present in the sensors' field of view.<\/jats:p>","DOI":"10.1145\/3550325","type":"journal-article","created":{"date-parts":[[2022,9,7]],"date-time":"2022-09-07T14:54:27Z","timestamp":1662562467000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":40,"title":["Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars"],"prefix":"10.1145","volume":"6","author":[{"given":"Dongjiang","family":"Cao","sequence":"first","affiliation":[{"name":"Southeast University, Nanjing, Jiangsu, China"}]},{"given":"Ruofeng","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Minnesota, Minnesota, Minnesota, United States"}]},{"given":"Hao","family":"Li","sequence":"additional","affiliation":[{"name":"Southeast University, Nanjing, Jiangsu, China"}]},{"given":"Shuai","family":"Wang","sequence":"additional","affiliation":[{"name":"Southeast University, Nanjing, Jiangsu, China"}]},{"given":"Wenchao","family":"Jiang","sequence":"additional","affiliation":[{"name":"Singapore University of Technology and Design, Singapore, Singapore"}]},{"given":"Chris Xiaoxuan","family":"Lu","sequence":"additional","affiliation":[{"name":"The University of Edinburgh, Edinburgh, United Kingdom"}]}],"member":"320","published-online":{"date-parts":[[2022,9,7]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2022. Kinect + Refinement. https:\/\/www.depthkit.tv\/tutorials\/azure-kinect-microsoft-volumetric-capture-depth-workflow-depthkit."},{"key":"e_1_2_1_2_1","unstructured":"2022. wholehome-ai-sensor. https:\/\/consumer.huawei.com\/cn\/wholehome\/ai-sensor\/."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818072"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuCAP.2012.6206694"},{"key":"e_1_2_1_5_1","volume-title":"ST-DBSCAN: An algorithm for clustering spatial-temporal data. Data & knowledge engineering 60, 1","author":"Birant Derya","year":"2007","unstructured":"Derya Birant and Alp Kut. 2007. ST-DBSCAN: An algorithm for clustering spatial-temporal data. Data & knowledge engineering 60, 1 (2007), 208--221."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3214266"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2014.2352114"},{"key":"e_1_2_1_8_1","first-page":"1","article-title":"Person reidentification based on automotive radar point clouds","volume":"60","author":"Cheng Yuwei","year":"2021","unstructured":"Yuwei Cheng and Yimin Liu. 2021. Person reidentification based on automotive radar point clouds. IEEE Transactions on Geoscience and Remote Sensing 60 (2021), 1--13.","journal-title":"IEEE Transactions on Geoscience and Remote Sensing"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2017.02.014"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01231-1_17"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3268866.3268868"},{"key":"e_1_2_1_12_1","volume-title":"Julian FP Kooij, and Eric Granger","author":"Hafner Frank","year":"2018","unstructured":"Frank Hafner, Amran Bhuiyan, Julian FP Kooij, and Eric Granger. 2018. A cross-modal distillation network for person re-identification in rgb-depth. arXiv preprint arXiv:1810.11641 (2018)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.138"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372224.3419202"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01228-1_44"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384419.3430733"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3300061.3345437"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00243"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00818"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432235"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210342"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386901.3388945"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384419.3430776"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i01.5430"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243043"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2013.17"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.01006"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Amol S Patwardhan. 2017. Hostile behavior detection from multiple view points using RGB-D sensor. In 2017 IEEE SmartWorld Ubiquitous Intelligence & Computing Advanced & Trusted Computed Scalable Computing & Communications Cloud & Big Data Computing Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/CBDCom\/IOP\/SCI). IEEE 1--6.","DOI":"10.1109\/UIC-ATC.2017.8397461"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TGRS.2020.3019915"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition. 652--660","author":"Qi Charles R","year":"2017","unstructured":"Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 652--660."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.1998.710701"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450268.3453532"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2018.8545849"},{"key":"e_1_2_1_34_1","volume-title":"Privacy-preserving fall detection with deep learning on mmWave radar signal. In 2019 IEEE Visual Communications and Image Processing (VCIP)","author":"Sun Yangfan","unstructured":"Yangfan Sun, Renlong Hang, Zhu Li, Mouqing Jin, and Kelvin Xu. 2019. Privacy-preserving fall detection with deep learning on mmWave radar signal. In 2019 IEEE Visual Communications and Image Processing (VCIP). IEEE, 1--4."},{"key":"e_1_2_1_35_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-60636-7_27"},{"key":"e_1_2_1_37_1","volume-title":"Adversarial Multi-scale Feature Learning for Person Re-identification. arXiv preprint arXiv:2012.14061","author":"Wang Xinglu","year":"2020","unstructured":"Xinglu Wang. 2020. Adversarial Multi-scale Feature Learning for Person Re-identification. arXiv preprint arXiv:2012.14061 (2020)."},{"key":"e_1_2_1_38_1","volume-title":"Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5","author":"Wang Yue","year":"2019","unstructured":"Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5 (2019), 1--12."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2675201"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.575"},{"key":"e_1_2_1_41_1","first-page":"1","article-title":"ivr: Integrated vision and radio localization with zero human effort","volume":"3","author":"Xu Jingao","year":"2019","unstructured":"Jingao Xu, Hengjie Chen, Kun Qian, Erqun Dong, Min Sun, Chenshu Wu, Li Zhang, and Zheng Yang. 2019. ivr: Integrated vision and radio localization with zero human effort. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--22.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01249-6_10"},{"key":"e_1_2_1_43_1","volume-title":"MU-ID: Multi-user Identification Through Gaits Using Millimeter Wave Radios. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 2589--2598","author":"Yang Xin","year":"2020","unstructured":"Xin Yang, Jian Liu, Yingying Chen, Xiaonan Guo, and Yucheng Xie. 2020. MU-ID: Multi-user Identification Through Gaits Using Millimeter Wave Radios. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 2589--2598."},{"key":"e_1_2_1_44_1","first-page":"2","article-title":"Visible thermal person re-identification via dual-constrained top-ranking","volume":"1","author":"Ye Mang","year":"2018","unstructured":"Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C Yuen. 2018. Visible thermal person re-identification via dual-constrained top-ranking.. In IJCAI, Vol. 1. 2.","journal-title":"IJCAI"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gaitpost.2021.04.005"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPSN.2016.7460727"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/DCOSS.2019.00028"},{"key":"e_1_2_1_48_1","volume-title":"Person re-identification in the 3d space. arXiv preprint arXiv:2006.04569","author":"Zheng Zhedong","year":"2020","unstructured":"Zhedong Zheng and Yi Yang. 2020. Person re-identification in the 3d space. arXiv preprint arXiv:2006.04569 (2020)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2019.8737624"},{"key":"e_1_2_1_50_1","volume-title":"Unsupervised 3d human mesh recovery from noisy point clouds. arXiv preprint arXiv:2107.07539","author":"Zuo Xinxin","year":"2021","unstructured":"Xinxin Zuo, Sen Wang, Minglun Gong, and Li Cheng. 2021. Unsupervised 3d human mesh recovery from noisy point clouds. arXiv preprint arXiv:2107.07539 (2021)."}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550325","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550325","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:42:47Z","timestamp":1752468167000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550325"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9,6]]}},"alternative-id":["10.1145\/3550325"],"URL":"https:\/\/doi.org\/10.1145\/3550325","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]},"assertion":[{"value":"2022-09-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}