{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T07:59:23Z","timestamp":1773734363180,"version":"3.50.1"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,3,27]]},"abstract":"<jats:p>Estimating 3D human mesh is appealing for various application scenarios. Current mainstream solution predicts the meshes either from the image or the human reflected RF-signals. In this paper, instead of investigating which approach is better, we propose to design a multi-modality fusion framework, namely MI-Mesh, which estimates 3D meshes by fusing image and mmWave. To realize this, we design a deep neural network model. It first automatically correlate mmWave point clouds to certain human joints and extracts useful fused features from two modalities. Then, the features are refined by predicting 2D joints and silhouette. Finally, we regress pose and shape parameters and feed them to SMPL model to generate the 3D human meshes. We build a prototype on commercial mmWave radar and camera. The experimental results demonstrate that with the integration of multi-modality strengths, MI-Mesh can effectively recover human meshes on dynamic motions and across different conditions.<\/jats:p>","DOI":"10.1145\/3580861","type":"journal-article","created":{"date-parts":[[2023,3,28]],"date-time":"2023-03-28T14:57:51Z","timestamp":1680015471000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["MI-Mesh"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5274-7988","authenticated-orcid":false,"given":"Han","family":"Ding","sequence":"first","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4800-9074","authenticated-orcid":false,"given":"Zhenbin","family":"Chen","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4603-4914","authenticated-orcid":false,"given":"Cui","family":"Zhao","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0750-6990","authenticated-orcid":false,"given":"Fei","family":"Wang","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3845-1646","authenticated-orcid":false,"given":"Ge","family":"Wang","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9348-2982","authenticated-orcid":false,"given":"Wei","family":"Xi","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1037-9619","authenticated-orcid":false,"given":"Jizhong","family":"Zhao","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]}],"member":"320","published-online":{"date-parts":[[2023,3,28]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"AWR1443 Single-Chip 76-GHz to 81-GHz Automotive Radar Sensor Evaluation Module. https:\/\/www.ti.com\/tool\/AWR1443BOOST? keyMatch=AWR1443BOOST. Texas Instruments","year":"2020","unstructured":"[n.d.]. AWR1443 Single-Chip 76-GHz to 81-GHz Automotive Radar Sensor Evaluation Module. https:\/\/www.ti.com\/tool\/AWR1443BOOST? keyMatch=AWR1443BOOST. Texas Instruments, 2020."},{"key":"e_1_2_1_2_1","volume-title":"Texas Instruments","year":"2019","unstructured":"[n.d.]. Real-Time Data-Capture Adapter for Radar Sensing Evaluation Module. https:\/\/www.ti.com\/tool\/DCA1000EVM. Texas Instruments, 2019."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818072"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of USENIX NSDI.","author":"Adib Fadel","year":"2014","unstructured":"Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3D Tracking via Body Radio Reflections. In Proceedings of USENIX NSDI."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.143"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.610"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3483251"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.584"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2017.141"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2809695.2809708"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02291478"},{"key":"e_1_2_1_12_1","unstructured":"Xiaonan Guo Jian Liu Cong Shi Hongbo Liu Yingying Chen and Mooi Choo Chuah. 2018. Device-Free Personalized Fitness Assistant using WiFi. (2018)."},{"key":"e_1_2_1_13_1","volume-title":"Identity Mappings in Deep Residual Networks","author":"He Kaiming","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity Mappings in Deep Residual Networks. In Proceedings of Springer ECCV."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372224.3380900"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00234"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00463"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01140"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517253"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448110"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00055"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00233"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of IEEE\/CVF CVPR.","author":"Qi Charles","year":"2017","unstructured":"Charles Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. 2017. Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of IEEE\/CVF CVPR."},{"key":"e_1_2_1_24_1","volume-title":"Deep Hierarchical Feature Learning on Point Sets in A Metric Space. Advances in Neural Information Processing Systems 30","author":"Qi Charles","year":"2017","unstructured":"Charles Qi, Li Yi, Hao Su, and Leonidas Guibas. 2017. Pointnet++: Deep Hierarchical Feature Learning on Point Sets in A Metric Space. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494973"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534605"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3081333.3081364"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3264947"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of NeurIPS.","author":"Tompson Jonathan J","year":"2014","unstructured":"Jonathan J Tompson, Arjun Jain, Yann LeCun, and Christoph Bregler. 2014. Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation. In Proceedings of NeurIPS."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.492"},{"key":"e_1_2_1_31_1","volume-title":"Attention is All You Need. Advances in Neural Information Processing Systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2016.2557792"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3300061.3300133"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01584"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458864.3467679"},{"key":"e_1_2_1_36_1","volume-title":"Lift: Learned Invariant Feature Transform","author":"Yi Kwang Moo","year":"2016","unstructured":"Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal Fua. 2016. Lift: Learned Invariant Feature Transform. In Proceedings of Springer ECCV."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00708"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.3026732"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.01021"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3326081"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00589"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580861","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580861","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:44:28Z","timestamp":1752468268000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580861"}},"subtitle":["3D Human Mesh Construction by Fusing Image and Millimeter Wave"],"short-title":[],"issued":{"date-parts":[[2023,3,27]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,3,27]]}},"alternative-id":["10.1145\/3580861"],"URL":"https:\/\/doi.org\/10.1145\/3580861","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,27]]},"assertion":[{"value":"2023-03-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}