{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,7]],"date-time":"2025-12-07T03:37:36Z","timestamp":1765078656010,"version":"3.41.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2024,10,29]],"date-time":"2024-10-29T00:00:00Z","timestamp":1730160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61871196 and 62302049"],"award-info":[{"award-number":["61871196 and 62302049"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003392","name":"Natural Science Foundation of Fujian Province","doi-asserted-by":"crossref","award":["2022J01317"],"award-info":[{"award-number":["2022J01317"]}],"id":[{"id":"10.13039\/501100003392","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Promotion Program for Young and Middle-aged Teacher in Science and Technology Research of Huaqiao University","award":["ZQN-YX601"],"award-info":[{"award-number":["ZQN-YX601"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of \u201cmoderate\u201d level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of \u201cmoderate\u201d level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.<\/jats:p>","DOI":"10.1145\/3686157","type":"journal-article","created":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T16:04:57Z","timestamp":1722614697000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6641-6469","authenticated-orcid":false,"given":"Huaijin","family":"Liu","sequence":"first","affiliation":[{"name":"School of Medical Technology and Information Engineering, Zhejiang Chinese Medical University, Hangzhou, China and College of Mechanical Engineering and Automation, Huaqiao University, Xiamen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2386-770X","authenticated-orcid":false,"given":"Jixiang","family":"Du","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Huaqiao University, Xiamen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5990-3154","authenticated-orcid":false,"given":"Yong","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Mechanical Engineering and Automation, Huaqiao University, Xiamen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5536-5224","authenticated-orcid":false,"given":"Hongbo","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Huaqiao University, Xiamen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1349-8077","authenticated-orcid":false,"given":"Jiandian","family":"Zeng","sequence":"additional","affiliation":[{"name":"Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,10,29]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01838"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594049"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00472"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2023.3307589"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2023.3309600"},{"key":"e_1_3_1_7_2","first-page":"7345","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Ming Liang","year":"2019","unstructured":"Liang Ming, Yang Bin, Chen Yun, Hu Rui, and Urtasun Raquel. 2019. Multi-task multi-sensor fusion for 3D object detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 7345\u20137353."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.3390\/s18103337"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01298"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01426"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00102"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00086"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01105"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i1.19897"},{"issue":"1","key":"e_1_3_1_15_2","first-page":"1","article-title":"Semantic map guided identity transfer GAN for person re-identification","volume":"9","author":"Wu Tian","year":"2023","unstructured":"Tian Wu, Rongbo Zhu, and Shaohua Wan. 2023. Semantic map guided identity transfer GAN for person re-identification. ACM Transactions on Multimedia Computing, Communications and Applications 9, 1 (2023), 1\u201319.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"issue":"4","key":"e_1_3_1_16_2","first-page":"2611","article-title":"DLS-GAN: generative adversarial nets for defect location sensitive data augmentation","volume":"19","author":"Li Wei","year":"2023","unstructured":"Wei Li, Chengchun Gu, Jinlin Chen, Chao Ma, Xiaowu Zhang, Bin Chen and Shaohua Wan. 2023. DLS-GAN: generative adversarial nets for defect location sensitive data augmentation. IEEE Transactions on Automation Science and Engineering 19, 4 (2023), 2611\u20132627.","journal-title":"IEEE Transactions on Automation Science and Engineering"},{"key":"e_1_3_1_17_2","first-page":"4604","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Sourabh Vora","year":"2020","unstructured":"Vora Sourabh, Lang Alex H, Helou Bassam, and Beijbom Oscar. 2020. Pointpainting: Sequential fusion for 3D object detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 4604\u20134612."},{"key":"e_1_3_1_18_2","first-page":"35","volume-title":"Proceedings of the 16th European Conference on Computer Vision (ECCV \u201920)","author":"Tengteng Huang","year":"2020","unstructured":"Huang Tengteng, Liu Zhe, Chen Xiwu, and Bai Xiang. 2020. Epnet: Enhancing point features with image semantics for 3d object detection. In Proceedings of the 16th European Conference on Computer Vision (ECCV \u201920), 35\u201352."},{"issue":"8","key":"e_1_3_1_19_2","first-page":"2647","article-title":"From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network","volume":"43","author":"Shi Shaoshuai","year":"2020","unstructured":"Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. 2020. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 8 (2020), 2647\u20132664.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i2.16207"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00274"},{"key":"e_1_3_1_22_2","first-page":"652","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Qi Charles R.","year":"2017","unstructured":"Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652\u2013660."},{"key":"e_1_3_1_23_2","first-page":"5099","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Qi Charles Ruizhongtai","year":"2017","unstructured":"Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Proceedings of the Advances in Neural Information Processing Systems, 5099\u20135108."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00204"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01054"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01437"},{"key":"e_1_3_1_27_2","first-page":"10386","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS \u201920)","author":"Su Pang","year":"2020","unstructured":"Pang Su, Morris Daniel, and Radha Hayder. 2020. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS \u201920), 10386\u201310393."},{"key":"e_1_3_1_28_2","first-page":"5291","volume-title":"Proceedings of the IEEE Transactions on Multimedia","volume":"25","author":"Hanqi Zhu","year":"2022","unstructured":"Zhu Hanqi, Deng Jiajun, Zhang Yu, Ji Jianmin, Mao Qiuyu, Li Houqiang, and Zhang Yanyong. 2022. VPFNet: Improving 3D object detection with virtual point based LiDAR and stereo data fusion. In Proceedings of the IEEE Transactions on Multimedia, Vol. 25, 5291\u20135304"},{"key":"e_1_3_1_29_2","first-page":"5418","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xiaopei Wu","year":"2022","unstructured":"Wu Xiaopei, Peng Liang, Yang Honghui, Xie Liang, Huang Chenxi, Deng Chengqi, Liu Haifeng and Cai Deng. 2022. Sparse fuse dense: Towards high quality 3D detection with depth completion. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 5418\u20135427."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01112"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/2354409.2354978"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00987"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01189"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i07.6837"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00315"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00738"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00272"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00329"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01295"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58583-9_43"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686157","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3686157","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:50Z","timestamp":1750295870000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686157"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,29]]},"references-count":41,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3686157"],"URL":"https:\/\/doi.org\/10.1145\/3686157","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2024,10,29]]},"assertion":[{"value":"2023-07-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}