{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:58:02Z","timestamp":1783439882646,"version":"3.54.6"},"reference-count":53,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,3,14]],"date-time":"2023-03-14T00:00:00Z","timestamp":1678752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Project of National Nature Science Foundation of China","award":["61932012"],"award-info":[{"award-number":["61932012"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Today, multi-sensor fusion detection frameworks in autonomous driving, especially sequence-based data-level fusion frameworks, face high latency and coupling issues and generally perform worse than LiDAR-only detectors. On this basis, we propose PMPF, point-cloud multiple-pixel fusion, for 3D object detection. PMPF projects the point cloud data onto the image plane, where the region pixels are processed to correspond with the points and decorated to the point cloud data, such that the fused point cloud data can be applied to LiDAR-only detectors with autoencoders. PMPF is a plug-and-play, decoupled multi-sensor fusion detection framework with low latency. Extensive experiments on the KITTI 3D object detection benchmark show that PMPF vastly improves upon most of the LiDAR-only detectors, e.g., PointPillars, SECOND, CIA-SSD, SE-SSD four state-of-the-art one-stage detectors, and PointRCNN, PV-RCNN, Part-A2 three two-stage detectors.<\/jats:p>","DOI":"10.3390\/rs15061580","type":"journal-article","created":{"date-parts":[[2023,3,14]],"date-time":"2023-03-14T06:14:58Z","timestamp":1678774498000},"page":"1580","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["PMPF: Point-Cloud Multiple-Pixel Fusion-Based 3D Object Detection for Autonomous Driving"],"prefix":"10.3390","volume":"15","author":[{"given":"Yan","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Mechanical Electronic & Information Engineering, China University of Mining & Technology (Beijing), Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8413-123X","authenticated-orcid":false,"given":"Kang","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Mechanical Electronic & Information Engineering, China University of Mining & Technology (Beijing), Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong","family":"Bao","sequence":"additional","affiliation":[{"name":"College of Robotic, Beijing Union University, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Mechanical Electronic & Information Engineering, China University of Mining & Technology (Beijing), Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Mechanical Electronic & Information Engineering, China University of Mining & Technology (Beijing), Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"eaav9843","DOI":"10.1126\/scirobotics.aav9843","article-title":"Self-Driving Cars: A City Perspective","volume":"4","author":"Duarte","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3135","DOI":"10.1109\/TITS.2019.2926042","article-title":"Is It Safe to Drive? An Overview of Factors, Metrics, and Datasets for Driveability Assessment in Autonomous Driving","volume":"21","author":"Guo","year":"2020","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"E1","DOI":"10.1038\/s41586-020-1987-4","article-title":"Life and Death Decisions of Autonomous Vehicles","volume":"579","author":"Bigman","year":"2020","journal-title":"Nature"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.1109\/TITS.2016.2639582","article-title":"Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds","volume":"18","author":"Huang","year":"2017","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5110","DOI":"10.1109\/TITS.2019.2949005","article-title":"Surrounding Vehicle Detection Using an FPGA Panoramic Camera and Deep CNNs","volume":"21","author":"Chen","year":"2020","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2018.2849505","article-title":"Traffic Light Recognition With High Dynamic Range Imaging and Deep Learning","volume":"20","author":"Wang","year":"2019","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision Meets Robotics: The KITTI Dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vora, S., Lang, A.H., Helou, B., and Beijbom, O. (2020, January 13\u201319). PointPainting: Sequential Fusion for 3D object detection. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00466"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 15\u201320). PointPillars: Fast Encoders for object detection From Point Clouds. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01298"},{"key":"ref_11","unstructured":"Yang, Z., Sun, Y., Liu, S., Shen, X., and Jia, J. (November, January 27). STD: Sparse-to-Dense 3D Object Detector for Point Cloud. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"722","DOI":"10.1109\/TITS.2020.3023541","article-title":"Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review","volume":"23","author":"Cui","year":"2022","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yan, Y., Mao, Y., and Li, B. (2018). SECOND: Sparsely Embedded Convolutional Detection. Sensors, 18.","DOI":"10.3390\/s18103337"},{"key":"ref_14","first-page":"3555","article-title":"CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud","volume":"35","author":"Zheng","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zheng, W., Tang, W., Jiang, L., and Fu, C.-W. (2021, January 20\u201325). SE-SSD: Self-Ensembling Single-Stage Object Detector from Point Cloud. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01426"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shi, S., Wang, X., and Li, H. (2019, January 15\u201320). PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00086"},{"key":"ref_17","unstructured":"Shi, S., Guo, C., Yang, J., and Li, H. (2020). PV-RCNN: The Top-Performing LiDAR-Only Solutions for 3D Detection \/ 3D Tracking \/ Domain Adaptation of Waymo Open Dataset Challenges. arXiv."},{"key":"ref_18","first-page":"2647","article-title":"From Points to Parts: 3D object detection from Point Cloud with Part-Aware and Part-Aggregation Network","volume":"43","author":"Shi","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA."},{"key":"ref_20","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917), Red Hook, NY, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2018, January 18\u201323). VoxelNet: End-to-End Learning for Point Cloud Based 3D object detection. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Graham, B., Engelcke, M., and van der Maaten, L. (2018, January 18\u201323). 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00961"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, J., Wang, J., Xu, D., and Li, Y. (2021). HCNET: A Point Cloud object detection Network Based on Height and Channel Attention. Remote Sens., 13.","DOI":"10.3390\/rs13245071"},{"key":"ref_24","unstructured":"Ge, R., Ding, Z., Hu, Y., Wang, Y., Chen, S., Huang, L., and Li, Y. (2020). AFDet: Anchor Free One Stage 3D object detection. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24553-9"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201323). Frustum PointNets for 3D object detection From RGB-D Data. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, Z., and Jia, K. (2019, January 3\u20138). Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D object detection. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968513"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Du, X., Ang, M.H., Karaman, S., and Rus, D. (2018, January 21\u201325). A General Pipeline for 3D Detection of Vehicles. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8461232"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shin, K., Kwon, Y.P., and Tomizuka, M. (2019, January 9\u201312). RoarNet: A Robust 3D object detection Based on RegiOn Approximation Refinement. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813895"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yang, Z., Sun, Y., Liu, S., Shen, X., and Jia, J. (2018). IPOD: Intensive Point-Based Object Detector for Point Cloud. arXiv.","DOI":"10.1109\/ICCV.2019.00204"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Imad, M., Doukhi, O., and Lee, D.-J. (2021). Transfer Learning Based Semantic Segmentation for 3D object detection from Point Cloud. Sensors, 21.","DOI":"10.3390\/s21123964"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Huang, T., Liu, Z., Chen, X., and Bai, X. (2020). EPNet: Enhancing Point Features with Image Se-mantics for 3D object detection. arXiv.","DOI":"10.1007\/978-3-030-58555-6_3"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21\u201326). Multi-View 3D object detection Network for Autonomous Driving. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.691"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ku, J., Mozifian, M., Lee, J., Harakeh, A., and Waslander, S.L. (2018, January 1\u20135). Joint 3D Proposal Generation and object detection from View Aggregation. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594049"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lu, H., Chen, X., Zhang, G., Zhou, Q., Ma, Y., and Zhao, Y. (2019, January 12\u201317). Scanet: Spatial-Channel Attention Network for 3D object detection. Proceedings of the ICASSP 2019\u20142019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682746"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yuan, Q., and Mohd Shafri, H.Z. (2022). Multi-Modal Feature Fusion Network with Adaptive Center Point Detector for Building Instance Extraction. Remote Sens., 14.","DOI":"10.3390\/rs14194920"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zheng, W., Xie, H., Chen, Y., Roh, J., and Shin, H. (2022). PIFNet: 3D object detection Using Joint Image and Point Cloud Features for Autonomous Driving. Appl. Sci., 12.","DOI":"10.3390\/app12073686"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Liu, L., He, J., Ren, K., Xiao, Z., and Hou, Y. (2022). A LiDAR\u2013Camera Fusion 3D object detection Algorithm. Information, 13.","DOI":"10.3390\/info13040169"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, J., Zhu, M., Wang, B., Sun, D., Wei, H., Liu, C., and Nie, H. (2020). KDA3D: Key-Point Densification and Multi-Attention Guidance for 3D object detection. Remote Sens., 12.","DOI":"10.3390\/rs12111895"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"90801","DOI":"10.1109\/ACCESS.2019.2927012","article-title":"MCF3D: Multi-Stage Complementary Fusion for Multi-Sensor 3D object detection","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pang, S., Morris, D., and Radha, H. (2020). CLOCs: Camera-LiDAR Object Candidates Fusion for 3D object detection. arXiv.","DOI":"10.1109\/IROS45743.2020.9341791"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Wang, S., and Urtasun, R. (2018, January 8\u201314). Deep Continuous Fusion for Multi-Sensor 3D object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01270-0_39"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Chen, Y., Hu, R., and Urtasun, R. (2019, January 15\u201320). Multi-Task Multi-Sensor Fusion for 3D object detection. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00752"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Sindagi, V.A., Zhou, Y., and Tuzel, O. (2019, January 20\u201324). MVX-Net: Multimodal VoxelNet for 3D object detection. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794195"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6\u201312). Sliding Shapes for 3D object detection in Depth Images. Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10599-4"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Song, S., and Xiao, J. (2016, January 27\u201330). Deep Sliding Shapes for Amodal 3D object detection in RGB-D Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.94"},{"key":"ref_47","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wang, S., Suo, S., Ma, W.-C., Pokrovsky, A., and Urtasun, R. (2018, January 18\u201323). Deep Parametric Continuous Convolutional Neural Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00274"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/3477.764879","article-title":"Genetic K-Means Algorithm","volume":"29","author":"Krishna","year":"1999","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.patrec.2009.09.011","article-title":"Data Clustering: 50 Years beyond K-Means","volume":"31","author":"Jain","year":"2010","journal-title":"Pattern Recognit. Lett."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"108796","DOI":"10.1016\/j.patcog.2022.108796","article-title":"3D object detection for Autonomous Driving: A Survey","volume":"130","author":"Qian","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10590-1"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/6\/1580\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:54:38Z","timestamp":1760122478000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/6\/1580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,14]]},"references-count":53,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["rs15061580"],"URL":"https:\/\/doi.org\/10.3390\/rs15061580","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,14]]}}}