{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:16:53Z","timestamp":1761581813370,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2019,3,23]],"date-time":"2019-03-23T00:00:00Z","timestamp":1553299200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61271353","61871389"],"award-info":[{"award-number":["61271353","61871389"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Three-dimensional (3D) object detection has important applications in robotics, automatic loading, automatic driving and other scenarios. With the improvement of devices, people can collect multi-sensor\/multimodal data from a variety of sensors such as Lidar and cameras. In order to make full use of various information advantages and improve the performance of object detection, we proposed a Complex-Retina network, a convolution neural network for 3D object detection based on multi-sensor data fusion. Firstly, a unified architecture with two feature extraction networks was designed, and the feature extraction of point clouds and images from different sensors realized synchronously. Then, we set a series of 3D anchors and projected them to the feature maps, which were cropped into 2D anchors with the same size and fused together. Finally, the object classification and 3D bounding box regression were carried out on the multipath of fully connected layers. The proposed network is a one-stage convolution neural network, which achieves the balance between the accuracy and speed of object detection. The experiments on KITTI datasets show that the proposed network is superior to the contrast algorithms in average precision (AP) and time consumption, which shows the effectiveness of the proposed network.<\/jats:p>","DOI":"10.3390\/s19061434","type":"journal-article","created":{"date-parts":[[2019,3,25]],"date-time":"2019-03-25T06:56:52Z","timestamp":1553497012000},"page":"1434","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["One-Stage Multi-Sensor Data Fusion Convolutional Neural Network for 3D Object Detection"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9226-0261","authenticated-orcid":false,"given":"Minle","family":"Li","sequence":"first","affiliation":[{"name":"State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yihua","family":"Hu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China"},{"name":"Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei 230037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nanxiang","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China"},{"name":"Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei 230037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qishu","family":"Qian","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering (National University of Defence Technology), Hefei 230037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lin, T., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (arXiv, 2015). Microsoft COCO: Common Objects in Context, arXiv.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Cordts, M. (arXiv, 2016). The Cityscapes Dataset for Semantic Urban Scene Understanding, arXiv.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_3","unstructured":"Tomov, N.-S., and Tomov, S. (arXiv, 2018). On Deep Neural Networks for Detecting Heart Disease, arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jin, D., and Szolovits, P. (arXiv, 2018). Advancing PICO Element Detection in Medical Text via Deep Neural Networks, arXiv.","DOI":"10.18653\/v1\/W18-2308"},{"key":"ref_5","unstructured":"Tokuda, K.E., Lockerman, Y., Ferreira, B.A.G., Sorrelgreen, E., Boyle, D., Cesar, M.R., and Silva, T.C. (arXiv, 2018). A new approach for pedestrian density estimation using moving sensors and computer vision, arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4130","DOI":"10.1109\/TGRS.2016.2537830","article-title":"Automated Detection of Three-Dimensional Cars in Mobile Laser Scanning Point Clouds Using DBM-Hough-Forests","volume":"54","author":"Yu","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 24\u201327). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","unstructured":"Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5\u201310). R-FCN: Object detection via region-based fully convolutional networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Cai, Z., and Vasconcelos, N. (2018, January 18\u201322). Cascade R-CNN: Delving into High Quality Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00644"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hu, H., Gu, J., Zhang, Z., Dai, J., and Wei, Y. (2018, January 18\u201322). Relation Networks for Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00378"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Singh, B., and Davis, S.L. (arXiv, 2017). An Analysis of Scale Invariance in Object Detection\u2014SNIP, arXiv.","DOI":"10.1109\/CVPR.2018.00377"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F. (2009, January 20\u201325). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8\u201316). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, Faster, Stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_18","unstructured":"Redmon, J., and Farhadi, A. (arXiv, 2018). YOLOv3: An Incremental Improvement, arXiv."},{"key":"ref_19","unstructured":"Fu, C.-Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (arXiv, 2017). DSSD: Deconvolutional Single Shot Detector, arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Qiao, S., Xie, C., Shen, W., Wang, B., and Yuille, L.A. (2018, January 18\u201322). Single-Shot Object Detection with Enriched Semantics. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00609"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017, January 22\u201329). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhou, P., Ni, B., Geng, C., Hu, J., and Xu, Y. (2018, January 18\u201322). Scale-Transferrable Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00062"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S. (2018, January 18\u201322). Single-Shot Refinement Neural Network for Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00442"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21\u201326). Multi-view 3D object detection network for autonomous driving. Proceedings of the IEEE Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.691"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ku, J., Mozifian, M., Lee, J., Harakeh, A., and Waslander, S. (2018, January 1\u20135). Joint 3D Proposal Generation and Object Detection from View Aggregation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594049"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2018, January 18\u201322). VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201322). Frustum PointNets for 3D Object Detection from RGB-D Data. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"},{"key":"ref_29","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Simon, M., Milz, S., Amende, K., and Gross, H.M. (arXiv, 2018). Complex-YOLO: Real-time 3D Object Detection on Point Clouds, arXiv.","DOI":"10.1109\/CVPRW.2019.00158"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ali, W., Abdelkarim, S., Zahran, M., Zidan, M., and Sallab, A.E. (arXiv, 2018). YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud, arXiv.","DOI":"10.1007\/978-3-030-11015-4_54"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, B., Luo, W., and Urtasun, R. (2018, January 18\u201322). PIXOR: Real-time 3D Object Detection from Point Clouds. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00798"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_34","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Beltran, J., Guindel, C., Moreno, F., Cruzado, D., Garcia, F., and Escalera, A. (arXiv, 2018). BirdNet: A 3D Object Detection Framework from LiDAR information, arXiv.","DOI":"10.1109\/ITSC.2018.8569311"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1109\/LRA.2018.2852843","article-title":"RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving","volume":"3","author":"Zeng","year":"2018","journal-title":"IEEE Robot. Autom Lett."},{"key":"ref_38","unstructured":"Gustafsson, F., and Linder-Nor\u00e9n, E. (2018). Automotive 3D Object Detection without Target Domain Annotations. [Master\u2019s Thesis, Link\u00f6ping University]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/6\/1434\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:40:12Z","timestamp":1760186412000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/6\/1434"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,23]]},"references-count":38,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2019,3]]}},"alternative-id":["s19061434"],"URL":"https:\/\/doi.org\/10.3390\/s19061434","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,3,23]]}}}