{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T07:14:48Z","timestamp":1761808488082,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,2,21]],"date-time":"2022-02-21T00:00:00Z","timestamp":1645401600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>One of the primary tasks undertaken by autonomous vehicles (AVs) is object detection, which comes ahead of object tracking, trajectory estimation, and collision avoidance. Vulnerable road objects (e.g., pedestrians, cyclists, etc.) pose a greater challenge to the reliability of object detection operations due to their continuously changing behavior. The majority of commercially available AVs, and research into them, depends on employing expensive sensors. However, this hinders the development of further research on the operations of AVs. In this paper, therefore, we focus on the use of a lower-cost single-beam LiDAR in addition to a monocular camera to achieve multiple 3D vulnerable object detection in real driving scenarios, all the while maintaining real-time performance. This research also addresses the problems faced during object detection, such as the complex interaction between objects where occlusion and truncation occur, and the dynamic changes in the perspective and scale of bounding boxes. The video-processing module works upon a deep-learning detector (YOLOv3), while the LiDAR measurements are pre-processed and grouped into clusters. The output of the proposed system is objects classification and localization by having bounding boxes accompanied by a third depth dimension acquired by the LiDAR. Real-time tests show that the system can efficiently detect the 3D location of vulnerable objects in real-time scenarios.<\/jats:p>","DOI":"10.3390\/s22041663","type":"journal-article","created":{"date-parts":[[2022,2,21]],"date-time":"2022-02-21T20:48:41Z","timestamp":1645476521000},"page":"1663","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Evaluation of 3D Vulnerable Objects\u2019 Detection Using a Multi-Sensors System for Autonomous Vehicles"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8006-7699","authenticated-orcid":false,"given":"Esraa","family":"Khatab","sequence":"first","affiliation":[{"name":"Arab Academy for Science, Technology and Maritime Transport, Alexandria 1029, Egypt"},{"name":"School of Engineering, University of Central Lancashire, Preston PR1 2HE, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0803-5374","authenticated-orcid":false,"given":"Ahmed","family":"Onsy","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Central Lancashire, Preston PR1 2HE, UK"}]},{"given":"Ahmed","family":"Abouelfarag","sequence":"additional","affiliation":[{"name":"Arab Academy for Science, Technology and Maritime Transport, Alexandria 1029, Egypt"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,21]]},"reference":[{"key":"ref_1","unstructured":"Department for Transport (2015). The Pathway to Driverless Cars: Summary Report and Action Plan."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5356","DOI":"10.1109\/ACCESS.2016.2603219","article-title":"Internet of vehicles: Motivation, layered architecture, network model, challenges, and future aspects","volume":"4","author":"Kaiwartya","year":"2016","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Arena, F., and Pau, G. (2019). An overview of vehicular communications. Future Internet, 11.","DOI":"10.3390\/fi11020027"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.trpro.2020.02.049","article-title":"How Do Autonomous Cars Work?","volume":"44","author":"Kolla","year":"2020","journal-title":"Transp. Res. Procedia"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.vlsi.2021.01.002","article-title":"Vulnerable objects detection for autonomous driving: A review","volume":"78","author":"Khatab","year":"2021","journal-title":"Integration"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ahangar, M.N., Ahmed, Q.Z., Khan, F.A., and Hafeez, M. (2021). A survey of autonomous vehicles: Enabling communication technologies and challenges. Sensors, 21.","DOI":"10.3390\/s21030706"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2584","DOI":"10.1109\/TITS.2017.2658662","article-title":"Overview of environment perception for intelligent vehicles","volume":"18","author":"Zhu","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1016\/j.trc.2018.02.012","article-title":"Autonomous vehicle perception: The technology of today and tomorrow","volume":"89","author":"Gruyer","year":"2018","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.iatssr.2019.11.005","article-title":"Automated driving recognition technologies for adverse weather conditions","volume":"43","author":"Yoneda","year":"2019","journal-title":"IATSS Res."},{"key":"ref_10","unstructured":"(2018). SAE On-Road Automated Vehicle Standards Committee and Others, Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, SAE International."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Dai, D., Chen, Z., Bao, P., and Wang, J. (2021). A Review of 3D Object Detection for Autonomous Driving of Electric Vehicles. World Electr. Veh. J., 12.","DOI":"10.3390\/wevj12030139"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kova\u010di\u0107, K., Ivanjko, E., and Gold, H. (2013). Computer vision systems in road vehicles: A review. arXiv.","DOI":"10.20532\/ccvw.2013.0002"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ilas, C. (2013, January 23\u201325). Electronic sensing technologies for autonomous ground vehicles: A review. Proceedings of the 2013 8th International Symposium on Advanced Topics in Electrical Engineering (Atee), Bucharest, Romania.","DOI":"10.1109\/ATEE.2013.6563528"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1897","DOI":"10.1186\/s40064-016-3573-7","article-title":"Review of visual odometry: Types, approaches, challenges, and applications","volume":"5","author":"Aqel","year":"2016","journal-title":"SpringerPlus"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.vlsi.2017.07.007","article-title":"Algorithm and hardware implementation for visual perception system in autonomous vehicle: A survey","volume":"59","author":"Shi","year":"2017","journal-title":"Integr. VLSI J."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Campbell, S., O\u2019Mahony, N., Krpalcova, L., Riordan, D., Walsh, J., Murphy, A., and Ryan, C. (2018, January 21\u201322). Sensor technology in autonomous vehicles: A review. Proceedings of the 2018 29th Irish Signals and Systems Conference (ISSC), Belfast, UK.","DOI":"10.1109\/ISSC.2018.8585340"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Koci\u0107, J., Jovi\u010di\u0107, N., and Drndarevi\u0107, V. (2018, January 20\u201321). Sensors and sensor fusion in autonomous vehicles. Proceedings of the 2018 26th Telecommunications Forum (TELFOR), Belgrade, Serbia.","DOI":"10.1109\/TELFOR.2018.8612054"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rosique, F., Navarro, P.J., Fern\u00e1ndez, C., and Padilla, A. (2019). A systematic review of perception system and simulators for autonomous vehicles research. Sensors, 19.","DOI":"10.3390\/s19030648"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Haris, M., and Glowacz, A. (2021). Road Object Detection: A Comparative Study of Deep Learning-Based Algorithms. Electronics, 10.","DOI":"10.3390\/electronics10161932"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1049\/iet-ipr.2017.1244","article-title":"Multiple hypothesis tracking algorithm for multi-target multi-camera tracking with disjoint views","volume":"12","author":"Yoon","year":"2018","journal-title":"IET Image Process."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Anguelov, D., Flynn, J., and Kosecka, J. (2017, January 21\u201326). 3d bounding box estimation using deep learning and geometry. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.597"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., and Urtasun, R. (2016, January 27\u201330). Monocular 3d object detection for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.236"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1109\/ACCESS.2019.2962554","article-title":"Multi-sensor fusion in automated driving: A survey","volume":"8","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.patrec.2017.09.038","article-title":"Multimodal vehicle detection: Fusing 3D-LIDAR and color camera data","volume":"115","author":"Asvadi","year":"2018","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, X., Xu, W., Dong, C., and Dolan, J.M. (2017, January 11\u201314). Efficient L-shape fitting for vehicle detection using laser scanners. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995698"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Taipalus, T., and Ahtiainen, J. (2011, January 7\u201311). Human detection and tracking with knee-high mobile 2D LIDAR. Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Thailand.","DOI":"10.1109\/ROBIO.2011.6181529"},{"key":"ref_27","unstructured":"Shao, X., Zhao, H., Nakamura, K., Katabira, K., Shibasaki, R., and Nakagawa, Y. (November, January 29). Detection and tracking of multiple pedestrians by using laser range scanners. Proceedings of the 2007 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2708","DOI":"10.1109\/TITS.2018.2790264","article-title":"Obstacle prediction for automated guided vehicles based on point clouds measured by a tilted LIDAR sensor","volume":"19","author":"Rozsa","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1017\/S0263574711000270","article-title":"Environment perception based on LIDAR sensors for real road applications","volume":"30","author":"Naranjo","year":"2012","journal-title":"Robotica"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Shi, S., Jiang, L., Deng, J., Wang, Z., Guo, C., Shi, J., Wang, X., and Li, H. (2021). PV-RCNN: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection. arXiv.","DOI":"10.1109\/CVPR42600.2020.01054"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2018, January 18\u201323). Voxelnet: End-to-end learning for point cloud based 3d object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, Z., Sun, Y., Liu, S., Shen, X., and Jia, J. (2019, January 15\u201320). Std: Sparse-to-dense 3d object detector for point cloud. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00204"},{"key":"ref_33","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honululu, HI, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201323). Frustum pointnets for 3d object detection from rgb-d data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Han, J., Liao, Y., Zhang, J., Wang, S., and Li, S. (2018). Target Fusion Detection of LiDAR and Camera Based on the Improved YOLO Algorithm. Mathematics, 6.","DOI":"10.3390\/math6100213"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Wang, S., and Urtasun, R. (2018, January 8\u201314). Deep continuous fusion for multi-sensor 3d object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01270-0_39"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"6646","DOI":"10.1016\/j.eswa.2014.04.034","article-title":"Context aided pedestrian detection for danger estimation based on laser scanner and computer vision","volume":"41","author":"Ponz","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1109\/MITS.2016.2620398","article-title":"Sensor fusion methodology for vehicle detection","volume":"9","author":"Garcia","year":"2017","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"R\u00f6vid, A., and Remeli, V. (2019, January 24\u201326). Towards raw sensor fusion in 3D object detection. Proceedings of the 2019 IEEE 17th World Symposium on Applied Machine Intelligence and Informatics (SAMI), Herlany, Slovakia.","DOI":"10.1109\/SAMI.2019.8782779"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Chen, Y., Hu, R., and Urtasun, R. (2019, January 16\u201320). Multi-task multi-sensor fusion for 3d object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00752"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Xu, D., Anguelov, D., and Jain, A. (2018, January 18\u201322). Pointfusion: Deep sensor fusion for 3d bounding box estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00033"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shin, K., Kwon, Y.P., and Tomizuka, M. (2019, January 9\u201312). Roarnet: A robust 3d object detection based on region approximation refinement. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8813895"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.isprsjprs.2019.10.015","article-title":"A Frustum-based probabilistic framework for 3D object detection by fusion of LiDAR and camera data","volume":"159","author":"Gong","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Dou, J., Xue, J., and Fang, J. (2019, January 20\u201324). SEG-VoxelNet for 3D vehicle detection from RGB and LiDAR data. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793492"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Fern\u00e1ndez, C., Izquierdo, R., Llorca, D.F., and Sotelo, M.A. (2014, January 8\u201311). Road curb and lanes detection for autonomous driving on urban scenarios. Proceedings of the 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), Qingdao, China.","DOI":"10.1109\/ITSC.2014.6957993"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCE.2020.2969156","article-title":"Traffic Light Detection in Autonomous Driving Systems","volume":"9","author":"Vitas","year":"2020","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Levinson, J., Askeland, J., Dolson, J., and Thrun, S. (2011, January 9\u201313). Traffic light mapping, localization, and state detection for autonomous vehicles. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5979714"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/S1005-8885(15)60624-0","article-title":"Traffic light detection and recognition for autonomous vehicles","volume":"22","author":"Mu","year":"2015","journal-title":"J. China Univ. Posts Telecommun."},{"key":"ref_49","unstructured":"Redmon, J. (2021, November 07). Darknet: Open Source Neural Networks in C. Available online: https:\/\/pjreddie.com\/darknet\/."},{"key":"ref_50","unstructured":"(2021, November 09). Darknet. Available online: https:\/\/github.com\/pjreddie\/darknet."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? The kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_52","unstructured":"Geiger, A., Lenz, P., Stiller, C., and Urtasun, R. (2021, November 09). The KITTI Vision Benchmark Suite. Available online: Http:\/\/Www.Cvlibs.Net\/Datasets\/Kitti."},{"key":"ref_53","unstructured":"(2021, November 09). Scanning Rangefinder Distance Data Output\/UTM-30LX Product Details|HOKUYO AUTOMATIC CO., LTD. Available online: https:\/\/www.hokuyo-aut.jp\/search\/single.php?serial=169."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Fang, Z., Zhao, S., Wen, S., and Zhang, Y. (2018). A Real-Time 3D Perception and Reconstruction System Based on a 2D Laser Scanner. J. Sens., 2018.","DOI":"10.1155\/2018\/2937694"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Choi, D., Bok, Y., Kim, J., Shim, I., and Kweon, I. (2017). Structure-From-Motion in 3D Space Using 2D Lidars. Sensors, 17.","DOI":"10.3390\/s17020242"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1663\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:23:50Z","timestamp":1760135030000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1663"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,21]]},"references-count":55,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22041663"],"URL":"https:\/\/doi.org\/10.3390\/s22041663","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,2,21]]}}}