{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T05:29:41Z","timestamp":1761629381465,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2018R1D1A3B07041729"],"award-info":[{"award-number":["2018R1D1A3B07041729"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002560","name":"Soonchunhyang University","doi-asserted-by":"publisher","award":["NA"],"award-info":[{"award-number":["NA"]}],"id":[{"id":"10.13039\/501100002560","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>An essential component for the autonomous flight or air-to-ground surveillance of a UAV is an object detection device. It must possess a high detection accuracy and requires real-time data processing to be employed for various tasks such as search and rescue, object tracking and disaster analysis. With the recent advancements in multimodal data-based object detection architectures, autonomous driving technology has significantly improved, and the latest algorithm has achieved an average precision of up to 96%. However, these remarkable advances may be unsuitable for the image processing of UAV aerial data directly onboard for object detection because of the following major problems: (1) Objects in aerial views generally have a smaller size than in an image and they are uneven and sparsely distributed throughout an image; (2) Objects are exposed to various environmental changes, such as occlusion and background interference; and (3) The payload weight of a UAV is limited. Thus, we propose employing a new real-time onboard object detection architecture, an RGB aerial image and a point cloud data (PCD) depth map image network (RGDiNet). A faster region-based convolutional neural network was used as the baseline detection network and an RGD, an integration of the RGB aerial image and the depth map reconstructed by the light detection and ranging PCD, was utilized as an input for computational efficiency. Performance tests and evaluation of the proposed RGDiNet were conducted under various operating conditions using hand-labeled aerial datasets. Consequently, it was shown that the proposed method has a superior performance for the detection of vehicles and pedestrians than conventional vision-based methods.<\/jats:p>","DOI":"10.3390\/s21051677","type":"journal-article","created":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T10:25:18Z","timestamp":1614594318000},"page":"1677","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["RGDiNet: Efficient Onboard Object Detection with Faster R-CNN for Air-to-Ground Surveillance"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8196-1089","authenticated-orcid":false,"given":"Jongwon","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Soonchunhyang University, Asan 31538, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5162-1745","authenticated-orcid":false,"given":"Jeongho","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Soonchunhyang University, Asan 31538, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3597","DOI":"10.1109\/TCYB.2016.2572609","article-title":"Statistical hypothesis detector for abnormal event detection in crowded scenes","volume":"47","author":"Yuan","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.trd.2017.02.017","article-title":"Delivery by drone: An evaluation of unmanned aerial vehicle technology in reducing CO2 emissions in the delivery service industry","volume":"61","author":"Goodchild","year":"2018","journal-title":"Transp. Res. Part D Transp. Environ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1016\/j.procs.2018.07.063","article-title":"Review on application of drone systems in precision agriculture","volume":"133","author":"Mogili","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Besada, J.A., Bergesio, L., Campa\u00f1a, I., Vaquero-Melchor, D., L\u00f3pez-Araquistain, J., Bernardos, A.M., and Casar, J.R. (2018). Drone mission definition and implementation for automated infrastructure inspection using airborne sensors. Sensors, 18.","DOI":"10.3390\/s18041170"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhu, P., Du, D., Wen, L., Bian, X., Ling, H., Hu, Q., and Liu, Z. (2019, January 27\u201328). VisDrone-VID2019: The vision meets drone object detection in video challenge results. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00031"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Law, H., and Deng, J. (2018, January 8\u201314). Cornernet: Detecting objects as paired keypoints. Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_45"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., and Urtasun, R. (2016, January 27\u201330). Monocular 3d object detection for autonomous driving. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.236"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Choi, W., Lin, Y., and Savarese, S. (2015, January 7\u201312). Data-driven 3d voxel patterns for object category recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298800"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Kaida, S., Kiawjak, P., and Matsushima, K. (2020, January 15\u201318). Behavior prediction using 3d box estimation in road environment. Proceedings of the 2020 International Conference on Computer and Communication Systems, Shanghai, China.","DOI":"10.1109\/ICCCS49078.2020.9118531"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.patrec.2017.09.038","article-title":"Multimodal vehicle detection: Fusing 3D-LIDAR and color camera data","volume":"115","author":"Asvadi","year":"2018","journal-title":"Pattern Recognit. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1177\/0278364917696568","article-title":"Robust LIDAR localization using multiresolution Gaussian mixture maps for autonomous driving","volume":"36","author":"Wolcott","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.patcog.2017.07.026","article-title":"Multi-modal deep feature learning for RGB-D object detection","volume":"72","author":"Xu","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhao, J.X., Cao, Y., Fan, D.P., Cheng, M.M., Li, X.Y., and Zhang, L. (2019, January 15\u201321). Contrast prior and fluid pyramid integration for RGBD salient object detection. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00405"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, B., Luo, W., and Urtasun, R. (2018, January 18\u201323). Pixor: Real-time 3d object detection from point clouds. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00798"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Su, H., Maji, S., Kalogerakis, E., and Learned-Miller, E. (2015, January 7\u201313). Multi-view convolutional neural networks for 3d shape recognition. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.114"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Su, H., Nie\u00dfner, M., Dai, A., Yan, M., and Guibas, L.J. (2016, January 27\u201330). Volumetric and multi-view cnns for object classification on 3d data. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.609"},{"key":"ref_23","unstructured":"Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. (2015, January 7\u201312). 3d shapenets: A deep representation for volumetric shapes. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Anguelov, D., Flynn, J., and Kosecka, J. (2017, January 21\u201326). 3d bounding box estimation using deep learning and geometry. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.597"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Wang, S., and Urtasun, R. (2018, January 8\u201314). Deep continuous fusion for multi-sensor 3d object detection. Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01270-0_39"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? The kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/LGRS.2016.2542358","article-title":"Convolutional neural network based automatic object detection on aerial images","volume":"13","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hsieh, M.R., Lin, Y.L., and Hsu, W.H. (2017, January 22\u201329). Drone-based object counting by spatially regularized regional proposal network. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.446"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mo, N., and Yan, L. (2020). Improved faster RCNN based on feature amplification and oversampling data augmentation for oriented vehicle detection in aerial images. Remote Sens., 12.","DOI":"10.3390\/rs12162558"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sommer, L.W., Schuchert, T., and Beyerer, J. (2017, January 24\u201331). Fast deep vehicle detection in aerial images. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.41"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., and Zhang, L. (2018, January 18\u201323). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00418"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"98","DOI":"10.4236\/jcc.2018.611009","article-title":"A vehicle detection method for aerial image based on YOLO","volume":"6","author":"Lu","year":"2018","journal-title":"J. Comput. Commun."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liu, M., Wang, X., Zhou, A., Fu, X., Ma, Y., and Piao, C. (2020). UAV-YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective. Sensors, 20.","DOI":"10.3390\/s20082238"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"864","DOI":"10.1109\/LGRS.2018.2888887","article-title":"Scale adaptive proposal network for object detection in remote sensing images","volume":"16","author":"Zhang","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gao, M., Yu, R., Li, A., Morariu, V.I., and Davis, L.S. (2018, January 18\u201323). Dynamic zoom-in network for fast object detection in large images. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00724"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ma, Y., Wu, X., Yu, G., Xu, Y., and Wang, Y. (2016). Pedestrian detection and tracking from low-resolution unmanned aerial vehicle thermal imagery. Sensors, 16.","DOI":"10.3390\/s16040446"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1016\/j.infrared.2018.06.023","article-title":"Moving object detection in aerial infrared images with registration accuracy prediction and feature points selection","volume":"92","author":"Xu","year":"2018","journal-title":"Infrared Phys. Technol."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Carrio, A., Vemprala, S., Ripoll, A., Saripalli, S., and Campoy, P. (2018, January 1\u20135). Drone detection using depth maps. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593405"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1016\/j.patcog.2018.08.007","article-title":"Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection","volume":"86","author":"Chen","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chang, L., Niu, X., Liu, T., Tang, J., and Qian, C. (2019). GNSS\/INS\/LiDAR-SLAM integrated navigation system based on graph optimization. Remote Sens., 11.","DOI":"10.3390\/rs11091009"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2274","DOI":"10.1109\/TIP.2017.2682981","article-title":"RGBD salient object detection via deep fusion","volume":"26","author":"Qu","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2491","DOI":"10.1364\/JOSAA.10.002491","article-title":"Spectral sensitivities of the human cones","volume":"10","author":"Stockman","year":"1993","journal-title":"JOSA A"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1002\/ima.20110","article-title":"Study on color space selection for detecting cast shadows in video surveillance","volume":"17","author":"Benedek","year":"2007","journal-title":"Int. J. Imaging Syst. Technol."},{"key":"ref_44","unstructured":"Rasouli, A., and Tsotsos, J.K. (2017). The effect of color space selection on detectability and discriminability of colored objects. arXiv."},{"key":"ref_45","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_46","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zhang, L., Lin, L., Liang, X., and He, K. (2016, January 11\u201314). Is faster R-CNN doing well for pedestrian detection?. Proceedings of the 2016 European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_28"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast r-cnn. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Douillard, B., Underwood, J., Kuntz, N., Vlaskine, V., Quadros, A., Morton, P., and Frenkel, A. (2011, January 9\u201313). On the segmentation of 3D LIDAR point clouds. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5979818"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Himmelsbach, M., Luettel, T., and Wuensche, H.J. (2009, January 11\u201315). Real-time object classification in 3D point clouds using point feature histograms. Proceedings of the 2009 IEEE\/RSJ International Conference on Intelligent Robots and Systems, St Louis, MO, USA.","DOI":"10.1109\/IROS.2009.5354493"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., and Zitnick, C.L. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the 2014 European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/5\/1677\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:30:48Z","timestamp":1760160648000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/5\/1677"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,1]]},"references-count":52,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21051677"],"URL":"https:\/\/doi.org\/10.3390\/s21051677","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,3,1]]}}}