{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:50:42Z","timestamp":1760233842865,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T00:00:00Z","timestamp":1613433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Electrical and Mechanical Services Department (EMSD), Hong Kong","award":["DTD\/M&V\/W0084\/S0016\/0523"],"award-info":[{"award-number":["DTD\/M&V\/W0084\/S0016\/0523"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The inspection of electrical and mechanical (E&amp;M) devices using unmanned aerial vehicles (UAVs) has become an increasingly popular choice in the last decade due to their flexibility and mobility. UAVs have the potential to reduce human involvement in visual inspection tasks, which could increase efficiency and reduce risks. This paper presents a UAV system for autonomously performing E&amp;M device inspection. The proposed system relies on learning-based detection for perception, multi-sensor fusion for localization, and path planning for fully autonomous inspection. The perception method utilizes semantic and spatial information generated by a 2-D object detector. The information is then fused with depth measurements for object state estimation. No prior knowledge about the location and category of the target device is needed. The system design is validated by flight experiments using a quadrotor platform. The result shows that the proposed UAV system enables the inspection mission autonomously and ensures a stable and collision-free flight.<\/jats:p>","DOI":"10.3390\/s21041385","type":"journal-article","created":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T22:13:38Z","timestamp":1613513618000},"page":"1385","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Learning-Based Autonomous UAV System for Electrical and Mechanical (E&amp;M) Device Inspection"],"prefix":"10.3390","volume":"21","author":[{"given":"Yurong","family":"Feng","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6260-7006","authenticated-orcid":false,"given":"Kwaiwa","family":"Tse","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1506-0615","authenticated-orcid":false,"given":"Shengyang","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1181-8786","authenticated-orcid":false,"given":"Chih-Yung","family":"Wen","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"},{"name":"Interdisciplinary Division of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6900-0901","authenticated-orcid":false,"given":"Boyang","family":"Li","sequence":"additional","affiliation":[{"name":"Interdisciplinary Division of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon 999077, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1109\/TGRS.2008.2009763","article-title":"Investigation of fish-eye lenses for small-UAV aerial photography","volume":"47","author":"Gurtner","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3390","DOI":"10.3390\/rs4113390","article-title":"Unmanned aerial vehicle (UAV) for monitoring soil erosion in Morocco","volume":"4","author":"Marzolff","year":"2012","journal-title":"Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, Z., Liu, Y., Hayward, R., Zhang, J., and Cai, J. (2008, January 26\u201328). Knowledge-based power line detection for UAV surveillance and inspection systems. Proceedings of the 2008 23rd International Conference Image and Vision Computing New Zealand, Christchurch, New Zealand.","DOI":"10.1109\/IVCNZ.2008.4762118"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14887","DOI":"10.3390\/s150714887","article-title":"Vision and control for UAVs: A survey of general methods and of inexpensive platforms for infrastructure inspection","volume":"15","year":"2015","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sun, J., Li, B., Jiang, Y., and Wen, C.-Y. (2016). A camera-based target detection and positioning UAV system for search and rescue (SAR) purposes. Sensors, 16.","DOI":"10.3390\/s16111778"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"M\u00e1th\u00e9, K., Bu\u015foniu, L., Barab\u00e1s, L., Iuga, C.-I., Miclea, L., and Braband, J. (2016, January 7\u201310). Vision-based control of a quadrotor for an object inspection scenario. Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA.","DOI":"10.1109\/ICUAS.2016.7502522"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Steich, K., Kamel, M., Beardsley, P., Obrist, M.K., Siegwart, R., and Lachat, T. (2016, January 9\u201314). Tree cavity inspection using aerial robots. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759713"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nikolic, J., Burri, M., Rehder, J., Leutenegger, S., Huerzeler, C., and Siegwart, R. (2013, January 2\u20139). A UAV system for inspection of industrial facilities. Proceedings of the 2013 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO.2013.6496959"},{"key":"ref_9","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_12","unstructured":"Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_13","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 24\u201327). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 13\u201316). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5-10). R-fcn: Object detection via region-based fully convolutional networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 22\u201325). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_20","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 19\u201321). Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00913"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, J., and Singh, S. (2014, January 12\u201316). LOAM: Lidar Odometry and Mapping in Real-time. Proceedings of the Robotics: Science and Systems, Berkeley, CA, USA. No. 9.","DOI":"10.15607\/RSS.2014.X.007"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel tracking and mapping for small AR workspaces. Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan.","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Forster, C., Pizzoli, M., and Scaramuzza, D. (June, January 31). SVO: Fast semi-direct monocular visual odometry. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6906584"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A multi-state constraint Kalman filter for vision-aided inertial navigation. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1177\/0278364913481251","article-title":"High-precision, consistent EKF-based visual-inertial odometry","volume":"32","author":"Li","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_30","unstructured":"Chen, S., Wen, C.-Y., Zou, Y., and Chen, W. (2020). Stereo Visual Inertial Pose Estimation Based on Feedforward-Feedback Loops. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","article-title":"A formal basis for the heuristic determination of minimum cost paths","volume":"4","author":"Hart","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Harabor, D.D., and Grastien, A. (2011, January 7\u201311). Online graph pruning for pathfinding on grid maps. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v25i1.7994"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1723","DOI":"10.1007\/s00500-017-2895-x","article-title":"An evolutionary approach to constrained path planning of an autonomous surface vehicle for maximizing the covered area of Ypacarai Lake","volume":"23","author":"Arzamendia","year":"2019","journal-title":"Soft Comput."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Peralta, F., Arzamendia, M., Gregor, D., Reina, D.G., and Toral, S. (2020). A Comparison of Local Path Planning Techniques of Autonomous Surface Vehicles for Monitoring Applications: The Ypacarai Lake Case-study. Sensors, 20.","DOI":"10.3390\/s20051488"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mellinger, D., and Kumar, V. (2011, January 9\u201313). Minimum snap trajectory generation and control for quadrotors. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980409"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Richter, C., Bry, A., and Roy, N. (2016). Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. Robotics Research, Springer.","DOI":"10.1007\/978-3-319-28872-7_37"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tijtgat, N., Van Ranst, W., Goedeme, T., Volckaert, B., and De Turck, F. (2017, January 22\u201329). Embedded real-time object detection for a UAV warning system. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.247"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1493","DOI":"10.1109\/LRA.2019.2895110","article-title":"Optimal trajectory generation for quadrotor teach-and-repeat","volume":"4","author":"Gao","year":"2019","journal-title":"IEEE Robot. Autom. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1385\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:24:58Z","timestamp":1760160298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1385"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,16]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041385"],"URL":"https:\/\/doi.org\/10.3390\/s21041385","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,2,16]]}}}