{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:07:48Z","timestamp":1760231268483,"version":"build-2065373602"},"reference-count":46,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,7]],"date-time":"2022-09-07T00:00:00Z","timestamp":1662508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Shaanxi Province Key Research and Development Program","award":["2021ZDLGY02-01","XWYCXY-012021003","B12026"],"award-info":[{"award-number":["2021ZDLGY02-01","XWYCXY-012021003","B12026"]}]},{"name":"Wuhu-Xidian University Industry-University-Research Cooperation Special Fund","award":["2021ZDLGY02-01","XWYCXY-012021003","B12026"],"award-info":[{"award-number":["2021ZDLGY02-01","XWYCXY-012021003","B12026"]}]},{"name":"National 111 Center","award":["2021ZDLGY02-01","XWYCXY-012021003","B12026"],"award-info":[{"award-number":["2021ZDLGY02-01","XWYCXY-012021003","B12026"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Vision-based unmanned aerial vehicle (UAV) localization is capable of providing real-time coordinates independently during GNSS interruption, which is important in security, agriculture, industrial mapping, and other fields. owever, there are problems with shadows, the tiny size of targets, interfering objects, and motion blurred edges in aerial images captured by UAVs. Therefore, a multi-order Siamese region proposal network (M-O SiamRPN) with weight adaptive joint multiple intersection over union (MIoU) loss function is proposed to overcome the above limitations. The normalized covariance of 2-O information based on1-O features is introduced in the Siamese convolutional neural network to improve the representation and sensitivity of the network to edges. We innovatively propose a spatial continuity criterion to select 1-O features with richer local details for the calculation of 2-O information, to ensure the effectiveness of M-O features. To reduce the effect of unavoidable positive and negative sample imbalance in target detection, weight adaptive coefficients were designed to automatically modify the penalty factor of cross-entropy loss. Moreover, the MIoU was constructed to constrain the anchor box regression from multiple perspectives. In addition, we proposed an improved Wallis shadow automatic compensation method to pre-process aerial images, providing the basis for subsequent image matching procedures. We also built a consumer-grade UAV acquisition platform to construct an aerial image dataset for experimental validation. The results show that our framework achieved excellent performance for each quantitative and qualitative metric, with the highest precision being 0.979 and a success rate of 0.732.<\/jats:p>","DOI":"10.3390\/rs14184467","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"4467","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["M-O SiamRPN with Weight Adaptive Joint MIoU for UAV Visual Localization"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3373-2561","authenticated-orcid":false,"given":"Kailin","family":"Wen","sequence":"first","affiliation":[{"name":"School of Microelectronics, Xidian University, Xi\u2019an 710071, China"},{"name":"Suzhou Honghu Qiji Electronic Technology Co., Ltd., Suzhou 215008, China"}]},{"given":"Jie","family":"Chu","sequence":"additional","affiliation":[{"name":"School of Microelectronics, Xidian University, Xi\u2019an 710071, China"}]},{"given":"Jiayan","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Microelectronics, Xidian University, Xi\u2019an 710071, China"}]},{"given":"Yu","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Microelectronics, Xidian University, Xi\u2019an 710071, China"},{"name":"The 54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"}]},{"given":"Jueping","family":"Cai","sequence":"additional","affiliation":[{"name":"School of Microelectronics, Xidian University, Xi\u2019an 710071, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9651","DOI":"10.1109\/TIE.2019.2899548","article-title":"Simultaneously detecting and counting dense vehicles from drone images","volume":"66","author":"Li","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ye, Z., Wei, J., Lin, Y., Guo, Q., Zhang, J., Zhang, H., Deng, H., and Yang, K. (2022). Extraction of Olive Crown Based on UAV Visible Images and the U2-Net Deep Learning Model. Remote Sens., 14.","DOI":"10.3390\/rs14061523"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Workman, S., Souvenir, R., and Jacobs, N. (2015, January 7\u201313). Wide-Area Image Geolocalization with Aerial Reference Imagery. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.451"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1109\/TAES.2021.3054067","article-title":"Tightly Coupled Inertial Navigation System with Signals of Opportunity Aiding","volume":"57","author":"Morales","year":"2021","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, F., Shan, B., Wang, Y., Hu, Y., and Teng, H. (2018, January 10\u201312). MIMU\/GPS Integrated Navigation Filtering Algorithm under the Condition of Satellite Missing. Proceedings of the IEEE CSAA Guidance, Navigation and Control Conference (GNCC), Xiamen, China.","DOI":"10.1109\/GNCC42960.2018.9019143"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8974","DOI":"10.1109\/JSEN.2022.3155166","article-title":"Deep Learning-Enabled Fusion to Bridge GPS Outages for INS\/GPS Integrated Navigation","volume":"22","author":"Liu","year":"2022","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6557","DOI":"10.1109\/TVT.2019.2914477","article-title":"Covert spoofing algorithm of UAV based on GPS\/INS-integrated navigation","volume":"68","author":"Guo","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wortsman, M., Ehsani, K., Rastegari, M., Farhadi, A., and Mottaghi, R. (2019, January 15\u201320). Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00691"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/LGRS.2021.3099166","article-title":"Robust Visual-Lidar Simultaneous Localization and Mapping System for UAV","volume":"19","author":"Qian","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zheng, Z., Wei, Y., and Yang, Y. (2020). University-1652: A multi-view multi-source benchmark for drone-based geo-localization. arXiv.","DOI":"10.1145\/3394171.3413896"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, Y., Tao, J., Kong, D., Zhang, Y., and Li, P. (2022). A Visual Compass Based on Point and Line Features for UAV High-Altitude Orientation Estimation. Remote Sens., 14.","DOI":"10.3390\/rs14061430"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1002\/rob.21585","article-title":"Air-ground matching: Appearance-based gps-denied urban localization of micro aerial vehicles","volume":"32","author":"Majdik","year":"2015","journal-title":"J. Field Robot."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chang, K., and Yan, L. (2022). LLNet: A Fusion Classification Network for Land Localization in Real-World Scenarios. Remote Sens., 14.","DOI":"10.3390\/rs14081876"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhai, R., and Yuan, Y. (2022). A Method of Vision Aided GNSS Positioning Using Semantic Information in Complex Urban Environment. Remote Sens., 14.","DOI":"10.3390\/rs14040869"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Nassar, A., Amer, K., ElHakim, R., and ElHelw, M. (2018, January 18\u201322). A Deep CNN-Based Framework for Enhanced Aerial Imagery Registration with Applications to UAV Geolocalization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00201"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ahn, S., Kang, H., and Lee, J. (2021, January 12\u201315). Aerial-Satellite Image Matching Framework for UAV Absolute Visual Localization using Contrastive Learning. Proceedings of the International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea.","DOI":"10.23919\/ICCAS52745.2021.9649999"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1109\/LRA.2020.3048668","article-title":"Reinforcement learning based visual navigation with information-theoretic regularization","volume":"6","author":"Wu","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cen, M., and Jung, C. (2018, January 7\u201310). Fully Convolutional Siamese Fusion Networks for Object Tracking. Proceedings of the IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451102"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., and Hu, W. (2018, January 8\u201314). Distractor-aware Siamese Networks for Visual Object Tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01240-3_7"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"He, A., Luo, C., Tian, X., and Zeng, W. (2018, January 18\u201323). A Twofold Siamese Network for Real-Time Object Tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00508"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Laurens, V., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18\u201323). High Performance Visual Tracking with Siamese Region Proposal Network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00935"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiang, H., Chen, A., Wu, Y., Zhang, C., Chi, Z., Li, M., and Wang, X. (2022). Vegetation Monitoring for Mountainous Regions Using a New Integrated Topographic Correction (ITC) of the SCS + C Correction and the Shadow-Eliminated Vegetation Index. Remote Sens., 14.","DOI":"10.3390\/rs14133073"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, J., Huang, B., Li, J., Wang, Y., Ren, M., and Xu, T. (2022). Learning Spatio-Temporal Attention Based Siamese Network for Tracking UAVs in theWild. Remote Sens., 14.","DOI":"10.3390\/rs14081797"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.neucom.2016.12.002","article-title":"Exploiting the complementary strengths of multi-layer CNN features for image retrieval","volume":"237","author":"Yu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"107339","DOI":"10.1016\/j.patcog.2020.107339","article-title":"Robust Visual Tracking by Embedding Combination and Weighted-Gradient Optimization","volume":"104","author":"Feng","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep learning in remote sensing: A comprehensive review and list of resources","volume":"5","author":"Zhu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","first-page":"1299","article-title":"Automatic Shadow Detection and Compensation of Aerial Remote Sensing Images","volume":"37","author":"Gao","year":"2012","journal-title":"Geomat. Inf. Sci. Wuhan Univ."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhou, T., Fu, H., Sun, C., and Wang, S. (2021). Shadow Detection and Compensation from Remote Sensing Images under Complex Urban Conditions. Remote Sens., 13.","DOI":"10.3390\/rs13040699"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s10107-003-0490-7","article-title":"Least frobenius norm updating of quadratic models that satisfy interpolation conditions","volume":"100","author":"Powell","year":"2004","journal-title":"Math Program."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27\u201330). Image Style Transfer Using Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.265"},{"key":"ref_35","first-page":"113225","article-title":"Deep covariance estimation hashing","volume":"7","author":"Wu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.patcog.2018.10.010","article-title":"Gaussian process approach for metric learning","volume":"87","author":"Li","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_37","first-page":"63","article-title":"The matrix Sign function and computations in systems","volume":"2","author":"Denman","year":"1976","journal-title":"Appl. Math. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Fan, Q., Zhuo, W., Tang, C.K., and Tai, Y.W. (2020, January 13\u201319). Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00407"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1109\/TSM.2020.2994357","article-title":"A Deep Convolutional Neural Network for Wafer Defect Identification on an Imbalanced Dataset in Semiconductor Manufacturing Processes","volume":"33","author":"Saqlain","year":"2020","journal-title":"IEEE Trans. Semicond. Manuf."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rezatofighi, H., Tsoi, N., Gwak, J.Y., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15\u201320). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (ICCV), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00075"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., and Ren, D. (2019). Distance-iou loss: Faster and better learning for bounding box regression. arXiv.","DOI":"10.1609\/aaai.v34i07.6999"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 15\u201320). SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Shen, Z., Dai, Y., and Rao, Z. (2021, January 20\u201325). CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01369"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Huang, L., Zhao, X., and Huang, K. (2019). GlobalTrack: A Simple and Strong Baseline for Long-term Tracking. arXiv.","DOI":"10.1609\/aaai.v34i07.6758"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, P., Xie, J., Wang, Q., and Zuo, W. (2017, January 22\u201329). Is Second-Order Information Helpful for Large-Scale Visual Recognition?. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.228"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TPAMI.2017.2723400","article-title":"Bilinear Convolutional Neural Networks for Fine-Grained Visual Recognition","volume":"40","author":"Lin","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4467\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:25:20Z","timestamp":1760142320000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4467"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,7]]},"references-count":46,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["rs14184467"],"URL":"https:\/\/doi.org\/10.3390\/rs14184467","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2022,9,7]]}}}