{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T08:48:19Z","timestamp":1779094099520,"version":"3.51.4"},"reference-count":54,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T00:00:00Z","timestamp":1630281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61725501"],"award-info":[{"award-number":["61725501"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The registration of multi-temporal remote sensing images with abundant information and complex changes is an important preprocessing step for subsequent applications. This paper presents a novel two-stage deep learning registration method based on sub-image matching. Unlike the conventional registration framework, the proposed network learns the mapping between matched sub-images and the geometric transformation parameters directly. In the first stage, the matching of sub-images (MSI), sub-images cropped from the images are matched through the corresponding heatmaps, which are made of the predicted similarity of each sub-image pairs. The second stage, the estimation of transformation parameters (ETP), a network with weight structure and position embedding estimates the global transformation parameters from the matched pairs. The network can deal with an uncertain number of matched sub-image inputs and reduce the impact of outliers. Furthermore, the sample sharing training strategy and the augmentation based on the bounding rectangle are introduced. We evaluated our method by comparing the conventional and deep learning methods qualitatively and quantitatively on Google Earth, ISPRS, and WHU Building Datasets. The experiments showed that our method obtained the probability of correct keypoints (PCK) of over 99% at \u03b1\u00a0= 0.05 (\u03b1: the normalized distance threshold) and achieved a maximum increase of 16.8% at \u03b1\u00a0= 0.01, compared with the latest method. The results demonstrated that our method has good robustness and improved the precision in the registration of optical remote sensing images with great variation.<\/jats:p>","DOI":"10.3390\/rs13173443","type":"journal-article","created":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T22:58:15Z","timestamp":1630450695000},"page":"3443","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Two-Stage Deep Learning Registration Method for Remote Sensing Images Based on Sub-Image Matching"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0289-5808","authenticated-orcid":false,"given":"Yuan","family":"Chen","sequence":"first","affiliation":[{"name":"School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China"},{"name":"Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China"},{"name":"Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1016\/S0262-8856(03)00137-9","article-title":"Image registration methods: A survey","volume":"21","author":"Flusser","year":"2003","journal-title":"Image Vis. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1404","DOI":"10.1088\/0957-0233\/18\/5\/028","article-title":"Development and analysis of a software tool for stitching three-dimensional surface topography data sets","volume":"18","author":"Marinello","year":"2007","journal-title":"Meas. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mulas, M., Ciccarese, G., Truffelli, G., and Corsini, A. (2020). Integration of digital image correlation of Sentinel-2 data and continuous GNSS for long-term slope movements monitoring in moderately rapid landslides. Remote Sens., 12.","DOI":"10.3390\/rs12162605"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/S0262-8856(00)00054-8","article-title":"Mutual information matching in multiresolution contexts","volume":"19","author":"Pluim","year":"2001","journal-title":"Image Vis. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ye, Z., Xu, Y., Chen, H., Zhu, J., Tong, X., and Stilla, U. (2020). Area-based dense image matching with subpixel accuracy for remote sensing applications: Practical analysis and comparative study. Remote Sens., 12.","DOI":"10.3390\/rs12040696"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1117\/1.JRS.13.016518","article-title":"Remote sensing image registration based on local structural information and global constraint","volume":"13","author":"Wu","year":"2019","journal-title":"J. Appl. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1109\/TGRS.2017.2749436","article-title":"A novel image registration method based on phase correlation using low-rank matrix factorization with mixture of gaussian","volume":"56","author":"Dong","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Wang, F., and You, H. (2018). An automatic and novel SAR image registration algorithm: A case study of the chinese GF-3 satellite. Sensors, 18.","DOI":"10.3390\/s18020672"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2589","DOI":"10.1109\/TGRS.2011.2109389","article-title":"Automatic image registration through image segmentation and SIFT","volume":"49","author":"Goncalves","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5283","DOI":"10.1109\/TGRS.2015.2420659","article-title":"Remote sensing image matching based on adaptive binning SIFT descriptor","volume":"53","author":"Sedaghat","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3078","DOI":"10.1109\/TGRS.2018.2790483","article-title":"OS-SIFT: A robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas","volume":"56","author":"Xiang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1742","DOI":"10.1109\/LGRS.2019.2955153","article-title":"Rank-based local self-similarity descriptor for optical-to-SAR image matching","volume":"17","author":"Xiong","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1109\/LGRS.2020.2980620","article-title":"A novel keypoint detector combining corners and blobs for remote sensing image registration","volume":"18","author":"Ye","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/LGRS.2016.2600858","article-title":"Remote sensing image registration with modified SIFT and enhanced feature matching","volume":"14","author":"Ma","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, K., Pan, A.N., Yang, Y., Zhang, S., Ong, S.H., and Tang, H.L. (2017). Remote sensing image registration using multiple image features. Remote Sens., 9.","DOI":"10.20944\/preprints201705.0027.v2"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, X., Li, H., Wang, P., and Jing, L.H. (2020). An image registration method for multisource high-resolution remote sensing images for earthquake disaster assessment. Sensors, 20.","DOI":"10.3390\/s20082286"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.isprsjprs.2019.04.018","article-title":"Illumination-robust remote sensing image matching based on oriented self-similarity","volume":"153","author":"Sedaghat","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liu, S., and Jiang, J. (2019). Registration algorithm based on line-intersection-line for satellite remote sensing images of urban areas. Remote Sens., 11.","DOI":"10.3390\/rs11121400"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lyu, C., and Jiang, J. (2017). Remote sensing image registration with line segments and their intersections. Remote Sens., 9.","DOI":"10.3390\/rs9050439"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, M., Habib, A., He, H.Q., Zhu, Q., and Zhang, W. (2017). Robust feature matching method for SAR and optical images by using gaussian-gamma-shaped bi-windows-based descriptor and geometric constraint. Remote Sens., 9.","DOI":"10.3390\/rs9090882"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1680","DOI":"10.1109\/LGRS.2017.2728604","article-title":"Remote sensing image registration based on multifeature and region division","volume":"14","author":"Ma","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5635","DOI":"10.1080\/01431161.2019.1580793","article-title":"Auto-registration of medium and high spatial resolution satellite images by integrating improved SIFT and spatial consistency constraints","volume":"40","author":"Li","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wu, Y., Di, L., Ming, Y., Lv, H., and Tan, H. (2019). High-resolution optical remote sensing image registration via reweighted random walk based hyper-graph matching. Remote Sens., 11.","DOI":"10.3390\/rs11232841"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/LGRS.2011.2175434","article-title":"RSCJ: Robust sample consensus judging algorithm for remote sensing image registration","volume":"9","author":"Li","year":"2012","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1109\/LGRS.2014.2325970","article-title":"A novel point-matching algorithm based on fast sample consensus for image registration","volume":"12","author":"Wu","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1109\/LGRS.2017.2783879","article-title":"PSOSAC: Particle swarm optimization sample consensus algorithm for remote sensing image registration","volume":"15","author":"Wu","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1142\/S0218001493000339","article-title":"Signature verification using a \u201dSiamese\u201d time delay neural network","volume":"7","author":"Bromley","year":"1993","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tian, Y., Fan, B., and Wu, F. (2017, January 21\u201326). L2-Net: Deep learning of discriminative patch descriptor in Euclidean space. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.649"},{"key":"ref_29","unstructured":"Mishchuk, A., Mishkin, D., Radenovic, F., and Matas, J. (2017, January 4\u20139). Working hard to know your neighbor\u2019s margins: Local descriptor learning loss. Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"38544","DOI":"10.1109\/ACCESS.2018.2853100","article-title":"Multi-temporal remote sensing image registration using deep convolutional features","volume":"6","author":"Yang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_31","unstructured":"Han, X., Leung, T., Jia, Y., Sukthankar, R., and Berg, A.C. (2015, January 7\u201312). MatchNet: Unifying feature and metric learning for patch-based matching. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, H.Q., Chen, M., Chen, T., and Li, D.J. (2018). Matching of remote sensing images with complex background variations via siamese convolutional neural network. Remote Sens., 10.","DOI":"10.3390\/rs10020355"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Hoffmann, S., Brust, C., Shadaydeh, M., and Denzler, J. (August, January 28). Registration of high resolution Sar and optical satellite imagery using fully convolutional networks. Proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2019), Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8898714"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Dong, Y.Y., Jiao, W.L., Long, T.F., Liu, L.F., He, G.J., Gong, C.J., and Guo, Y.T. (2019). Local deep descriptor for remote sensing image feature matching. Remote Sens., 11.","DOI":"10.3390\/rs11040430"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.isprsjprs.2017.12.012","article-title":"A deep learning framework for remote sensing image registration","volume":"145","author":"Wang","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3028","DOI":"10.1109\/JSTARS.2019.2916560","article-title":"Registration of multimodal remote sensing image based on deep fully convolutional neural network","volume":"12","author":"Zhang","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4834","DOI":"10.1109\/TGRS.2019.2893310","article-title":"A novel two-step registration method for remote sensing images based on deep and local features","volume":"57","author":"Ma","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/LGRS.2017.2781741","article-title":"Remote sensing image registration using convolutional neural network features","volume":"15","author":"Ye","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_39","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2021, June 13). Deep Image Homography Estimation. Available online: https:\/\/arxiv.org\/abs\/1606.03798."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhang, J., Wang, C., Liu, S., Jia, L., Ye, N., Wang, J., Zhou, J., and Sun, J. (2020, January 23\u201328). Content-aware unsupervised deep homography estimation. Proceedings of the 16th European Conference on Computer Vision (ECCV 2020), Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_38"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1788","DOI":"10.1109\/TNNLS.2020.3015660","article-title":"Attention-based road registration for GPS-denied UAS navigation","volume":"32","author":"Wang","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Seo, P.H., Lee, J., Jung, D., Han, B., and Cho, M. (2018, January 8\u201314). Attentive semantic alignment with offset-aware correlation kernels. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01225-0_22"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2553","DOI":"10.1109\/TPAMI.2018.2865351","article-title":"Convolutional neural network architecture for geometric matching","volume":"41","author":"Rocco","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Vakalopoulou, M., Christodoulidis, S., Sahasrabudhe, M., Mougiakakou, S., and Paragios, N. (August, January 28). Image registration of satellite imagery with deep convolutional neural networks. Proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2019), Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8898220"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Park, J.H., Nam, W.J., and Lee, S.W. (2020). A two-stream symmetric network with bidirectional ensemble for aerial image matching. Remote Sens., 12.","DOI":"10.3390\/rs12030465"},{"key":"ref_46","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_50","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"135","DOI":"10.5194\/isprsannals-II-3-W4-135-2015","article-title":"ISPRS benchmark for multi-platform photogrammetry","volume":"II-3\/W4","author":"Nex","year":"2015","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TGRS.2018.2858817","article-title":"Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set","volume":"57","author":"Ji","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2878","DOI":"10.1109\/TPAMI.2012.261","article-title":"Articulated human detection with flexible mixtures of parts","volume":"35","author":"Yang","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/17\/3443\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:55:43Z","timestamp":1760165743000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/17\/3443"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,30]]},"references-count":54,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["rs13173443"],"URL":"https:\/\/doi.org\/10.3390\/rs13173443","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,30]]}}}