{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T03:23:22Z","timestamp":1768706602525,"version":"3.49.0"},"reference-count":37,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,1,20]],"date-time":"2022-01-20T00:00:00Z","timestamp":1642636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Shenzhen Special Project for Innovation and Entrepreneurship","award":["JSGG20191129103003903"],"award-info":[{"award-number":["JSGG20191129103003903"]}]},{"name":"the Basic scientific research project of Chinese Academy of Surveying and Mapping","award":["AR2106"],"award-info":[{"award-number":["AR2106"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical\u2013optical, optical\u2013infrared, optical\u2013depth, optical\u2013map, optical\u2013SAR, and day\u2013night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available.<\/jats:p>","DOI":"10.3390\/rs14030478","type":"journal-article","created":{"date-parts":[[2022,1,20]],"date-time":"2022-01-20T22:51:06Z","timestamp":1642719066000},"page":"478","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":44,"title":["3MRS: An Effective Coarse-to-Fine Matching Method for Multimodal Remote Sensing Imagery"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5765-4070","authenticated-orcid":false,"given":"Zhongli","family":"Fan","sequence":"first","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing, Chinese Academy of Surveying and Mapping (CASM), Beijing 100036, China"}]},{"given":"Yuxian","family":"Liu","sequence":"additional","affiliation":[{"name":"Shenzhen Investigation & Research Institute Co., Ltd., Shenzhen 518026, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4394-1989","authenticated-orcid":false,"given":"Yuxuan","family":"Liu","sequence":"additional","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing, Chinese Academy of Surveying and Mapping (CASM), Beijing 100036, China"}]},{"given":"Li","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing, Chinese Academy of Surveying and Mapping (CASM), Beijing 100036, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8086-3401","authenticated-orcid":false,"given":"Junjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Baidu Times Technology (Beijing) Co., Ltd., Beijing 100085, China"}]},{"given":"Yushan","family":"Sun","sequence":"additional","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing, Chinese Academy of Surveying and Mapping (CASM), Beijing 100036, China"}]},{"given":"Haibin","family":"Ai","sequence":"additional","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing, Chinese Academy of Surveying and Mapping (CASM), Beijing 100036, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.isprsjprs.2020.02.005","article-title":"LiDAR-guided dense matching for detecting changes and updating of buildings in Airborne LiDAR data","volume":"162","author":"Zhou","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"De Alban, J.D.T., Connette, G.M., Oswald, P., and Webb, E.L. (2018). Combined Landsat and L-band SAR data improves land cover classification and change detection in dynamic tropical landscapes. Remote Sens., 10.","DOI":"10.3390\/rs10020306"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/LGRS.2018.2868704","article-title":"A conditional adversarial network for change detection in heterogeneous images","volume":"16","author":"Niu","year":"2018","journal-title":"IEEE Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/TIP.2019.2933747","article-title":"Multimodal change detection in remote sensing images using an unsupervised pixel pairwise-based Markov random field model","volume":"29","author":"Touati","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1016\/j.patcog.2018.08.007","article-title":"Multimodal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection","volume":"86","author":"Chen","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1109\/JSTARS.2020.3041316","article-title":"YOLOrs: Object Detection in Multimodal Remote Sensing Imagery","volume":"14","author":"Sharma","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.isprsjprs.2018.04.003","article-title":"Multi-scale object detection in remote sensing imagery with convolutional neural networks","volume":"145","author":"Deng","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, Z., Li, X., and Yeh, A.G.-O. (2019). Integration of convolutional neural networks and object-based post-classification refinement for land use and land cover mapping with optical and SAR data. Remote Sens., 11.","DOI":"10.3390\/rs11060690"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5569","DOI":"10.1109\/JSTARS.2017.2748341","article-title":"Stacked sparse autoencoder modeling using the synergy of airborne LiDAR and satellite optical and SAR data to map forest above-ground biomass","volume":"10","author":"Shao","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_10","first-page":"87","article-title":"Exploring the optimal integration levels between SAR and optical data for better urban land cover mapping in the Pearl River Delta","volume":"64","author":"Zhang","year":"2018","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Campos-Taberner, M., Garc\u00eda-Haro, F.J., Camps-Valls, G., Grau-Muedra, G., Nutini, F., Busetto, L., Katsantonis, D., Stavrakoudis, D., Minakou, C., and Gatti, L. (2017). Exploitation of SAR and optical sentinel data to detect rice crop and estimate seasonal dynamics of leaf area index. Remote Sens., 9.","DOI":"10.3390\/rs9030248"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.isprsjprs.2021.05.011","article-title":"Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model","volume":"178","author":"Hong","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.inffus.2021.02.012","article-title":"A review of multimodal image matching: Methods and applications","volume":"73","author":"Jiang","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4834","DOI":"10.1109\/TGRS.2019.2893310","article-title":"A novel two-step registration method for remote sensing images based on deep and local features","volume":"57","author":"Ma","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhang, H., and Huang, Y. (2021). A Rotation-Invariant Optical and SAR Image Registration Algorithm Based on Deep and Gaussian Features. Remote Sens., 13.","DOI":"10.3390\/rs13132628"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Merkle, N., Luo, W., Auer, S., M\u00fcller, R., and Urtasun, R. (2017). Exploiting deep matching and SAR data for the geo-localization accuracy improvement of optical satellite images. Remote Sens., 9.","DOI":"10.3390\/rs9060586"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.isprsjprs.2020.09.012","article-title":"A deep learning framework for matching of SAR and optical imagery","volume":"169","author":"Hughes","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2829","DOI":"10.1109\/TGRS.2010.2042813","article-title":"Fully automatic subpixel image registration of multiangle CHRIS\/Proba data","volume":"48","author":"Ma","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1495","DOI":"10.1109\/TIP.2003.819237","article-title":"Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient","volume":"12","author":"Johnson","year":"2003","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","unstructured":"Dalal, N., and Triggs, B. (2005, January 25). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2941","DOI":"10.1109\/TGRS.2017.2656380","article-title":"Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity","volume":"55","author":"Ye","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","unstructured":"Kovesi, P. (2003, January 10\u201312). Phase congruency detects corners and edges. Proceedings of the Digital Image Computing: Techniques and Applications 2003, Sydney, Australia."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"9059","DOI":"10.1109\/TGRS.2019.2924684","article-title":"Fast and robust matching for multimodal remote sensing image registration","volume":"57","author":"Ye","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fan, Z., Zhang, L., Liu, Y., Wang, Q., and Zlatanova, S. (2021). Exploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images. Remote Sens., 13.","DOI":"10.3390\/rs13173535"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/LGRS.2016.2600858","article-title":"Remote sensing image registration with modified SIFT and enhanced feature matching","volume":"14","author":"Ma","year":"2016","journal-title":"IEEE Geosci. Remote Sens."},{"key":"ref_27","first-page":"1727","article-title":"Heterologous Images Matching Considering Anisotropic Weighted Moment and Absolute Phase Orientation","volume":"46","author":"Yao","year":"2021","journal-title":"Geomat. Inf. Sci. Wuhan Univ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3296","DOI":"10.1109\/TIP.2019.2959244","article-title":"RIFT: Multimodal image matching based on radiation-variation insensitive feature transform","volume":"29","author":"Li","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1109\/LGRS.2014.2325970","article-title":"A novel point-matching algorithm based on fast sample consensus for image registration","volume":"12","author":"Wu","year":"2014","journal-title":"IEEE Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, Y., Mo, F., and Tao, P. (2017). Matching multi-source optical satellite imagery exploiting a multi-stage approach. Remote Sens., 9.","DOI":"10.3390\/rs9121249"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2233","DOI":"10.1109\/TCSVT.2017.2720175","article-title":"Coarse-to-fine PatchMatch for dense correspondence","volume":"28","author":"Li","year":"2017","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"107029","DOI":"10.1016\/j.patcog.2019.107029","article-title":"Fast and robust template matching with majority neighbour similarity and annulus projection transformation","volume":"98","author":"Lai","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s11263-006-0026-8","article-title":"Self-invertible 2D log-Gabor wavelets","volume":"75","author":"Fischer","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Horn, B., Klaus, B., and Horn, P. (1986). Robot Vision, MIT Press."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/TPAMI.2008.275","article-title":"Faster and better: A machine learning approach to corner detection","volume":"32","author":"Rosten","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6451","DOI":"10.1109\/TGRS.2020.2976865","article-title":"OS-PC: Combining feature representation and 3-D phase correlation for subpixel optical and SAR image registration","volume":"58","author":"Xiang","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.isprsjprs.2021.09.010","article-title":"Robust registration of aerial images and LiDAR data using spatial constraints and Gabor structural features","volume":"181","author":"Zhu","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/3\/478\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:04:33Z","timestamp":1760133873000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/3\/478"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,20]]},"references-count":37,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["rs14030478"],"URL":"https:\/\/doi.org\/10.3390\/rs14030478","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,20]]}}}