{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T06:50:06Z","timestamp":1762325406966,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T00:00:00Z","timestamp":1663113600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Nature Science Foundation of China","award":["62102436","6142217210503","202250E050","202250E060","2021CFB279"],"award-info":[{"award-number":["62102436","6142217210503","202250E050","202250E060","2021CFB279"]}]},{"name":"the National Key Laboratory of Science and Technology","award":["62102436","6142217210503","202250E050","202250E060","2021CFB279"],"award-info":[{"award-number":["62102436","6142217210503","202250E050","202250E060","2021CFB279"]}]},{"name":"the Projects Foundation of University","award":["62102436","6142217210503","202250E050","202250E060","2021CFB279"],"award-info":[{"award-number":["62102436","6142217210503","202250E050","202250E060","2021CFB279"]}]},{"name":"the Hubei Province Natural Science Foundation","award":["62102436","6142217210503","202250E050","202250E060","2021CFB279"],"award-info":[{"award-number":["62102436","6142217210503","202250E050","202250E060","2021CFB279"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Automatic matching of multimodal remote sensing images remains a vital yet challenging task, particularly for remote sensing and computer vision applications. Most traditional methods mainly focus on key point detection and description of the original image, thus ignoring the deep semantic feature information such as semantic road features, with the result that the traditional method can not effectively resist nonlinear grayscale distortion, and has low matching efficiency and poor accuracy. Motivated by this, this paper proposes a novel automatic matching method named LURF via learned unstructured road features for the multimodal images. There are four main contributions in LURF. To begin with, the semantic road features were extracted from multimodal images based on segmentation model CRESIv2. Next, based on semantic road features, a stable and reliable intersection point detector has been proposed to detect unstructured key points. Moreover, a local entropy descriptor has been designed to describe key points with the local skeleton feature. Finally, a global optimization strategy is adopted to achieve the correct matching. The extensive experimental results demonstrate that the proposed LURF outperforms other state-of-the-art methods in terms of both accuracy and efficiency on different multimodal image data sets.<\/jats:p>","DOI":"10.3390\/rs14184595","type":"journal-article","created":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T23:16:36Z","timestamp":1663197396000},"page":"4595","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Automatic Matching of Multimodal Remote Sensing Images via Learned Unstructured Road Feature"],"prefix":"10.3390","volume":"14","author":[{"given":"Kun","family":"Yu","sequence":"first","affiliation":[{"name":"National Key Laboratory of Science and Technology on Vessel Integrated Power System, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengcheng","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Science and Technology on Vessel Integrated Power System, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junfeng","family":"Ding","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinghua","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Science and Technology on Vessel Integrated Power System, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xianqiang","family":"Bao","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Science and Technology on Vessel Integrated Power System, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaohua","family":"Qiu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Science and Technology on Vessel Integrated Power System, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/j.patcog.2019.04.001","article-title":"Feature-guided Gaussian mixture model for image matching","volume":"92","author":"Ma","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_2","first-page":"213","article-title":"A study of a matching pixel by pixel (MPP) algorithm to establish an empirical model of water quality mapping, as based on unmanned aerial vehicle (UAV) images","volume":"58","author":"Su","year":"2017","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.inffus.2018.02.004","article-title":"Infrared and visible image fusion methods and applications: A survey","volume":"45","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.02.001","article-title":"Infrared and visible image fusion via gradient transfer and total variation minimization","volume":"31","author":"Ma","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"813","DOI":"10.14358\/PERS.83.12.813","article-title":"4FP-Structure: A Robust Local Region Feature Descriptor","volume":"83","author":"Li","year":"2017","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4441","DOI":"10.1109\/JSTARS.2021.3073573","article-title":"Multimodal Urban Remote Sensing Image Registration Via Roadcross Triangular Feature","volume":"14","author":"Yu","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4435","DOI":"10.1109\/TGRS.2018.2820040","article-title":"Guided Locality Preserving Feature Matching for Remote Sensing Image Registration","volume":"56","author":"Ma","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s11263-020-01359-2","article-title":"Image Matching from Handcrafted to Deep Features: A Survey","volume":"129","author":"Ma","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.1007\/s11760-014-0621-z","article-title":"Advanced free-form deformation and Kullback\u2013Lieblier divergence measure for digital elevation model registration","volume":"9","author":"Dawn","year":"2015","journal-title":"Signal Image Video Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Burger, W., and Burge, M. (2016). Digital Image Processing, Springer. Texts in Computer Science.","DOI":"10.1007\/978-1-4471-6684-9"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"685","DOI":"10.2307\/2314845","article-title":"The Fourier Transform and its Applications","volume":"73","author":"Boehme","year":"1966","journal-title":"Am. Math. Mon."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4328","DOI":"10.1109\/TGRS.2013.2281391","article-title":"A Novel Coarse-to-Fine Scheme for Automatic Image Registration Based on SIFT and Mutual Information","volume":"52","author":"Gong","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2099","DOI":"10.1109\/TIP.2011.2171696","article-title":"Correlation-Coefficient-Based Fast Template Matching Through Partial Elimination","volume":"21","author":"Mahmood","year":"2012","journal-title":"IEEE Trans. Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1109\/83.506761","article-title":"An FFT-based technique for translation, rotation, and scale-invariant image registration","volume":"5","author":"Reddy","year":"1996","journal-title":"IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc."},{"key":"ref_16","first-page":"5615371","article-title":"To Align Multimodal Lumbar Spine Images via Bending Energy Constrained Normalized Mutual Information","volume":"2020","author":"Wu","year":"2020","journal-title":"BioMed Res. Int."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Liu, G., Chen, S., Zhou, X., Wang, X., Guan, Q., and Yu, H. (2014, January 17\u201319). Combining SIFT and Individual Entropy Correlation Coefficient for Image Registration. Proceedings of the CCPR, Changsha, China.","DOI":"10.1007\/978-3-662-45643-9_14"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4045","DOI":"10.1109\/TIP.2019.2906490","article-title":"LMR: Learning a Two-Class Classifier for Mismatch Removal","volume":"28","author":"Ma","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"LoweDavid, G. (2004). Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vis.","DOI":"10.1023\/B:VISI.0000029664.99615.94"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Gool, L.V. (2006, January 7\u201313). SURF: Speeded Up Robust Features. Proceedings of the ECCV, Graz, Austria.","DOI":"10.1007\/11744023_32"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"9","DOI":"10.5194\/isprs-annals-III-1-9-2016","article-title":"Hopc: A novel similarity metric based on geometric structural properties for multi-modal remote sensing image matching","volume":"3","author":"Ye","year":"2016","journal-title":"ISPRS Ann. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2941","DOI":"10.1109\/TGRS.2017.2656380","article-title":"Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity","volume":"55","author":"Ye","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3296","DOI":"10.1109\/TIP.2019.2959244","article-title":"RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform","volume":"29","author":"Li","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1109\/TBME.2010.2042169","article-title":"A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration","volume":"57","author":"Chen","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1109\/76.927424","article-title":"Color and texture descriptors","volume":"11","author":"Manjunath","year":"2001","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Aguilera-Carrasco, C.A., Sappa, A.D., and Toledo, R. (2015, January 27\u201330). LGHD: A feature descriptor for matching across non-linear intensity variations. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.","DOI":"10.1109\/ICIP.2015.7350783"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.inffus.2021.02.012","article-title":"A review of multimodal image matching: Methods and applications","volume":"73","author":"Jiang","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_28","unstructured":"Han, X., Leung, T., Jia, Y., Sukthankar, R., and Berg, A.C. (2015, January 7\u201312). MatchNet: Unifying feature and metric learning for patch-based matching. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yi, K.M., Trulls, E., Lepetit, V., and Fua, P.V. (2016). LIFT: Learned Invariant Feature Transform. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46466-4_28"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1109\/LGRS.2020.2972361","article-title":"Multispectral Remote Sensing Image Matching via Image Transfer by Regularized Conditional Generative Adversarial Networks and Local Feature","volume":"18","author":"Ma","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Etten, A.V. (2020, January 2\u20135). City-Scale Road Extraction from Satellite Imagery v2: Road Speeds and Travel Times. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093593"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the MICCAI, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_34","unstructured":"Allen-Zhu, Z., Li, Y., and Song, Z. (2019, January 10\u201315). A Convergence Theory for Deep Learning via Over-Parameterization. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_35","unstructured":"Kingma, D.P., and Ba, J. (2015). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/34.659930","article-title":"An Unbiased Detector of Curvilinear Structures","volume":"20","author":"Steger","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1023\/A:1009745219419","article-title":"Density-Based Clustering in Spatial Databases: The Algorithm GDBSCAN and Its Applications","volume":"2","author":"Sander","year":"2004","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1016\/j.ins.2012.07.049","article-title":"Local Shannon entropy measure with statistical tests for image randomness","volume":"222","author":"Wu","year":"2013","journal-title":"Inf. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1007\/s11263-018-1117-z","article-title":"Locality Preserving Matching","volume":"127","author":"Ma","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_40","unstructured":"Etten, A.V., Lindenbaum, D., and Bacastow, T.M. (2018). SpaceNet: A Remote Sensing Dataset and Challenge Series. arXiv."},{"key":"ref_41","first-page":"1","article-title":"Lambertian Model-Based Normal Guided Depth Completion for LiDAR-Camera System","volume":"19","author":"An","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_42","unstructured":"Viswanathan, D. (2009, January 6\u20138). Features from Accelerated Segment Test (FAST). Proceedings of the 10th Workshop on Image Analysis for Multimedia Interactive Services, London, UK."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"111","DOI":"10.5194\/isprs-archives-XLII-4-W4-111-2017","article-title":"Automatic centerline extraction of covered roads by surrounding objects from high resolution satellite images","volume":"42","author":"Kamangir","year":"2017","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4595\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:31:28Z","timestamp":1760142688000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,14]]},"references-count":43,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["rs14184595"],"URL":"https:\/\/doi.org\/10.3390\/rs14184595","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2022,9,14]]}}}