{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T07:23:49Z","timestamp":1763018629867,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T00:00:00Z","timestamp":1684368000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Strategic Priority Research Program of the Chinese Academy of Sciences","award":["XDA19010102","62205083"],"award-info":[{"award-number":["XDA19010102","62205083"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["XDA19010102","62205083"],"award-info":[{"award-number":["XDA19010102","62205083"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Ground Control Points (GCPs) are of great significance for applications involving the registration and fusion of heterologous remote sensing images (RSIs). However, utilizing low-level information rather than deep features, traditional methods based on intensity and local image features turn out to be unsuitable for heterologous RSIs because of the large nonlinear radiation difference (NRD), inconsistent resolutions, and geometric distortions. Additionally, the limitations of current heterologous datasets and existing deep-learning-based methods make it difficult to obtain enough precision GCPs from different kinds of heterologous RSIs, especially for thermal infrared (TIR) images that present low spatial resolution and poor contrast. In this paper, to address the problems above, we propose a convolutional neural network-based (CNN-based) layer-adaptive GCPs extraction method for TIR RSIs. Particularly, the constructed feature extraction network is comprised of basic and layer-adaptive modules. The former is used to achieve the coarse extraction, and the latter is designed to obtain high-accuracy GCPs by adaptively updating the layers in the module to capture the fine communal homogenous features of the heterologous RSIs until the GCP precision meets the preset threshold. Experimental results evaluated on TIR images of SDGSAT-1 TIS and near infrared (NIR), short wave infrared (SWIR), and panchromatic (PAN) images of Landsat-8 OLI show that the matching root-mean-square error (RMSE) of TIS images with SWIR and NIR images could reach 0.8 pixels and an even much higher accuracy of 0.1 pixels could be reached between TIS and PAN images, which performs better than those of the traditional methods, such as SIFT, RIFT, and the CNN-based method like D2-Net.<\/jats:p>","DOI":"10.3390\/rs15102628","type":"journal-article","created":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T06:32:58Z","timestamp":1684391578000},"page":"2628","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["A CNN-Based Layer-Adaptive GCPs Extraction Method for TIR Remote Sensing Images"],"prefix":"10.3390","volume":"15","author":[{"given":"Lixing","family":"Zhao","sequence":"first","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Jingjie","family":"Jiao","sequence":"additional","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Lan","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"}]},{"given":"Wenhao","family":"Pan","sequence":"additional","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Fanjun","family":"Zeng","sequence":"additional","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7003-1644","authenticated-orcid":false,"given":"Xiaoyan","family":"Li","sequence":"additional","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2244-8327","authenticated-orcid":false,"given":"Fansheng","family":"Chen","sequence":"additional","affiliation":[{"name":"Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China"},{"name":"State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Shanghai Frontier Base of Intelligent Optoelectronics and Perception, Institute of Optoelectronics, Fudan University, Shanghai 200433, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"38043","DOI":"10.1364\/OE.470476","article-title":"Extrapolating distortion correction with local measurements for space-based multi-module splicing large-format infrared cameras","volume":"30","author":"Jiang","year":"2022","journal-title":"Opt. Express"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"7507705","DOI":"10.1109\/LGRS.2022.3201073","article-title":"Resolution-Normalizing Image Stitching for Long-Linear-Array and Wide-Swath Whiskbroom Payloads","volume":"19","author":"Yang","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_3","first-page":"7000205","article-title":"GCPs Extraction with Geometric Texture Pattern for Thermal Infrared Remote Sensing Images","volume":"19","author":"Li","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1109\/42.563664","article-title":"Multimodality image registration by maximization of mutual information","volume":"16","author":"Maes","year":"1997","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1495","DOI":"10.1109\/TIP.2003.819237","article-title":"Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient","volume":"12","author":"Johnson","year":"2003","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random Sample Consensus\u2014A Paradigm for Model-Fitting with Applications to Image-Analysis and Automated Cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_8","unstructured":"Harris, C., and Stephens, M. (1988). Alvey Vision Conference, Alvety Vision Club."},{"key":"ref_9","unstructured":"Moravec, H.P. (1980). Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover. [Ph.D. Dissertation, Stanford University]."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Rosten, E., and Drummond, T. (2005, January 17\u201320). Fusing points and lines for high performance tracking. Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China.","DOI":"10.1109\/ICCV.2005.104"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1023\/A:1007963824710","article-title":"SUSAN\u2014A new approach to low level image processing","volume":"23","author":"Smith","year":"1997","journal-title":"Int. J. Comput. Vis."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"024507","DOI":"10.1117\/1.JRS.14.024507","article-title":"On-orbit installation matrix calibration and its application on AGRI of FY-4A","volume":"14","author":"Chen","year":"2020","journal-title":"J. Appl. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7\u201313). SURF: Speeded up robust features. Proceedings of the 9th European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_32"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"11","DOI":"10.5201\/ipol.2011.my-asift","article-title":"ASIFT: An Algorithm for Fully Affine Invariant Comparison","volume":"1","author":"Yu","year":"2011","journal-title":"Image Process. Line"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/LGRS.2016.2600858","article-title":"Remote Sensing Image Registration with Modified SIFT and Enhanced Feature Matching","volume":"14","author":"Ma","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ye, Y., and Shen, L. (2016, January 12\u201319). HOPC: A Novel Similarity Metric Based on Geometric Structural Properties for Multi-Modal Remote Sensing Image Matching. Proceedings of the 23rd ISPRS Congress, Prague, Czech Republic.","DOI":"10.5194\/isprsannals-III-1-9-2016"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3296","DOI":"10.1109\/TIP.2019.2959244","article-title":"RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform","volume":"29","author":"Li","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","unstructured":"Han, X.F., Leung, T., Jia, Y.Q., Sukthankar, R., and Berg, A.C. (2015, January 7\u201312). MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., and Sattler, T. (2019, January 16\u201320). D2-Net: A Trainable CNN for Joint Description and Detection of Local Features. Proceedings of the 32nd IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00828"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201322). SuperPoint: Self-Supervised Interest Point Detection and Description. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2015, January 7\u201312). Learning to Compare Image Patches via Convolutional Neural Networks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299064"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet Classification with Deep Convolutional Neural Networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"38544","DOI":"10.1109\/ACCESS.2018.2853100","article-title":"Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features","volume":"6","author":"Yang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4834","DOI":"10.1109\/TGRS.2019.2893310","article-title":"A Novel Two-Step Registration Method for Remote Sensing Images Based on Deep and Local Features","volume":"57","author":"Ma","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/LGRS.2017.2781741","article-title":"Remote Sensing Image Registration Using Convolutional Neural Network Features","volume":"15","author":"Ye","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2853","DOI":"10.1109\/TNNLS.2018.2888757","article-title":"A Novel Neural Network for Remote Sensing Image Matching","volume":"30","author":"Zhu","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1109\/LGRS.2018.2799232","article-title":"Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN","volume":"15","author":"Hughes","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3028","DOI":"10.1109\/JSTARS.2019.2916560","article-title":"Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network","volume":"12","author":"Zhang","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_29","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Li, Z.Q., and Snavely, N. (2018, January 18\u201323). MegaDepth: Learning single-view depth prediction from internet photos. Proceedings of the 31st IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00218"},{"key":"ref_32","unstructured":"Mishchuk, A., Mishkin, D., Radenovic, F., and Matas, J. (2017, January 4\u20139). Working hard to know your neighbor\u2019s margins: Local descriptor learning loss. Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Schonberger, J.L., and Frahm, J.M. (2016, January 27\u201330). Structure-from-Motion Revisited. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR.2016.445"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/10\/2628\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:37:39Z","timestamp":1760125059000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/10\/2628"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,18]]},"references-count":33,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["rs15102628"],"URL":"https:\/\/doi.org\/10.3390\/rs15102628","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,5,18]]}}}