{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T12:08:12Z","timestamp":1762085292505,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T00:00:00Z","timestamp":1675814400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62176242","62176096","61906177","42001413"],"award-info":[{"award-number":["62176242","62176096","61906177","42001413"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Removing incorrect keypoint correspondences between two images is a fundamental yet challenging task in computer vision. A popular pipeline first computes a feature vector for each correspondence and then trains a binary classifier using these features. In this paper, we propose a novel robust feature to better fulfill the above task. The basic observation is that the relative order of neighboring points around a correct match should be consistent from one view to another, while it may change a lot for an incorrect match. To this end, the feature is designed to measure the bidirectional relative ranking difference for the neighbors of a reference correspondence. To reduce the negative effect of incorrect correspondences in the neighborhood when computing the feature, we propose to combine spatially nearest neighbors with geometrically \u201cgood\u201d neighbors. We also design an iterative neighbor weighting strategy, which considers both goodness and correctness of a correspondence, to enhance correct correspondences and suppress incorrect correspondences. As the relative order of neighbors encodes structure information between them, we name the proposed feature the Mutual Structure Shift Feature (MSSF). Finally, we use the proposed features to train a random forest classifier in a supervised manner. Extensive experiments on both raw matching quality and downstream tasks are conducted to verify the performance of the proposed method.<\/jats:p>","DOI":"10.3390\/rs15040926","type":"journal-article","created":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T05:37:31Z","timestamp":1675834651000},"page":"926","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["MSSF: A Novel Mutual Structure Shift Feature for Removing Incorrect Keypoint Correspondences between Images"],"prefix":"10.3390","volume":"15","author":[{"given":"Juan","family":"Liu","sequence":"first","affiliation":[{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, School of Computer Sciences, China University of Geosciences, Wuhan 430074, China"},{"name":"Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9503-3969","authenticated-orcid":false,"given":"Kun","family":"Sun","sequence":"additional","affiliation":[{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, School of Computer Sciences, China University of Geosciences, Wuhan 430074, China"},{"name":"Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7799-650X","authenticated-orcid":false,"given":"San","family":"Jiang","sequence":"additional","affiliation":[{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, School of Computer Sciences, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kunqian","family":"Li","sequence":"additional","affiliation":[{"name":"College of Engineering, Ocean University of China, Qingdao 266100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenbing","family":"Tao","sequence":"additional","affiliation":[{"name":"Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education, Wuhan 430074, China"},{"name":"National Key Laboratory of Science and Technology on Multi-Spectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s11263-020-01359-2","article-title":"Image Matching from Handcrafted to Deep Features: A Survey","volume":"129","author":"Ma","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107345","DOI":"10.1016\/j.patcog.2020.107345","article-title":"Point set registration with mixture framework and variational inference","volume":"104","author":"Ma","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2784","DOI":"10.1109\/TFUZZ.2020.2974433","article-title":"Adaptive Hierarchical Probabilistic Model Using Structured Variational Inference for Point Set Registration","volume":"28","author":"He","year":"2020","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, T., Jiang, Z., and Yan, J. (2020, January 7\u201312). Clustering-aware Multiple Graph Matching via Decayed Pairwise Matching Composition. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i02.5528"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, R., Yan, J., and Yang, X. (2020). Combinatorial Learning of Robust Deep Graph Matching: An Embedding based Approach. IEEE Trans. Pattern Anal. Mach. Intell., early access.","DOI":"10.1109\/TPAMI.2020.3005590"},{"key":"ref_6","unstructured":"Min, J., Lee, J., Ponce, J., and Cho, M. (November, January 27). Hyperpixel flow: Semantic correspondence with multi-layer neural features. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2121","DOI":"10.1109\/JAS.2022.106082","article-title":"SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness","volume":"9","author":"Tang","year":"2022","journal-title":"IEEE CAA J. Autom. Sin."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_9","first-page":"404","article-title":"SURF: Speeded Up Robust Features","volume":"Volume 3951","author":"Bay","year":"2006","journal-title":"Proceedings of the European Conference on Computer Vision"},{"key":"ref_10","first-page":"778","article-title":"BRIEF: Binary Robust Independent Elementary Features","volume":"Volume 6314","author":"Calonder","year":"2010","journal-title":"Proceedings of the European Conference on Computer Vision"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Leutenegger, S., Chli, M., and Siegwart, R. (2011, January 20\u201325). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision, Washington, DC, USA.","DOI":"10.1109\/ICCV.2011.6126542"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Tian, Y., Fan, B., and Wu, F. (2017, January 21\u201326). L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.649"},{"key":"ref_13","unstructured":"Mishchuk, A., Mishkin, D., Radenovic, F., and Matas, J. (2017, January 4\u20139). Working hard to know your neighbor\u2019s margins: Local descriptor learning loss. Proceedings of the Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., and Sattler, T. (2019, January 15\u201320). D2-Net: A Trainable CNN for Joint Description and Detection of Local Features. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Computer Vision Foundation, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00828"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tian, Y., Yu, X., Fan, B., Wu, F., Heijnen, H., and Balntas, V. (2019, January 16\u201320). SOSNet: Second Order Similarity Regularization for Local Descriptor Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01127"},{"key":"ref_16","first-page":"757","article-title":"Learning Feature Descriptors Using Camera Pose Supervision","volume":"Volume 12346","author":"Wang","year":"2020","journal-title":"Proceedings of the ECCV"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ranftl, R., and Koltun, V. (2018, January 8\u201314). Deep fundamental matrix estimation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_18"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3519","DOI":"10.1016\/j.patcog.2013.05.017","article-title":"Regularized vector field learning with sparse approximation for mismatch removal","volume":"46","author":"Ma","year":"2013","journal-title":"Pattern Recognit."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1706","DOI":"10.1109\/TIP.2014.2307478","article-title":"Robust point matching via vector field consensus","volume":"23","author":"Ma","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3584","DOI":"10.1109\/TNNLS.2018.2872528","article-title":"Nonrigid point set registration with robust transformation learning under manifold regularization","volume":"30","author":"Ma","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, H., and Yan, S. (2010, January 13\u201318). Common visual pattern discovery via spatially coherent correspondences. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539780"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2602142","article-title":"Feature matching with bounded distortion","volume":"33","author":"Lipman","year":"2014","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1007\/s11263-018-1117-z","article-title":"Locality preserving matching","volume":"127","author":"Ma","year":"2019","journal-title":"Int. J. Comput. Vis."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., and Cheng, M.M. (2017, January 17\u201319). Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, PR, USA.","DOI":"10.1109\/CVPR.2017.302"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1016\/j.neucom.2021.10.050","article-title":"Multi-perspective social recommendation method with graph representation learning","volume":"468","author":"Liu","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_26","unstructured":"Lhuillier, M., and Quan, L. (1999, January 23\u201325). Image Interpolation by Joint View Triangulation. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, USA."},{"key":"ref_27","unstructured":"Lee, I.C., He, S., Lai, P.L., and Yilmaz, A. (2010, January 26\u201330). BUILDING Point Grouping Using View-Geometry Relations. Proceedings of the ASPRS 2010 Annual Conference, San Diego, CA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3980","DOI":"10.3182\/20110828-6-IT-1002.01656","article-title":"Automatic Epipolar Geometry Recovery Using Two Images","volume":"44","author":"Takimoto","year":"2011","journal-title":"IFAC Proc. Vol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Moo Yi, K., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., and Fua, P. (2018, January 18\u201323). Learning to find good correspondences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00282"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4045","DOI":"10.1109\/TIP.2019.2906490","article-title":"LMR: Learning a two-class classifier for mismatch removal","volume":"28","author":"Ma","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhao, C., Cao, Z., Li, C., Li, X., and Yang, J. (2019, January 15\u201320). NM-Net: Mining reliable neighbors for robust feature correspondences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00030"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_33","unstructured":"Chum, O., and Matas, J. (2005, January 20\u201326). Matching with PROSAC-progressive sample consensus. Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tran, Q.H., Chin, T.J., Carneiro, G., Brown, M.S., and Suter, D. (2012, January 7\u201313). In defence of RANSAC for outlier rejection in deformable registration. Proceedings of the European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33765-9_20"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11263-010-0318-x","article-title":"Rejecting mismatches by correspondence function","volume":"89","author":"Li","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6469","DOI":"10.1109\/TGRS.2015.2441954","article-title":"Robust feature matching for remote sensing image registration via locally linear transforming","volume":"53","author":"Ma","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/TPAMI.2017.2652468","article-title":"CODE: Coherence based decision boundaries for feature correspondence","volume":"40","author":"Lin","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1109\/TIP.2019.2934572","article-title":"Robust feature matching using spatial clustering with heavy outliers","volume":"29","author":"Jiang","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"6462","DOI":"10.1109\/TGRS.2019.2906183","article-title":"Multiscale locality and rank preservation for robust feature matching of remote sensing images","volume":"57","author":"Jiang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhao, C., Ge, Y., Zhu, F., Zhao, R., Li, H., and Salzmann, M. (2021, January 11\u201317). Progressive Correspondence Pruning by Consensus Learning. Proceedings of the ICCV, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00640"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3110","DOI":"10.1109\/TPAMI.2020.3048013","article-title":"OANet: Learning Two-View Correspondences and Geometry Using Order-Aware Network","volume":"44","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Liu, Y., Liu, L., Lin, C., Dong, Z., and Wang, W. (2021, January 19\u201325). Learnable Motion Coherence for Correspondence Pruning. Proceedings of the CVPR. Computer Vision Foundation, Virtual.","DOI":"10.1109\/CVPR46437.2021.00325"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sun, W., Jiang, W., Trulls, E., Tagliasacchi, A., and Yi, K.M. (2020, January 14\u201319). ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning. Proceedings of the CVPR. Computer Vision Foundation, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01130"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s11263-016-0902-9","article-title":"Large-Scale Data for Multiple-View Stereopsis","volume":"120","author":"Jensen","year":"2016","journal-title":"Int. J. Comput. Vis."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1109\/TPAMI.2009.77","article-title":"DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo","volume":"32","author":"Tola","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1007\/s11263-020-01385-0","article-title":"Image Matching Across Wide Baselines: From Paper to Practice","volume":"129","author":"Jin","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Sch\u00f6nberger, J.L., and Frahm, J. (2016, January 27\u201330). Structure-from-Motion Revisited. Proceedings of the CVPR, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.445"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/4\/926\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:27:42Z","timestamp":1760120862000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/4\/926"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,8]]},"references-count":47,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["rs15040926"],"URL":"https:\/\/doi.org\/10.3390\/rs15040926","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,2,8]]}}}