{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T02:22:51Z","timestamp":1776478971326,"version":"3.51.2"},"reference-count":42,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T00:00:00Z","timestamp":1697155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42371442"],"award-info":[{"award-number":["42371442"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2023AFB568"],"award-info":[{"award-number":["2023AFB568"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2021-114"],"award-info":[{"award-number":["2021-114"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hubei Provincial Natural Science Foundation of China","award":["42371442"],"award-info":[{"award-number":["42371442"]}]},{"name":"Hubei Provincial Natural Science Foundation of China","award":["2023AFB568"],"award-info":[{"award-number":["2023AFB568"]}]},{"name":"Hubei Provincial Natural Science Foundation of China","award":["2021-114"],"award-info":[{"award-number":["2021-114"]}]},{"name":"Hong Kong Scholars Program","award":["42371442"],"award-info":[{"award-number":["42371442"]}]},{"name":"Hong Kong Scholars Program","award":["2023AFB568"],"award-info":[{"award-number":["2023AFB568"]}]},{"name":"Hong Kong Scholars Program","award":["2021-114"],"award-info":[{"award-number":["2021-114"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Spherical images have the advantage of recording full scenes using only one camera exposure and have been becoming an important data source for 3D reconstruction. However, geometric distortions inevitably exist due to the spherical camera imaging model. Thus, this study proposes a reliable feature matching algorithm for spherical images via the combination of local geometric rectification and CNN (convolutional neural network) learned descriptor. First, image patches around keypoints are reprojected to their corresponding tangent planes based on a spherical camera imaging model, which uses scale and orientation data from the keypoints to achieve both rotation and scale invariance. Second, feature descriptors are then calculated from the rectified image patches by using a pre-trained separate detector and descriptor learning network, which improves the discriminability by exploiting the high representation learning ability of the CNN. Finally, after classical feature matching with the ratio test and cross check, refined matches are obtained based on an essential matrix-based epipolar geometry constraint for outlier removal. By using three real spherical images and an incremental structure from motion (SfM) engine, the proposed algorithm is verified and compared in terms of feature matching and image orientation. The experiment results demonstrate that the geometric distortions can be efficiently reduced from rectified image patches, and the increased ratio of the match numbers ranges from 26.8% to 73.9%. For SfM-based spherical image orientation, the proposed algorithm provides reliable feature matches to achieve complete reconstruction with comparative accuracy.<\/jats:p>","DOI":"10.3390\/rs15204954","type":"journal-article","created":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T10:04:39Z","timestamp":1697191479000},"page":"4954","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Reliable Feature Matching for Spherical Images via Local Geometric Rectification and Learned Descriptor"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7799-650X","authenticated-orcid":false,"given":"San","family":"Jiang","sequence":"first","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"},{"name":"Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong 999077, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9452-3282","authenticated-orcid":false,"given":"Junhuan","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5610-3223","authenticated-orcid":false,"given":"Yaxin","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong 999077, China"}]},{"given":"Duojie","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong 999077, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1787-5191","authenticated-orcid":false,"given":"Wu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong 999077, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1109\/MGRS.2021.3122248","article-title":"Unmanned Aerial Vehicle-Based Photogrammetric 3D Mapping: A survey of techniques, applications, and challenges","volume":"10","author":"Jiang","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.isprsjprs.2018.03.004","article-title":"Integration of aerial oblique imagery and terrestrial imagery for optimized 3D modeling in urban areas","volume":"139","author":"Wu","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"28","DOI":"10.4995\/var.2018.5958","article-title":"UAV photogrammetry for archaeological site survey. 3D models at the Hierapolis in Phrygia (Turkey)","volume":"9","author":"Chiabrando","year":"2018","journal-title":"Virtual Archaeol. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jiang, S., Jiang, W., Huang, W., and Yang, L. (2017). UAV-based oblique photogrammetry for outdoor data acquisition and offsite visual inspection of transmission line. Remote Sens., 9.","DOI":"10.3390\/rs9030278"},{"key":"ref_5","unstructured":"Jiang, S., You, K., Li, Y., Weng, D., and Chen, W. (2023). 3D Reconstruction of Spherical Images based on Incremental Structure from Motion. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Torii, A., Havlena, M., and Pajdla, T. (October, January 29). From google street view to 3d city models. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, Kyoto, Japan.","DOI":"10.1109\/ICCVW.2009.5457551"},{"key":"ref_7","first-page":"1","article-title":"Review on panoramic imaging and its applications in scene understanding","volume":"71","author":"Gao","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/LGRS.2020.3048150","article-title":"Integrating UAV and ground panoramic images for point cloud analysis of damaged building","volume":"19","author":"Jhan","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"331","DOI":"10.5194\/isprs-archives-XLII-2-331-2018","article-title":"Improving spherical photogrammetry using 360 omni-cameras: Use cases and new applications","volume":"42","author":"Fangi","year":"2018","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Janiszewski, M., Torkan, M., Uotinen, L., and Rinne, M. (2022). Rapid photogrammetry with a 360-degree camera for tunnel mapping. Remote Sens., 14.","DOI":"10.3390\/rs14215494"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2019.04.006","article-title":"Reliable image matching via photometric and geometric constraints structured by Delaunay triangulation","volume":"153","author":"Jiang","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pagani, A., and Stricker, D. (2011, January 6\u201313). Structure from motion using full spherical panoramic cameras. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130266"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/phor.12315","article-title":"Geometric modelling and calibration of a spherical camera imaging system","volume":"35","author":"Lichti","year":"2020","journal-title":"Photogramm. Rec."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"25","DOI":"10.14358\/PERS.84.1.25","article-title":"Rectified feature matching for spherical panoramic images","volume":"84","author":"Chuang","year":"2018","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"84","DOI":"10.2197\/ipsjtcva.7.84","article-title":"Robust feature matching for distorted projection by spherical cameras","volume":"7","author":"Taira","year":"2015","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, Y., Cai, S., Li, S.J., Liu, Y., Guo, Y., Li, T., and Cheng, M.M. (2018, January 2\u20136). CubemapSLAM: A piecewise-pinhole monocular fisheye SLAM system. Proceedings of the Asian Conference on Computer Vision, Perth, WA, Australia.","DOI":"10.1007\/978-3-030-20876-9_3"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s11263-014-0787-4","article-title":"SPHORB: A fast and robust binary feature on the sphere","volume":"113","author":"Zhao","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Guan, H., and Smith, W.A. (2017, January 21\u201326). BRISKS: Binary features for spherical images on a geodesic grid. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.519"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1080\/10095020.2020.1843376","article-title":"Feature detection and description for image matching: From hand-crafted design to deep learning","volume":"24","author":"Chen","year":"2021","journal-title":"Geo-Spat. Inf. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10583","DOI":"10.1109\/JSTARS.2021.3119990","article-title":"Learned local features for structure from motion of uav images: A comparative evaluation","volume":"14","author":"Jiang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","unstructured":"Han, X., Leung, T., Jia, Y., Sukthankar, R., and Berg, A.C. (2015, January 7\u201312). Matchnet: Unifying feature and metric learning for patch-based matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_24","unstructured":"Kumar BG, V., Carneiro, G., and Reid, I. (2016, January 27\u201330). Learning local image descriptors with deep siamese and triplet convolutional networks by minimising global loss functions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Luo, Z., Shen, T., Zhou, L., Zhu, S., Zhang, R., Yao, Y., Fang, T., and Quan, L. (2018, January 8\u201314). Geodesc: Learning local descriptors by integrating geometry constraints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01240-3_11"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tian, Y., Fan, B., and Wu, F. (2017, January 21\u201326). L2-net: Deep learning of discriminative patch descriptor in euclidean space. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.649"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., and Sattler, T. (2019, January 15\u201320). D2-net: A trainable cnn for joint description and detection of local features. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00828"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Luo, Z., Zhou, L., Bai, X., Chen, H., Zhang, J., Yao, Y., Li, S., Fang, T., and Quan, L. (2020, January 13\u201319). Aslfeat: Learning local features of accurate shape and localization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00662"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Eder, M., Shvets, M., Lim, J., and Frahm, J.M. (2020, January 13\u201319). Tangent images for mitigating spherical distortion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01244"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"20748","DOI":"10.1109\/ACCESS.2018.2825477","article-title":"Descriptor matching for a discrete spherical image with a convolutional neural network","volume":"6","author":"Shan","year":"2018","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Coors, B., Condurache, A.P., and Geiger, A. (2018, January 8\u201314). Spherenet: Learning spherical representations for detection and classification in omnidirectional images. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01240-3_32"},{"key":"ref_32","unstructured":"Su, Y.C., and Grauman, K. (2017). Learning spherical convolution for fast features from 360 imagery. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhao, Q., Zhu, C., Dai, F., Ma, Y., Jin, G., and Zhang, Y. (2018, January 13\u201319). Distortion-aware CNNs for Spherical Images. Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/167"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Su, Y.C., and Grauman, K. (2019, January 15\u201320). Kernel transformer networks for compact spherical convolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00967"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mei, C., and Rives, P. (2007, January 10\u201314). Single view point omnidirectional camera calibration from planar grids. Proceedings of the Proceedings 2007 IEEE International Conference on Robotics and Automation, Rome, Italy.","DOI":"10.1109\/ROBOT.2007.364084"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Scaramuzza, D., Martinelli, A., and Siegwart, R. (2006, January 4\u20137). A flexible technique for accurate omnidirectional camera calibration and structure from motion. Proceedings of the Fourth IEEE International Conference on Computer Vision Systems (ICVS\u201906), New York, NY, USA.","DOI":"10.1109\/ICVS.2006.3"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"229","DOI":"10.14358\/PERS.80.3.229","article-title":"Comparison of two panoramic sensor models for precise 3d measurements","volume":"80","author":"Ji","year":"2014","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_38","unstructured":"Mishchuk, A., Mishkin, D., Radenovic, F., and Matas, J. (2017, January 4\u20139). Working hard to know your neighbor\u2019s margins: Local descriptor learning loss. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_40","unstructured":"Wu, C. (2023, October 10). SiftGPU: A GPU Implementation of Sift. Available online: http:\/\/cs.unc.edu\/~ccwu\/siftgpu."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1109\/34.88573","article-title":"Least-squares estimation of transformation parameters between two point patterns","volume":"13","author":"Umeyama","year":"1991","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/20\/4954\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:06:28Z","timestamp":1760130388000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/20\/4954"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,13]]},"references-count":42,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["rs15204954"],"URL":"https:\/\/doi.org\/10.3390\/rs15204954","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,13]]}}}