{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,15]],"date-time":"2025-12-15T14:16:45Z","timestamp":1765808205698,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2023,7,5]],"date-time":"2023-07-05T00:00:00Z","timestamp":1688515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Three-dimensional (3D) scene reconstruction plays an important role in digital cities, virtual reality, and simultaneous localization and mapping (SLAM). In contrast to perspective images, a single panoramic image can contain the complete scene information because of the wide field of view. The extraction and matching of image feature points is a critical and difficult part of 3D scene reconstruction using panoramic images. We attempted to solve this problem using convolutional neural networks (CNNs). Compared with traditional feature extraction and matching algorithms, the SuperPoint (SP) and SuperGlue (SG) algorithms have advantages for handling images with distortions. However, the rich content of panoramic images leads to a significant disadvantage of these algorithms with regard to time loss. To address this problem, we introduce the Improved Cube Projection Model: First, the panoramic image is projected into split-frame perspective images with significant overlap in six directions. Second, the SP and SG algorithms are used to process the six split-frame images in parallel for feature extraction and matching. Finally, matching points are mapped back to the panoramic image through coordinate inverse mapping. Experimental results in multiple environments indicated that the algorithm can not only guarantee the number of feature points extracted and the accuracy of feature point extraction but can also significantly reduce the computation time compared to other commonly used algorithms.<\/jats:p>","DOI":"10.3390\/rs15133411","type":"journal-article","created":{"date-parts":[[2023,7,6]],"date-time":"2023-07-06T00:41:27Z","timestamp":1688604087000},"page":"3411","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Leveraging CNNs for Panoramic Image Matching Based on Improved Cube Projection Model"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0466-1001","authenticated-orcid":false,"given":"Tian","family":"Gao","sequence":"first","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaozhen","family":"Lan","sequence":"additional","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Longhao","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2736-8692","authenticated-orcid":false,"given":"Wenjun","family":"Huang","sequence":"additional","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fushan","family":"Yao","sequence":"additional","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zijun","family":"Wei","sequence":"additional","affiliation":[{"name":"Institute of Geospatial Information, The PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"66","DOI":"10.26833\/ijeg.589489","article-title":"Integration of Custom Street View and Low Cost Motion Sensors","volume":"5","author":"Bakirman","year":"2020","journal-title":"Int. J. Eng. Geosci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Peng, C., Chen, B.Y., and Tsai, C.H. (2010, January 16\u201318). Integrated google maps and smooth street view videos for route planning. Proceedings of the 2010 International Computer Symposium (ICS2010), Tainan, Taiwan.","DOI":"10.1109\/COMPSYM.2010.5685494"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1828","DOI":"10.1109\/TVCG.2019.2898799","article-title":"Megaparallax: Casual 360 panoramas with motion parallax","volume":"25","author":"Bertel","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2895","DOI":"10.1109\/TVCG.2018.2868533","article-title":"Collaborative large-scale dense 3d reconstruction with online inter-agent pose optimisation","volume":"24","author":"Golodetz","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3217","DOI":"10.1109\/TVCG.2019.2919619","article-title":"Heterofusion: Dense scene reconstruction integrating multi-sensors","volume":"26","author":"Yang","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3137","DOI":"10.1109\/TVCG.2017.2786233","article-title":"Mixedfusion: Real-time reconstruction of an indoor scene with dynamic objects","volume":"24","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2993","DOI":"10.1109\/TVCG.2018.2868527","article-title":"Towards fully mobile 3D face, body, and environment capture using only head-worn cameras","volume":"24","author":"Cha","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3446","DOI":"10.1109\/TVCG.2020.3023634","article-title":"Mobile3DRecon: Real-time monocular 3D reconstruction on a mobile phone","volume":"26","author":"Yang","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/TVCG.2019.2930691","article-title":"Flyfusion: Realtime dynamic scene reconstruction using a flying depth camera","volume":"27","author":"Xu","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1109\/TVCG.2018.2793599","article-title":"Saliency in VR: How do people explore virtual environments?","volume":"24","author":"Sitzmann","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2102","DOI":"10.1109\/TVCG.2019.2899231","article-title":"SLAMCast: Large-scale, real-time 3D reconstruction and streaming for immersive multi-client live telepresence","volume":"25","author":"Stotko","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3052","DOI":"10.1109\/TVCG.2019.2932216","article-title":"Hierarchical topic model based object association for semantic SLAM","volume":"25","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1109\/TVCG.2010.266","article-title":"Sketch-based image retrieval: Benchmark and bag-of-features descriptors","volume":"17","author":"Eitz","year":"2010","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1885","DOI":"10.1109\/TVCG.2013.15","article-title":"Supermatching: Feature matching using supersymmetric geometric constraints","volume":"19","author":"Cheng","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1109\/TVCG.2003.1207439","article-title":"Three-dimensional flow characterization using vector pattern matching","volume":"9","author":"Heiberg","year":"2003","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3073","DOI":"10.1109\/TVCG.2019.2932172","article-title":"AR HMD guidance for controlled hand-held 3D acquisition","volume":"25","author":"Andersen","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1049\/iet-ipr.2012.0323","article-title":"Three-dimensional positioning from Google street view panoramas","volume":"7","author":"Tsai","year":"2013","journal-title":"Iet Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant interest points","volume":"60","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-Up Robust Features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Daniilidis, K., Maragos, P., and Paragios, N. (2010, January 5\u201311). BRIEF: Binary Robust Independent Elementary Features. Proceedings of the Computer Vision\u2013ECCV 2010, Heraklion, Greece.","DOI":"10.1007\/978-3-642-15561-1"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"122","DOI":"10.2478\/msr-2013-0021","article-title":"A Comparative Study of SIFT and its Variants","volume":"13","author":"Wu","year":"2013","journal-title":"Meas. Sci. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2227","DOI":"10.1109\/TPAMI.2014.2321376","article-title":"Scalable nearest neighbor algorithms for high dimensional data","volume":"36","author":"Muja","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1007\/s10044-014-0427-1","article-title":"CSIFT based locality-constrained linear coding for image classification","volume":"18","author":"Chen","year":"2015","journal-title":"Pattern Anal. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1109\/TVCG.2018.2865138","article-title":"Evaluating \u2018graphical perception\u2019with CNNs","volume":"25","author":"Haehn","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1109\/TVCG.2020.3030461","article-title":"Cnnpruner: Pruning convolutional neural networks with visual analytics","volume":"27","author":"Li","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2015, January 7\u201312). Learning to compare image patches via convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299064"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/LGRS.2017.2781741","article-title":"Remote sensing image registration using convolutional neural network features","volume":"15","author":"Ye","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yi, K.M., Trulls, E., Lepetit, V., and Fua, P. (2016, January 11\u201314). Lift: Learned invariant feature transform. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_28"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201322). Superpoint: Self-supervised interest point detection and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Noh, H., Araujo, A., Sim, J., Weyand, T., and Han, B. (2017, January 22\u201329). Large-scale image retrieval with attentive deep local features. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.374"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., and Sattler, T. (2019). D2-net: A trainable cnn for joint detection and description of local features. arXiv.","DOI":"10.1109\/CVPR.2019.00828"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sarlin, P.E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 13\u201319). Superglue: Learning feature matching with graph neural networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00499"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, W., Qian, Y., K\u00e4m\u00e4r\u00e4inen, J.K., Cricri, F., and Fan, L. (2018, January 20\u201324). Object detection in equirectangular panorama. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8546070"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"11754","DOI":"10.1109\/ACCESS.2019.2961184","article-title":"A fast epipolar line matching method based on 3D spherical panorama","volume":"8","author":"Liu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.image.2018.03.013","article-title":"360-aware saliency estimation with conventional image saliency predictors","volume":"69","author":"Startsev","year":"2018","journal-title":"Signal Process. Image Commun."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, J., Wen, Z., Li, S., Zhao, Y., Guo, B., and Wen, J. (2016, January 25\u201328). Novel tile segmentation scheme for omnidirectional video. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532381"},{"key":"ref_38","first-page":"1371","article-title":"Automatically measuring the coordinates of streetlights in vehicle-borne spherical images","volume":"23","author":"Zhixuan","year":"2018","journal-title":"J. Image Graph."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, F.E., Yeh, Y.H., Sun, M., Chiu, W.C., and Tsai, Y.H. (2020, January 13). Bifuse: Monocular 360 depth estimation via bi-projection fusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00054"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, Y., Cai, S., Li, S.J., Liu, Y., Guo, Y., Li, T., and Cheng, M.M. (2018, January 2\u20136). Cubemapslam: A piecewise-pinhole monocular fisheye slam system. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.","DOI":"10.1007\/978-3-030-20876-9_3"},{"key":"ref_41","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"17329","DOI":"10.1007\/s00521-022-07395-y","article-title":"Estimation of voting behavior in election using support vector machine, extreme learning machine and deep learning","volume":"34","author":"Tanyildizi","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_43","first-page":"285","article-title":"An improved ASIFT algorithm for indoor panorama image matching","volume":"Volume 10420","author":"Fu","year":"2017","journal-title":"Proceedings of the Ninth International Conference on Digital Image Processing (ICDIP 2017)"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/13\/3411\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:06:51Z","timestamp":1760126811000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/13\/3411"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,5]]},"references-count":43,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["rs15133411"],"URL":"https:\/\/doi.org\/10.3390\/rs15133411","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,7,5]]}}}