{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:03:32Z","timestamp":1760238212818,"version":"build-2065373602"},"reference-count":57,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2020,7,27]],"date-time":"2020-07-27T00:00:00Z","timestamp":1595808000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Zhejiang Provincial Public Fund","award":["No.2016C33136"],"award-info":[{"award-number":["No.2016C33136"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual Place Recognition (VPR) addresses visual instance retrieval tasks against discrepant scenes and gives precise localization. During a traverse, the captured images (query images) would be traced back to the already existing positions in the database images, rendering vehicles or pedestrian navigation devices distinguish ambient environments. Unfortunately, diverse appearance variations can bring about huge challenges for VPR, such as illumination changing, viewpoint varying, seasonal cycling, disparate traverses (forward and backward), and so on. In addition, the majority of current VPR algorithms are designed for forward-facing images, which can only provide with narrow Field of View (FoV) and come with severe viewpoint influences. In this paper, we propose a panoramic localizer, which is based on coarse-to-fine descriptors, leveraging panoramas for omnidirectional perception and sufficient FoV up to 360\u2218. We adopt NetVLAD descriptors in the coarse matching in a panorama-to-panorama way, for their robust performances in distinguishing different appearances, utilizing Geodesc keypoint descriptors in the fine stage in the meantime, for their capacity of detecting detailed information, formatting powerful coarse-to-fine descriptors. A comprehensive set of experiments is conducted on several datasets including both public benchmarks and our real-world campus scenes. Our system is proved to be with high recall and strong generalization capacity across various appearances. The proposed panoramic localizer can be integrated into mobile navigation devices, available for a variety of localization application scenarios.<\/jats:p>","DOI":"10.3390\/s20154177","type":"journal-article","created":{"date-parts":[[2020,7,28]],"date-time":"2020-07-28T10:16:49Z","timestamp":1595931409000},"page":"4177","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Panoramic Localizer Based on Coarse-to-Fine Descriptors for Navigation Assistance"],"prefix":"10.3390","volume":"20","author":[{"given":"Yicheng","family":"Fang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1090-667X","authenticated-orcid":false,"given":"Kailun","family":"Yang","sequence":"additional","affiliation":[{"name":"Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7951-196X","authenticated-orcid":false,"given":"Ruiqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"given":"Lei","family":"Sun","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou 310027, China"}]},{"given":"Kaiwei","family":"Wang","sequence":"additional","affiliation":[{"name":"National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou 310058, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lin, S., Wang, K., Cheng, R., and Yang, K. (2018). Visual Localizer: Outdoor Localization Based on ConvNet Descriptor and Global Optimization for Visually Impaired Pedestrians. Sensors, 18.","DOI":"10.3390\/s18082476"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Cheng, R., Wang, K., Lin, S., Hu, W., Yang, K., Huang, X., Li, H., Sun, D., and Bai, J. (2019, January 27\u201330). Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917508"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s10707-013-0176-0","article-title":"Active learning of user\u2019s preferences estimation towards a personalized 3D navigation of geo-referenced scenes","volume":"18","author":"Yiakoumettis","year":"2014","journal-title":"GeoInformatica"},{"key":"ref_4","first-page":"313","article-title":"Using a modified invasive weed optimization algorithm for a personalized urban multi-criteria path optimization problem","volume":"18","author":"Pahlavani","year":"2012","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_5","first-page":"322","article-title":"Multi-criteria, personalized route planning using quantifier-guided ordered weighted averaging operators","volume":"13","author":"Nadi","year":"2011","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Doulamis, N., Yiakoumettis, C., Miaoulis, G., and Protopapadakis, E. (2013). A constraint inductive learning-spectral clustering methodology for personalized 3D navigation. International Symposium on Visual Computing, Springer.","DOI":"10.1007\/978-3-642-41939-3_11"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Brilhault, A., Kammoun, S., Gutierrez, O., Truillet, P., and Jouffrais, C. (2011, January 7\u201310). Fusion of Artificial Vision and GPS to Improve Blind Pedestrian Positioning. Proceedings of the 2011 4th IFIP International Conference on New Technologies, Mobility and Security, Paris, France.","DOI":"10.1109\/NTMS.2011.5721061"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1016\/j.asr.2018.09.010","article-title":"Error constrained control of a pseudo-satellite with disturbance","volume":"63","author":"Lin","year":"2019","journal-title":"Adv. Space Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1109\/MCOM.2015.7060497","article-title":"WiFi-based indoor positioning","volume":"53","author":"Yang","year":"2015","journal-title":"IEEE Commun. Mag."},{"key":"ref_10","unstructured":"Chung, W.C., and Ha, D. (2003, January 16\u201319). An accurate ultra wideband (UWB) ranging for precision asset location. Proceedings of the IEEE Conference on Ultra Wideband Systems and Technologies, Reston, VA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1078","DOI":"10.1109\/TITS.2016.2595618","article-title":"Sensor fusion-based low-cost vehicle localization system for complex urban environments","volume":"18","author":"Suhr","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","first-page":"21","article-title":"Visual place recognition based on multilevel descriptors for the visually impaired people","volume":"Volume 11158","author":"Fang","year":"2019","journal-title":"Target and Background Signatures V. International Society for Optics and Photonics"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7\u201313). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.336"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"124004","DOI":"10.1088\/1361-6501\/ab2106","article-title":"OpenMPR: Recognize places using multimodal data for people with visual impairments","volume":"30","author":"Cheng","year":"2019","journal-title":"Meas. Sci. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cheng, R., Wang, K., Lin, L., and Yang, K. (2018, January 20\u201324). Visual localization of key positions for visually impaired people. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545141"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kameda, Y., and Ohta, Y. (2010, January 23\u201326). Image retrieval of first-person vision for pedestrian navigation in urban area. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.1140"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"S\u00fcnderhauf, N., Shirazi, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., and Milford, M. (2015). Place recognition with convnet landmarks: Viewpoint-robust, condition-robust, training-free. Robotics: Science and Systems XI, Robotics: Science and Systems Conference.","DOI":"10.15607\/RSS.2015.XI.022"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pepperell, E., Corke, P.I., and Milford, M.J. (June, January 31). Automatic image scaling for place recognition in changing environments. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA.2015.7139316"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kanji, T. (October, January 28). Cross-season place recognition using nbnn scene descriptor. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353453"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Milford, M.J., and Wyeth, G.F. (2012, January 14\u201318). SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224623"},{"key":"ref_21","unstructured":"Vysotska, O., and Stachniss, C. (2017, January 24\u201328). Relocalization under substantial appearance changes using hashing. Proceedings of the IROS Workshop on Planning, Perception and Navigation for Intelligent Vehicles, Vancouver, BC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.1007\/s10514-017-9684-3","article-title":"BoCNF: Efficient image matching with Bag of ConvNet features for scalable and robust visual place recognition","volume":"42","author":"Hou","year":"2018","journal-title":"Auton. Robot."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Garg, S., Suenderhauf, N., and Milford, M. (2018). Lost? appearance-invariant place recognition for opposite viewpoints using visual semantics. arXiv.","DOI":"10.15607\/RSS.2018.XIV.022"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yang, K., Hu, X., Chen, H., Xiang, K., Wang, K., and Stiefelhagen, R. (2019). DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing. arXiv.","DOI":"10.1109\/IV47402.2020.9304706"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yu, L., Joly, C., Bresson, G., and Moutarde, F. (2016, January 13\u201315). Monocular Urban Localization using Street View. Proceedings of the 14th International Conference on Control, Automation, Robotics and Vision (ICARCV\u20192016), Phuket, Thailand.","DOI":"10.1109\/ICARCV.2016.7838744"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1002\/rob.21531","article-title":"Dense omnidirectional RGB-D mapping of large scale outdoor environments for real-time localisation and autonomous navigation","volume":"32","author":"Meilland","year":"2015","journal-title":"J. Field Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/BF02328575","article-title":"Characterization of the panoramic annular lens","volume":"36","author":"Lehner","year":"1996","journal-title":"Exp. Mech."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"S\u00fcnderhauf, N., and Protzel, P. (2011, January 25\u201330). BRIEF-Gist - closing the loop by simple means. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048590"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1177\/0278364908090961","article-title":"FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance","volume":"27","author":"Cummins","year":"2008","journal-title":"Int. J. Robot. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hou, Y., Zhang, H., and Zhou, S. (2015, January 8\u201310). Convolutional neural network-based image representation for visual loop closure detection. Proceedings of the 2015 IEEE International Conference on Information and Automation, Lijiang, China.","DOI":"10.1109\/ICInfA.2015.7279659"},{"key":"ref_32","unstructured":"Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y. (2013). OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. arXiv."},{"key":"ref_33","unstructured":"Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. (2013). DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Razavian, A.S., Azizpour, H., Sullivan, J., and Carlsson, S. (2014, January 23\u201328). CNN Features Off-the-Shelf: An Astounding Baseline for Recognition. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.","DOI":"10.1109\/CVPRW.2014.131"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Arandjelovi\u0107, R., Gronat, P., Torii, A., Pajdla, T., and Sivic, J. (2016, January 27\u201330). NetVLAD: CNN architecture for weakly supervised place recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.572"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Luo, Z., Shen, T., Zhou, L., Zhu, S., Zhang, R., Yao, Y., Fang, T., and Quan, L. (2018). GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints (ECCV), Springer International Publishing.","DOI":"10.1007\/978-3-030-01240-3_11"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3219","DOI":"10.1364\/AO.55.003219","article-title":"Comparison of two panoramic front unit arrangements in design of a super wide angle panoramic annular lens","volume":"55","author":"Zhou","year":"2016","journal-title":"Appl. Opt."},{"key":"ref_38","first-page":"111660L","article-title":"A multimodal vision sensor for autonomous driving","volume":"Volume 11166","author":"Sun","year":"2019","journal-title":"Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"014006","DOI":"10.1088\/1361-6501\/ab40d9","article-title":"An indoor positioning framework based on panoramic visual odometry for visually impaired people","volume":"31","author":"Hu","year":"2019","journal-title":"Meas. Sci. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 million image database for scene recognition","volume":"40","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Liao, Z., Shi, J., Qi, X., Zhang, X., Wang, W., He, Y., Wei, R., and Liu, X. (2019). Coarse-To-Fine Visual Localization Using Semantic Compact Map. arXiv.","DOI":"10.1109\/ICCR51572.2020.9344376"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Holliday, A., and Dudek, G. (2018, January 1\u20135). Scale-robust localization using general object landmarks. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594011"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Iscen, A., Tolias, G., Avrithis, Y., Furon, T., and Chum, O. (2017). Panorama to Panorama Matching for Location Recognition. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, Association for Computing Machinery. ICMR \u201917.","DOI":"10.1145\/3078971.3079033"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1006\/cviu.2001.0909","article-title":"Estimating the Fundamental Matrix by Transforming Image Points in Projective Space","volume":"82","author":"Zhang","year":"2001","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_45","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Neural Information Processing Systems Conference, Lake Tahoe, NV, USA."},{"key":"ref_46","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_48","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201322). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_50","unstructured":"Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1109\/T-C.1975.224110","article-title":"An Algorithm for Finding Nearest Neighbors","volume":"C-24","author":"Friedman","year":"1975","journal-title":"IEEE Trans. Comput."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/0146-664X(80)90054-4","article-title":"Euclidean Distance Mapping","volume":"14","author":"Danielsson","year":"1980","journal-title":"Comput. Graph. Image Proc."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Balntas, V., Lenc, K., Vedaldi, A., and Mikolajczyk, K. (2017, January 21\u201326). HPatches: A benchmark and evaluation of handcrafted and learned local descriptors. Proceedings of the CVPR, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.410"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Torii, A., Sivic, J., Pajdla, T., and Okutomi, M. (2013, January 25\u201327). Visual Place Recognition with Repetitive Structures. Proceedings of the CVPR, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.119"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2013\/853283","article-title":"A Fast Image Stitching Algorithm via Multiple-Constraint Corner Matching","volume":"2013","author":"Zhu","year":"2013","journal-title":"Math. Problems Eng."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Siva, S., and Zhang, H. (2018, January 21\u201326). Omnidirectional Multisensory Perception Fusion for Long-Term Place Recognition. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461042"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Cheng, M.M., Zhang, Z., Lin, W.Y., and Torr, P. (2014, January 24\u201327). BING: Binarized normed gradients for objectness estimation at 300fps. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.414"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/15\/4177\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:52:09Z","timestamp":1760176329000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/15\/4177"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,27]]},"references-count":57,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["s20154177"],"URL":"https:\/\/doi.org\/10.3390\/s20154177","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,7,27]]}}}