{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:15:39Z","timestamp":1760710539971,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T00:00:00Z","timestamp":1617321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003176","name":"Ministerio de Educaci\u00f3n, Cultura y Deporte","doi-asserted-by":"publisher","award":["FPU17\/04512"],"award-info":[{"award-number":["FPU17\/04512"]}],"id":[{"id":"10.13039\/501100003176","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010198","name":"Ministerio de Econom\u00eda, Industria y Competitividad, Gobierno de Espa\u00f1a","doi-asserted-by":"publisher","award":["DPI2017-84827-R","PID2020-117057"],"award-info":[{"award-number":["DPI2017-84827-R","PID2020-117057"]}],"id":[{"id":"10.13039\/501100010198","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper addresses appearance-based robot localization in 2D with a sparse, lightweight map of the environment composed of descriptor\u2013pose image pairs. Based on previous research in the field, we assume that image descriptors are samples of a low-dimensional Descriptor Manifold that is locally articulated by the camera pose. We propose a piecewise approximation of the geometry of such Descriptor Manifold through a tessellation of so-called Patches of Smooth Appearance Change (PSACs), which defines our appearance map. Upon this map, the presented robot localization method applies both a Gaussian Process Particle Filter (GPPF) to perform camera tracking and a Place Recognition (PR) technique for relocalization within the most likely PSACs according to the observed descriptor. A specific Gaussian Process (GP) is trained for each PSAC to regress a Gaussian distribution over the descriptor for any particle pose lying within that PSAC. The evaluation of the observed descriptor in this distribution gives us a likelihood, which is used as the weight for the particle. Besides, we model the impact of appearance variations on image descriptors as a white noise distribution within the GP formulation, ensuring adequate operation under lighting and scene appearance changes with respect to the conditions in which the map was constructed. A series of experiments with both real and synthetic images show that our method outperforms state-of-the-art appearance-based localization methods in terms of robustness and accuracy, with median errors below 0.3 m and 6\u00b0.<\/jats:p>","DOI":"10.3390\/s21072483","type":"journal-article","created":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T10:34:09Z","timestamp":1617359649000},"page":"2483","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Appearance-Based Sequential Robot Localization Using a Patchwise Approximation of a Descriptor Manifold"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0434-7004","authenticated-orcid":false,"given":"Alberto","family":"Jaenal","sequence":"first","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2997-7571","authenticated-orcid":false,"given":"Francisco-Angel","family":"Moreno","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3845-3497","authenticated-orcid":false,"given":"Javier","family":"Gonzalez-Jimenez","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lowe, D.G. (1999, January 20\u201327). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1744","DOI":"10.1109\/TPAMI.2016.2611662","article-title":"Efficient & effective prioritized matching for large-scale image-based localization","volume":"39","author":"Sattler","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TRO.2019.2899783","article-title":"PL-SLAM: A stereo SLAM system through the combination of points and line segments","volume":"35","author":"Moreno","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sattler, T., Torii, A., Sivic, J., Pollefeys, M., Taira, H., Okutomi, M., and Pajdla, T. (2017, January 21\u201326). Are large-scale 3d models really necessary for accurate visual localization?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.654"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1023\/A:1011139631724","article-title":"Modeling the shape of the scene: A holistic representation of the spatial envelope","volume":"42","author":"Oliva","year":"2001","journal-title":"Int. J. Comput. Vis."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., and Sivic, J. (2016, January 27\u201330). NetVLAD: CNN architecture for weakly supervised place recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.572"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1177\/0278364908090961","article-title":"FAB-MAP: Probabilistic localization and mapping in the space of appearance","volume":"27","author":"Cummins","year":"2008","journal-title":"Int. J. Robot. Res."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Milford, M.J., and Wyeth, G.F. (2012, January 14\u201318). SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224623"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pepperell, E., Corke, P.I., and Milford, M.J. (June, January 31). All-environment visual place recognition with SMART. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6907067"},{"key":"ref_12","first-page":"1655","article-title":"Fine-tuning CNN image retrieval with no human annotation","volume":"41","author":"Tolias","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1016\/S0262-8856(00)00108-6","article-title":"Continuity properties of the appearance manifold for mobile robot position estimation","volume":"19","author":"Crowley","year":"2001","journal-title":"Image Vis. Comput."},{"key":"ref_14","unstructured":"Ham, J., Lin, Y., and Lee, D.D. (2005, January 2\u20136). Learning nonlinear appearance manifolds for robot localization. Proceedings of the 2005 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1002\/rob.20170","article-title":"Topological map learning from outdoor image sequences","volume":"23","author":"He","year":"2006","journal-title":"J. Field Robot."},{"key":"ref_16","unstructured":"Gomez-Ojeda, R., Lopez-Antequera, M., Petkov, N., and Gonzalez-Jimenez, J. (2015). Training a convolutional neural network for appearance-invariant place recognition. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1007\/s10514-009-9119-x","article-title":"GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models","volume":"27","author":"Ko","year":"2009","journal-title":"Auton. Robot."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lopez-Antequera, M., Petkov, N., and Gonzalez-Jimenez, J. (2016, January 4\u20137). Image-based localization using Gaussian processes. Proceedings of the 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Alcala de Henares, Spain.","DOI":"10.1109\/IPIN.2016.7743697"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2323","DOI":"10.1126\/science.290.5500.2323","article-title":"Nonlinear dimensionality reduction by locally linear embedding","volume":"290","author":"Roweis","year":"2000","journal-title":"Science"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","article-title":"A global geometric framework for nonlinear dimensionality reduction","volume":"290","author":"Tenenbaum","year":"2000","journal-title":"Science"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"83","DOI":"10.2197\/ipsjtcva.1.83","article-title":"A survey of manifold learning for images","volume":"1","author":"Pless","year":"2009","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wakin, M.B., Donoho, D.L., Choi, H., and Baraniuk, R.G. The multiscale structure of non-differentiable image manifolds. Proceedings of the Wavelets XI, Proceedings of the SPIE, San Diego, CA, USA, 17 September 2005.","DOI":"10.1117\/12.617822"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.patrec.2017.04.017","article-title":"Appearance-invariant place recognition by discriminatively training a convolutional neural network","volume":"92","author":"Petkov","year":"2017","journal-title":"Pattern Recognit. Lett."},{"key":"ref_24","unstructured":"Thoma, J., Paudel, D.P., Chhatkuli, A., and Van Gool, L. (2020). Learning Condition Invariant Features for Retrieval-Based Localization from 1M Images. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Jaenal, A., Zu\u00f1iga-N\u00f6el, D., Gomez-Ojeda, R., and Gonzalez-Jimenez, J. (January, January 24). Improving Visual SLAM in Car-Navigated Urban Environments with Appearance Maps. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341451"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2015.2496823","article-title":"Visual place recognition: A survey","volume":"32","author":"Lowry","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lenc, K., and Vedaldi, A. (2015, January 7\u201312). Understanding image representations by measuring their equivariance and equivalence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298701"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jaenal, A., Moreno, F.A., and Gonzalez-Jimenez, J. (2019, January 7\u20139). Experimental study of the suitability of CNN-based holistic descriptors for accurate visual localization. Proceedings of the 2nd International Conference on Applications of Intelligent Systems, Las Palmas de Gran Canaria, Spain.","DOI":"10.1145\/3309772.3309800"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sattler, T., Weyand, T., Leibe, B., and Kobbelt, L. (2012, January 3\u20137). Image Retrieval for Image-Based Localization Revisited. Proceedings of the British Machine Vision Conference, Surrey, UK.","DOI":"10.5244\/C.26.76"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Torii, A., Arandjelovic, R., Sivic, J., Okutomi, M., and Pajdla, T. (2015, January 7\u201312). 24\/7 place recognition by view synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298790"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Laskar, Z., Melekhov, I., Kalia, S., and Kannala, J. (2017, January 22\u201329). Camera relocalization by computing pairwise relative poses using convolutional neural network. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.113"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Balntas, V., Li, S., and Prisacariu, V. (2018, January 8\u201314). RelocNet: Continuous Metric Learning Relocalisation using Neural Nets. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_46"},{"key":"ref_33","unstructured":"Ding, M., Wang, Z., Sun, J., Shi, J., and Luo, P. (November, January 27). CamNet: Coarse-to-fine retrieval for camera re-localization. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_34","unstructured":"S\u00fcnderhauf, N., Neubert, P., and Protzel, P. (2013, January 6\u201310). Are we there yet? Challenging SeqSLAM on a 3000 km journey across all four seasons. Proceedings of the Workshop on Long-Term Autonomy, IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Naseer, T., Spinello, L., Burgard, W., and Stachniss, C. (2014, January 27\u201331). Robust visual robot localization across seasons using network flows. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Quebec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.9057"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1109\/TRO.2017.2788045","article-title":"Robust visual localization across seasons","volume":"34","author":"Naseer","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Thoma, J., Paudel, D.P., Chhatkuli, A., Probst, T., and Gool, L.V. (2019, January 15\u201320). Mapping, localization and path planning for image-based navigation using visual features and map. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00756"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1177\/0278364912438273","article-title":"CAT-SLAM: Probabilistic localisation and mapping using a continuous appearance-based trajectory","volume":"31","author":"Maddern","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Rasmussen, C.E. (2004). Gaussian processes in machine learning. Advanced Lectures on Machine Learning, Springer.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Huhle, B., Schairer, T., Schilling, A., and Stra\u00dfer, W. (2010, January 18\u201322). Learning to localize with Gaussian process regression on omnidirectional image data. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5650977"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Schairer, T., Huhle, B., Vorst, P., Schilling, A., and Stra\u00dfer, W. (2011, January 25\u201330). Visual mapping with uncertainty for correspondence-free localization using Gaussian process regression. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048206"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lopez-Antequera, M., Petkov, N., and Gonzalez-Jimenez, J. (2017, January 6\u20138). City-scale continuous visual localization. Proceedings of the 2017 European Conference on Mobile Robots (ECMR), Paris, France.","DOI":"10.1109\/ECMR.2017.8098692"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Doucet, A., de Freitas, N., and Gordon, N. (2001). Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science; Springer Science & Business Media.","DOI":"10.1007\/978-1-4757-3437-9"},{"key":"ref_44","unstructured":"Blanco, J.L. (2010). A Tutorial on SE(3) Transformation Parameterizations and On-Manifold Optimization, Universidad de M\u00e1laga. Technical Report."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1177\/0278364909103912","article-title":"COLD: The CoSy localization database","volume":"28","author":"Pronobis","year":"2009","journal-title":"Int. J. Robot. Res."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Song, S., Yu, F., Zeng, A., Chang, A.X., Savva, M., and Funkhouser, T. (2017, January 21\u201326). Semantic scene completion from a single depth image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.28"},{"key":"ref_47","unstructured":"Wu, Y., Wu, Y., Gkioxari, G., and Tian, Y. (2018). Building generalizable agents with a realistic and rich 3D environment. arXiv."},{"key":"ref_48","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_50","unstructured":"GPy (2021, March 30). GPy: A Gaussian Process Framework in Python. Since 2012. Available online: http:\/\/github.com\/SheffieldML\/GPy."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2483\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:33:42Z","timestamp":1760362422000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2483"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,2]]},"references-count":50,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21072483"],"URL":"https:\/\/doi.org\/10.3390\/s21072483","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,4,2]]}}}