{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:50:11Z","timestamp":1760230211187,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ARPEGGIO","award":["PID2020-117057","P20_01302","FPU19\/00704"],"award-info":[{"award-number":["PID2020-117057","P20_01302","FPU19\/00704"]}]},{"name":"HOUNDBOT","award":["PID2020-117057","P20_01302","FPU19\/00704"],"award-info":[{"award-number":["PID2020-117057","P20_01302","FPU19\/00704"]}]},{"name":"Spanish grant program","award":["PID2020-117057","P20_01302","FPU19\/00704"],"award-info":[{"award-number":["PID2020-117057","P20_01302","FPU19\/00704"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes LTC-Mapping, a method for building object-oriented semantic maps that remain consistent in the long-term operation of mobile robots. Among the different challenges that compromise this aim, LTC-Mapping focuses on two of the more relevant ones: preventing duplicate instances of objects (instance duplication) and handling dynamic scenes. The former refers to creating multiple instances of the same physical object in the map, usually as a consequence of partial views or occlusions. The latter deals with the typical assumption made by object-oriented mapping methods that the world is static, resulting in outdated representations when the objects change their positions. To face these issues, we model the detected objects with 3D bounding boxes, and analyze the visibility of their vertices to detect occlusions and partial views. Besides this geometric modeling, the boxes are augmented with semantic information regarding the categories of the objects they represent. Both the geometric entities (bounding boxes) and their semantic content are propagated over time through data association and a fusion technique. In addition, in order to keep the map curated, the non-detection of objects in the areas where they should appear is also considered, proposing a mechanism that removes them from the map once there is evidence that they have been moved (i.e., multiple non-detections occur). To validate our proposal, a number of experiments have been carried out using the Robot@VirtualHome ecosystem, comparing its performance with a state-of-the-art alternative. The results report a superior performance of LTC-Mapping when modeling both geometric and semantic information of objects, and also support its online execution.<\/jats:p>","DOI":"10.3390\/s22145308","type":"journal-article","created":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T01:53:22Z","timestamp":1658109202000},"page":"5308","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["LTC-Mapping, Enhancing Long-Term Consistency of Object-Oriented Semantic Maps in Robotics"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4123-7330","authenticated-orcid":false,"given":"Jose-Luis","family":"Matez-Bandera","sequence":"first","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga, 29016 Malaga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5596-1103","authenticated-orcid":false,"given":"David","family":"Fernandez-Chaves","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga, 29016 Malaga, Spain"},{"name":"Johann Bernoulli Institute of Mathematics and Computing Science, University of Groningen, 9712 CP Groningen, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9929-5309","authenticated-orcid":false,"given":"Jose-Raul","family":"Ruiz-Sarmiento","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga, 29016 Malaga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7869-7811","authenticated-orcid":false,"given":"Javier","family":"Monroy","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga, 29016 Malaga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2163-8647","authenticated-orcid":false,"given":"Nicolai","family":"Petkov","sequence":"additional","affiliation":[{"name":"Johann Bernoulli Institute of Mathematics and Computing Science, University of Groningen, 9712 CP Groningen, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3845-3497","authenticated-orcid":false,"given":"Javier","family":"Gonzalez-Jimenez","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga, 29016 Malaga, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,15]]},"reference":[{"key":"ref_1","unstructured":"Chatila, R., and Laumond, J. (1985, January 25\u201328). Position referencing and consistent world modeling for mobile robots. Proceedings of the 1985 IEEE International Conference on Robotics and Automation, St. Louis, MO, USA."},{"key":"ref_2","first-page":"129","article-title":"Modeling Spatial Knowledge","volume":"2","author":"Kuipers","year":"1978","journal-title":"Cogn. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.knosys.2016.12.016","article-title":"Building Multiversal Semantic Maps for Mobile Robot Operation","volume":"119","author":"Galindo","year":"2017","journal-title":"Knowl.-Based Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1016\/j.robot.2008.08.001","article-title":"Towards Semantic Maps for Mobile Robots","volume":"56","author":"Hertzberg","year":"2008","journal-title":"Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1016\/j.robot.2008.08.007","article-title":"Robot Task Planning Using Semantic Maps","volume":"56","author":"Galindo","year":"2008","journal-title":"Robot. Auton. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pronobis, A., and Jensfelt, P. (2012, January 14\u201318). Large-scale semantic mapping and reasoning with heterogeneous modalities. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224637"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"S\u00fcnderhauf, N., Dayoub, F., McMahon, S., Talbot, B., Schulz, R., Corke, P., Wyeth, G., Upcroft, B., and Milford, M. (2016, January 16\u201321). Place categorization and semantic mapping on a mobile robot. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487796"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mutlu, B., Roy, N., and \u0160abanovi\u0107, S. (2016). Cognitive human\u2013robot interaction. Springer Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-319-32552-1_71"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"108022","DOI":"10.1016\/j.knosys.2021.108022","article-title":"Efficient semantic place categorization by a robot through active line-of-sight selection","volume":"240","author":"Monroy","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.robot.2014.12.006","article-title":"Semantic mapping for mobile robotics tasks: A survey","volume":"66","author":"Kostavelis","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"S\u00fcnderhauf, N., Pham, T.T., Latif, Y., Milford, M., and Reid, I. (2017, January 24\u201328). Meaningful maps with object-oriented semantic mapping. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206392"},{"key":"ref_12","unstructured":"Kaszuba, S., Sabbella, S.R., Suriani, V., Riccio, F., and Nardi, D. (2021). RoSmEEry: Robotic Simulated Environment for Evaluation and Benchmarking of Semantic Mapping Algorithms. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dengler, N., Zaenker, T., Verdoja, F., and Bennewitz, M. (September, January 31). Online Object-Oriented Semantic Mapping and Map Updating. Proceedings of the 2021 European Conference on Mobile Robots (ECMR), Bonn, Germany.","DOI":"10.1109\/ECMR50962.2021.9568817"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fernandez-Chaves, D., Ruiz-Sarmiento, J., Jaenal, A., Petkov, N., and Gonzalez-Jimenez, J. (Expert Syst. Appl., 2021). Robot@VirtualHome, an ecosystem of virtual environments and tools for realistic indoor robotic simulation, Expert Syst. Appl., under review.","DOI":"10.1016\/j.eswa.2022.117970"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TRO.2019.2899783","article-title":"PL-SLAM: A stereo SLAM system through the combination of points and line segments","volume":"35","author":"Moreno","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1109\/TRO.2021.3075644","article-title":"Orb-slam3: An accurate open-source library for visual, visual\u2013inertial, and multimap slam","volume":"37","author":"Campos","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2300000059","article-title":"Semantics for robotic mapping, perception and interaction: A survey","volume":"8","author":"Garg","year":"2020","journal-title":"Found. Trends Robot."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Han, X., Li, S., Wang, X., and Zhou, W. (2021). Semantic Mapping for Mobile Robots in Indoor Scenes: A Survey. Information, 12.","DOI":"10.3390\/info12020092"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernandez-Madrigal, J.A., and Gonzalez, J. (2005, January 2\u20136). Multi-hierarchical semantic maps for mobile robotics. Proceedings of the 2005 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada.","DOI":"10.1109\/IROS.2005.1545511"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2050013","DOI":"10.1142\/S0219843620500139","article-title":"Classifying obstacles and exploiting class information for humanoid navigation through cluttered environments","volume":"17","author":"Regier","year":"2020","journal-title":"Int. J. Hum. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tateno, K., Tombari, F., and Navab, N. (October, January 28). Real-time and scalable incremental segmentation on dense slam. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7354011"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"McCormac, J., Handa, A., Davison, A., and Leutenegger, S. (June, January 29). Semanticfusion: Dense 3d semantic mapping with convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Robotics and automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989538"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xiang, Y., and Fox, D. (2017). Da-rnn: Semantic mapping with data associated recurrent neural networks. arXiv.","DOI":"10.15607\/RSS.2017.XIII.013"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, J., Liu, Y., Wang, J., Yan, M., and Yao, Y. (2018, January 25\u201327). 3D semantic mapping based on convolutional neural networks. Proceedings of the 2018 37th Chinese Control Conference (CCC), Wuhan, China.","DOI":"10.23919\/ChiCC.2018.8482938"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3749","DOI":"10.1109\/LRA.2018.2856268","article-title":"Recurrent-octomap: Learning state-based map refinement for long-term semantic mapping with 3-d-lidar data","volume":"3","author":"Sun","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Narita, G., Seno, T., Ishikawa, T., and Kaji, Y. (2019, January 3\u20138). Panopticfusion: Online volumetric semantic mapping at the level of stuff and things. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8967890"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3037","DOI":"10.1109\/LRA.2019.2923960","article-title":"Volumetric instance-aware semantic mapping and 3D object discovery","volume":"4","author":"Grinvald","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3206","DOI":"10.1109\/ACCESS.2018.2887022","article-title":"Efficient object-oriented semantic mapping with object detector","volume":"7","author":"Nakajima","year":"2018","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.robot.2018.03.011","article-title":"Grounding semantic maps in spatial databases","volume":"105","author":"Deeken","year":"2018","journal-title":"Robot. Auton. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, L., Li, R., Sun, J., Liu, X., Zhao, L., Seah, H.S., Quah, C.K., and Tandianus, B. (2019). Multi-view fusion-based 3D object detection for robot indoor scene perception. Sensors, 19.","DOI":"10.3390\/s19194092"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/34.683777","article-title":"Support vector machines for 3D object recognition","volume":"20","author":"Pontil","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","unstructured":"Nister, D., and Stewenius, H. (2006, January 17\u201322). Scalable recognition with a vocabulary tree. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Valentin, J., Sengupta, S., Warrell, J., Shahrokni, A., and Torr, P. (2013, January 23\u201328). Mesh Based Semantic Modelling for Indoor and Outdoor Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013), Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.269"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.knosys.2015.05.032","article-title":"Exploiting Semantic Knowledge for Robot Object Recognition","volume":"86","author":"Galindo","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"8805","DOI":"10.1016\/j.eswa.2015.07.033","article-title":"Scene Object Recognition for Mobile Robots Through Semantic Knowledge and Probabilistic Graphical Models","volume":"42","author":"Galindo","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.knosys.2019.01.005","article-title":"Ontology-based conditional random fields for object recognition","volume":"168","author":"Galindo","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1017\/S0269888900007797","article-title":"Ontologies: Principles, methods and applications","volume":"11","author":"Uschold","year":"1996","journal-title":"Knowl. Eng. Rev."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Pham, T.T., Do, T.T., S\u00fcnderhauf, N., and Reid, I. (2018, January 21\u201325). Scenecut: Joint geometric and object segmentation for indoor scenes. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461108"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21\u201326). Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.549"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Hu, R., Doll\u00e1r, P., He, K., Darrell, T., and Girshick, R. (2018, January 18\u201323). Learning to segment every thing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00445"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1109\/34.161346","article-title":"Thinning methodologies-a comprehensive survey","volume":"14","author":"Lam","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_47","unstructured":"Ester, M., Kriegel, H.P., Sander, J., and Xu, X. (1996, January 2\u20134). A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the kdd, Portland, Oregon."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Greene, N., Kass, M., and Miller, G. (1993, January 2\u20136). Hierarchical Z-buffer visibility. Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, CA, USA.","DOI":"10.1145\/166117.166147"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Jonker, P.P. (2000, January 13\u201315). Morphological operations on 3D and 4D images: From shape primitive detection to skeletonization. Proceedings of the International Conference on Discrete Geometry for Computer Imagery, Uppsala, Sweden.","DOI":"10.1007\/3-540-44438-6_31"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Gonzalez-Jimenez, J., Galindo, C., and Ruiz-Sarmiento, J. (2012, January 9\u201313). Technical improvements of the Giraff telepresence robot based on users\u2019 evaluation. Proceedings of the RO-MAN, Paris, France.","DOI":"10.1109\/ROMAN.2012.6343854"},{"key":"ref_51","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2022, July 12). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"107440","DOI":"10.1016\/j.knosys.2021.107440","article-title":"ViMantic, a distributed robotic architecture for semantic mapping in indoor environments","volume":"232","author":"Petkov","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1080\/00029890.1993.11990380","article-title":"Pick\u2019s Theorem","volume":"100","author":"Grunbaum","year":"1993","journal-title":"Am. Math. Mon."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5308\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:51:31Z","timestamp":1760140291000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,15]]},"references-count":54,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145308"],"URL":"https:\/\/doi.org\/10.3390\/s22145308","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,7,15]]}}}