{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:11:11Z","timestamp":1760231471435,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T00:00:00Z","timestamp":1663718400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Basic Scientific Research Project of Colleges and Universities from the Educational Department of Liaoning Province","award":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"],"award-info":[{"award-number":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"]}]},{"name":"Liaoning Doctor Scientific Research Initial Fund in 2022 from Department of Science &amp; Technology of Liaoning Province","award":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"],"award-info":[{"award-number":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"]}]},{"name":"Research support project for introducing high-level talents of Shenyang Ligong University","award":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"],"award-info":[{"award-number":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"]}]},{"name":"Research and innovation team building project of Shenyang Ligong University","award":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"],"award-info":[{"award-number":["LJKZ0258","2022-BS-187","1010147001012","SYLUTD202106"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Loop closure detection based on a residual network (ResNet) and a capsule network (CapsNet) is proposed to address the problems of low accuracy and poor robustness for mobile robot simultaneous localization and mapping (SLAM) in complex scenes. First, the residual network of a feature coding strategy is introduced to extract the shallow geometric features and deep semantic features of images, reduce the amount of image noise information, accelerate the convergence speed of the model, and solve the problems of gradient disappearance and network degradation of deep neural networks. Then, the dynamic routing mechanism of the capsule network is optimized through the entropy peak density, and a vector is used to represent the spatial position relationship between features, which can improve the ability of image feature extraction and expression to optimize the overall performance of networks. Finally, the optimized residual network and capsule network are fused to retain the differences and correlations between features, and the global feature descriptors and feature vectors are combined to calculate the similarity of image features for loop closure detection. The experimental results show that the proposed method can achieve loop closure detection for mobile robots in complex scenes, such as view changes, illumination changes, and dynamic objects, and improve the accuracy and robustness of mobile robot SLAM.<\/jats:p>","DOI":"10.3390\/s22197137","type":"journal-article","created":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T23:07:55Z","timestamp":1663888075000},"page":"7137","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Loop Closure Detection Based on Residual Network and Capsule Network for Mobile Robot"],"prefix":"10.3390","volume":"22","author":[{"given":"Xin","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Shenyang Ligong University, Shenyang 110159, China"},{"name":"Shenyang Institute of Computing Technology Co., Ltd., Chinese Academy of Sciences, Shenyang 110168, China"},{"name":"Software College, Northeastern University, Shenyang 110169, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liaomo","family":"Zheng","sequence":"additional","affiliation":[{"name":"Shenyang Institute of Computing Technology Co., Ltd., Chinese Academy of Sciences, Shenyang 110168, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9870-8925","authenticated-orcid":false,"given":"Zhenhua","family":"Tan","sequence":"additional","affiliation":[{"name":"Software College, Northeastern University, Shenyang 110169, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suo","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Shenyang Ligong University, Shenyang 110159, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TRO.2016.2624754","article-title":"Past, Present, and Future of Simultaneous Localizationand Mapping: Toward the Robust-Perception Age","volume":"32","author":"Cadena","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TPAMI.2017.2658577","article-title":"Direct sparse odometry","volume":"40","author":"Engel","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1109\/LRA.2018.2800113","article-title":"Maplab: An open framework for research in visual-inertial mapping and localization","volume":"3","author":"Schneider","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1109\/LRA.2018.2889156","article-title":"Loosely-coupled semi-direct monocular slam","volume":"44","author":"Lee","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"107193","DOI":"10.1016\/j.patcog.2019.107193","article-title":"UcoSLAM:Simultaneous Localization and Mapping by Fusion of KeyPoints and Squared Planar Markers","volume":"101","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1007\/s00371-019-01720-8","article-title":"Integrating global and local image features for enhanced loop closure detection in RGB-D SLAM systems","volume":"36","author":"Guclu","year":"2019","journal-title":"Vis. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kuo, J., Muglikar, M., and Scaramuzza, D. (August, January 31). Redesigning SLAM for Arbitrary Multi-CameraSystems. Proceedings of the 2020 IEEE International Conference on Robotics and Automation, Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197553"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tsintotas, K.A., Bampis, L., and Gasteratos, A. (2022). The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2022.3175656"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A Versatile and Accurate Monocular SLAM System","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2:An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","first-page":"91","article-title":"Distinctive image features from scale-invariant key points","volume":"20","author":"Lowe","year":"2003","journal-title":"Int. J. Comput. Vis."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Gool, L.V. (2006). SURF: Speeded up robust features. European Conference on Computer Vision, Springer.","DOI":"10.1007\/11744023_32"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2084","DOI":"10.1109\/LRA.2021.3060729","article-title":"Resolving Place Recognition Inconsistencies Using Intra-Set Similarities","volume":"6","author":"Neubert","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1023\/A:1011139631724","article-title":"Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope","volume":"42","author":"Oliva","year":"2001","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TRO.2013.2242375","article-title":"Appearance-based loop closure detection for online large-scale and long-term operation","volume":"29","author":"Labbe","year":"2013","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10514-015-9516-2","article-title":"Unsupervised learning to detect loops using deep neural networks for visual SLAM system","volume":"41","author":"Gao","year":"2017","journal-title":"Auton. Robot."},{"key":"ref_18","first-page":"1","article-title":"BoCNF:efficient image matching with Bag of ConvNet features for scalable and robust visual place recognition","volume":"42","author":"Hou","year":"2017","journal-title":"Auton. Robot."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"S\u00fcnderhauf, N., Shirazi, S., Dayoub, F., Upcroft, B., and Milford, M. (October, January 28). On the performance of convnet features for place recognition. Proceedings of the 2015 IEEE Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353986"},{"key":"ref_20","first-page":"1","article-title":"Very Deep convolutional networks for large-scale image recognition","volume":"6","author":"Simonyan","year":"2014","journal-title":"Comput. Vis. Pattern Recognit."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4407","DOI":"10.1109\/LRA.2018.2869640","article-title":"Vlocnet++: Deep multitask learning for semantic visuallocalization and odometry","volume":"3","author":"Radwan","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, A., Wang, M., Wu, H., Jiang, K., and Iwahori, Y. (2020). A novel LiDAR data classification algorithm combined capsnet with resnet. Sensors, 20.","DOI":"10.3390\/s20041151"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"109608","DOI":"10.1016\/j.ejrad.2021.109608","article-title":"3-D Res-CapsNet convolutional neural network on automated breast ultrasound tumor diagnosis","volume":"138","author":"Xiang","year":"2021","journal-title":"Eur. J. Radiol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"107851","DOI":"10.1016\/j.patcog.2021.107851","article-title":"CapsNet regularization and its conjugation with ResNet for signature identification","volume":"120","author":"Jampour","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.chaos.2021.110749","article-title":"Using Handpicked Features in Conjunction with ResNet-50 for Improved Detection of COVID-19 from Chest X-ray Images","volume":"145","author":"Rajpal","year":"2021","journal-title":"Chaos Solitons Fractals"},{"key":"ref_27","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Identity mappings in deep residual networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1109\/TPAMI.2011.235","article-title":"Aggregating Local Image Descriptors into Compact Codes","volume":"34","author":"Perronnin","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1109\/TPAMI.2017.2711011","article-title":"NetVLAD: CNN Architecture for Weakly Supervised Place Recognition","volume":"40","author":"Arandjelovic","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhong, Y., Arandjelovi, R., and Zisserman, A. (2018, January 2\u20136). GhostVLAD for set-based face recognition. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.","DOI":"10.1007\/978-3-030-20890-5_3"},{"key":"ref_31","unstructured":"Sabour, S., Frosst, N., and Hinton, G.E. (2017, January 4\u20139). Dynamic routing between capsules. Proceedings of the 31st International Conference on Neural Information Processing Systems, New York, NY, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1080\/2150704X.2020.1766722","article-title":"Using Siamese Capsule Networks for Remote Sensing Scene Classification","volume":"11","author":"Zhou","year":"2020","journal-title":"Remote Sens. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/s00138-021-01221-6","article-title":"Gabor capsule network with preprocessing blocks for the recognition of complex images","volume":"32","author":"Yu","year":"2021","journal-title":"Mach. Vis. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"79876","DOI":"10.1109\/ACCESS.2020.2990700","article-title":"Multi-lane capsule network for classifying images with complex background","volume":"8","author":"Chang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"6466","DOI":"10.1109\/TII.2020.2964117","article-title":"A robust weight-shared capsule network for intelligent machinery fault diagnosis","volume":"16","author":"Huang","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2145","DOI":"10.1109\/TGRS.2018.2871782","article-title":"Capsule networks for hyperspectral image classification","volume":"57","author":"Paoletti","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1492","DOI":"10.1126\/science.1242072","article-title":"Clustering by fast search and find of density peaks","volume":"344","author":"Rodriguez","year":"2014","journal-title":"Science"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_39","unstructured":"Glover, A. (2014, March 09). Gardens Point Walking Dataset. 2014. Available online: https:\/\/wiki.qut.edu.au\/display\/cyphy\/Open+datasets+and+software."},{"key":"ref_40","unstructured":"Hernan, B., Daniel, H., and Takeo, K. (2011, July 28). The CMU Visual Localization Dataset. 2011. Available online: http:\/\/3dvis.ri.cmu.edu\/datasets\/localization."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_42","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, SN, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chatfield, K., Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 1\u20135). Return of the Devilin the Details: Delving Deep into Convolutional Nets. Proceedings of the British Machine Vision Conference (BMVC), Nottingham, UK.","DOI":"10.5244\/C.28.6"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_46","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7\u201312). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Montreal, QC, USA."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (VOC) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_48","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50\u00d7 fewer parameters and <0.5 MB model size. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7137\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:36:09Z","timestamp":1760142969000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,21]]},"references-count":48,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197137"],"URL":"https:\/\/doi.org\/10.3390\/s22197137","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,9,21]]}}}