{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T03:11:02Z","timestamp":1768792262679,"version":"3.49.0"},"reference-count":49,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T00:00:00Z","timestamp":1636416000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The visual-inertial simultaneous localization and mapping (SLAM) is a feasible indoor positioning system that combines the visual SLAM with inertial navigation. There are accumulated drift errors in inertial navigation due to the state propagation and the bias of the inertial measurement unit (IMU) sensor. The visual-inertial SLAM can correct the drift errors via loop detection and local pose optimization. However, if the trajectory is not a closed loop, the drift error might not be significantly reduced. This paper presents a novel pedestrian dead reckoning (PDR)-aided visual-inertial SLAM, taking advantage of the enhanced vanishing point (VP) observation. The VP is integrated into the visual-inertial SLAM as an external observation without drift error to correct the system drift error. Additionally, the estimated trajectory\u2019s scale is affected by the IMU measurement errors in visual-inertial SLAM. Pedestrian dead reckoning (PDR) velocity is employed to constrain the double integration result of acceleration measurement from the IMU. Furthermore, to enhance the proposed system\u2019s robustness and the positioning accuracy, the local optimization based on the sliding window and the global optimization based on the segmentation window are proposed. A series of experiments are conducted using the public ADVIO dataset and a self-collected dataset to compare the proposed system with the visual-inertial SLAM. Finally, the results demonstrate that the proposed optimization method can effectively correct the accumulated drift error in the proposed visual-inertial SLAM system.<\/jats:p>","DOI":"10.3390\/s21227428","type":"journal-article","created":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T21:39:07Z","timestamp":1636493947000},"page":"7428","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["An Enhanced Pedestrian Visual-Inertial SLAM System Aided with Vanishing Point in Indoor Environments"],"prefix":"10.3390","volume":"21","author":[{"given":"Wennan","family":"Chai","sequence":"first","affiliation":[{"name":"College of Sino-German Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2647-0740","authenticated-orcid":false,"given":"Chao","family":"Li","sequence":"additional","affiliation":[{"name":"College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"given":"Mingyue","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Sino-German Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"given":"Zhen","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"given":"Hao","family":"Yuan","sequence":"additional","affiliation":[{"name":"College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"given":"Fanyu","family":"Lin","sequence":"additional","affiliation":[{"name":"College of Sino-German Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China"}]},{"given":"Qingdang","family":"Li","sequence":"additional","affiliation":[{"name":"College of Sino-German Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"905","DOI":"10.1007\/s10514-021-09992-7","article-title":"VIR-SLAM: Visual, inertial, and ranging SLAM for single and multi-robot systems","volume":"45","author":"Cao","year":"2021","journal-title":"Auton. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4649","DOI":"10.1109\/LRA.2020.3003275","article-title":"Distributed Consistent Multi-Robot Semantic Localization and Mapping","volume":"5","author":"Tchuiev","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"25442","DOI":"10.1109\/ACCESS.2020.2970238","article-title":"A Mobile Robot Visual SLAM System With Enhanced Semantics Segmentation","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2576","DOI":"10.1109\/LRA.2019.2904733","article-title":"RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes","volume":"4","author":"Sun","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"8675","DOI":"10.1007\/s11042-015-2780-5","article-title":"Local optimized and scalable frame-to-model SLAM","volume":"75","author":"Li","year":"2015","journal-title":"Multimedia Tools Appl."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.robot.2015.04.010","article-title":"Robust linear pose graph-based SLAM","volume":"72","author":"Cheng","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6264","DOI":"10.1364\/AO.424280","article-title":"Panoramic Annular SLAM with Loop Closure and Global Optimization","volume":"60","author":"Chen","year":"2021","journal-title":"Appl. Opt."},{"key":"ref_8","unstructured":"Lee, J.-K., and Yoon, K.-J. (2015, January 7\u201312). Real-Time Joint Estimation of Camera Orientation and Vanishing Points. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"994","DOI":"10.1109\/LRA.2018.2793357","article-title":"Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High-Speed Scenarios","volume":"3","author":"Vidal","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_10","unstructured":"Edgar, S., and Jean-Bernard, H. (2018, January 21\u201325). Bayesian Scale Estimation for Monocular SLAM Based on Generic Object Detection for Correcting Scale Drift. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Qin, T., Zheng, Y., Chen, T., Chen, Y., and Su, Q. (June, January 30). RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561663"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, P., Zhang, G., Zhou, J., Yao, R., and Zhang, X. (2019, January 26\u201328). Study on Slam Algorithm Based on Object Detection in Dynamic Scene. Proceedings of the 2019 International Conference on Advanced Mechatronic Systems (ICAMechS), Kusatsu, Japan.","DOI":"10.1109\/ICAMechS.2019.8861669"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual\u2013inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/TASE.2016.2550621","article-title":"Monocular Visual\u2013Inertial State Estimation With Online Initialization and Camera\u2013IMU Extrinsic Calibration","volume":"14","author":"Yang","year":"2017","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/LRA.2017.2653359","article-title":"Visual-Inertial Monocular SLAM With Map Reuse","volume":"2","author":"Tardos","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shen, S., Michael, N., and Kumar, V. (2011, January 9\u201313). Autonomous Multi-Floor Indoor Navigation with a Computationally Constrained MAV. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980357"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Shen, S., Michael, N., and Kumar, V. (2015, January 26\u201330). Tightly-Coupled Monocular Visual-Inertial Fusion for Autonomous Flight of Rotorcraft MAVs. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139939"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Engel, J., Schops, T., and Cremers, D. (2014, January 6\u201312). LSD-SLAM: Large-Scale Direct Monocular SLAM. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10605-2_54"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","author":"Raul","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1007\/s10514-016-9548-2","article-title":"Low-drift and real-time lidar odometry and mapping","volume":"41","author":"Zhang","year":"2017","journal-title":"Auton. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chen, S., Zhou, B., Jiang, C., Xue, W., and Li, Q. (2021). A LiDAR\/Visual SLAM Backend with Loop Closure Detection and Graph Optimization. Remote. Sens., 13.","DOI":"10.3390\/rs13142720"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7057","DOI":"10.1109\/LRA.2021.3096741","article-title":"Direct Near-Infrared-Depth Visual SLAM with Active Lighting","volume":"6","author":"Kong","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1109\/TVT.2015.2388780","article-title":"StructSLAM: Visual SLAM With Building Structure Lines","volume":"64","author":"Zhou","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Camposeco, F., and Pollefeys, M. (2015, January 26\u201330). Using vanishing points to improve visual-inertial odometry. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139926"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4899","DOI":"10.1109\/LRA.2021.3070298","article-title":"Hybrid Monocular SLAM Using Double Window Optimization","volume":"6","author":"Luo","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1109\/MRA.2012.2182810","article-title":"Visual Odometry: Part II: Matching, Robustness, Optimization, and Applications","volume":"19","author":"Fraundorfer","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhao, S., Zhang, H., Wang, P., Nogueira, L., and Scherer, S. (2021). Super Odometry IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments. arXiv.","DOI":"10.1109\/IROS51168.2021.9635862"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Khattak, S., Nguyen, H., Mascarich, F., Dang, T., and Alexis, K. (2020, January 1\u20134). Complementary Multi\u2013Modal Sensor Fusion for Resilient Robot Pose Estimation in Subterranean Environments. Proceedings of the 2020 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.","DOI":"10.1109\/ICUAS48674.2020.9213865"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"68","DOI":"10.3389\/frobt.2020.00068","article-title":"Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios","volume":"7","author":"Camurri","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1670","DOI":"10.1109\/LRA.2021.3059564","article-title":"Bidirectional Trajectory Computation for Odometer-Aided Visual-Inertial SLAM","volume":"6","author":"Liu","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5191","DOI":"10.1109\/LRA.2021.3068640","article-title":"DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM","volume":"6","author":"Bescos","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_32","first-page":"382","article-title":"Camera Calibration from Vanishing Points in Image ofArchitectural Scenes","volume":"38","author":"Cipolla","year":"1999","journal-title":"Br. Mach. Vis. Conf."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Khac, C., Choi, Y., Park, J., and Jung, H.-Y. (2021). A Robust Road Vanishing Point Detection Adapted to the Real-world Driving Scenes. Sensors, 21.","DOI":"10.3390\/s21062133"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"163015","DOI":"10.1109\/ACCESS.2020.3019318","article-title":"Vanishing Point Detection and Rail Segmentation Based on Deep Multi-Task Learning","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2004). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kocur, V., and Ftacnik, M. (2021). Traffic Camera Calibration via Vehicle Vanishing Point Detection. arXiv.","DOI":"10.1007\/978-3-030-86383-8_50"},{"key":"ref_37","first-page":"295","article-title":"Geometric Context and Orientation Map Combination for Indoor Corridor Modeling Using a Single Image","volume":"XLI-B4","author":"Jahromi","year":"2016","journal-title":"ISPRS"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1049\/iet-cvi.2014.0187","article-title":"Efficient vanishing point detection method in complex urban road environments","volume":"9","author":"Ding","year":"2015","journal-title":"IET Comput. Vis."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2906","DOI":"10.1109\/JSEN.2014.2382568","article-title":"SmartPDR: Smartphone-Based Pedestrian Dead Reckoning for Indoor Localization","volume":"15","author":"Kang","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Forster, C., Carlone, M., Dellaert, F., and Scaramuzza, D. (2015). IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation, Georgia Institute of Technology.","DOI":"10.15607\/RSS.2015.XI.006"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ho, N.-H., Truong, P.H., and Jeong, G.-M. (2016). Step-Detection and Adaptive Step-Length Estimation for Pedestrian Dead-Reckoning at Various Walking Speeds Using a Smartphone. Sensors, 16.","DOI":"10.3390\/s16091423"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s11370-015-0166-1","article-title":"RGB-D SLAM using vanishing point and door plate information in corridor environment","volume":"8","author":"Ji","year":"2015","journal-title":"Intell. Serv. Robot."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/1687814019884767","article-title":"Vanishing point detection in corridor for autonomous mobile robots using monocular low-resolution fisheye vision","volume":"11","author":"Ni","year":"2019","journal-title":"Adv. Mech. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Cort\u00e9s, S., Solin, A., Rahtu, E., and Kannala, J. (2018, January 8\u201314). ADVIO: An Authentic Dataset for Visual-Inertial Odometry. Proceedings of the 15th European Conference, Munich, Germany.","DOI":"10.1007\/978-3-030-01249-6_26"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018, January 1\u20135). DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593691"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sheng, C., Pan, S., Gao, W., Tan, Y., and Zhao, T. (2020). Dynamic-DSO: Direct Sparse Odometry Using Objects Semantic Information for Dynamic Environments. Appl. Sci., 10.","DOI":"10.3390\/app10041467"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Solin, A., Cortes, S., Rahtu, E., and Kannala, J. (2018, January 10\u201313). Inertial Odometry on Handheld Smartphones. Proceedings of the 2018 21st International Conference on Information Fusion (FUSION), Cambridge, UK.","DOI":"10.23919\/ICIF.2018.8455482"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_49","unstructured":"Qin, T., Pan, J., Cao, S., and S, S. (2019). A General Optimization-Based Framework for Local Odometry Estimation with Multiple Sensors. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/22\/7428\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:27:49Z","timestamp":1760167669000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/22\/7428"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,9]]},"references-count":49,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["s21227428"],"URL":"https:\/\/doi.org\/10.3390\/s21227428","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,9]]}}}