{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T16:00:51Z","timestamp":1774281651131,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2019,11,5]],"date-time":"2019-11-05T00:00:00Z","timestamp":1572912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Jilin Province Key Technology and Development Program","award":["20190302077GX"],"award-info":[{"award-number":["20190302077GX"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51705185"],"award-info":[{"award-number":["51705185"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key Technologies R&amp;D Program of China","award":["2017YFC0601604"],"award-info":[{"award-number":["2017YFC0601604"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual-Inertial Odometry (VIO) is subjected to additional unobservable directions under the special motions of ground vehicles, resulting in larger pose estimation errors. To address this problem, a tightly-coupled Ackermann visual-inertial odometry (ACK-MSCKF) is proposed to fuse Ackermann error state measurements and the Stereo Multi-State Constraint Kalman Filter (S-MSCKF) with a tightly-coupled filter-based mechanism. In contrast with S-MSCKF, in which the inertial measurement unit (IMU) propagates the vehicle motion and then the propagation is corrected by stereo visual measurements, we successively update the propagation with Ackermann error state measurements and visual measurements after the process model and state augmentation. This way, additional constraints from the Ackermann measurements are exploited to improve the pose estimation accuracy. Both qualitative and quantitative experimental results evaluated under real-world datasets from an Ackermann steering vehicle lead to the following demonstration: ACK-MSCKF can significantly improve the pose estimation accuracy of S-MSCKF under the special motions of autonomous vehicles, and keep accurate and robust pose estimation available under different vehicle driving cycles and environmental conditions. This paper accompanies the source code for the robotics community.<\/jats:p>","DOI":"10.3390\/s19214816","type":"journal-article","created":{"date-parts":[[2019,11,7]],"date-time":"2019-11-07T02:48:31Z","timestamp":1573094911000},"page":"4816","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["ACK-MSCKF: Tightly-Coupled Ackermann Multi-State Constraint Kalman Filter for Autonomous Vehicle Localization"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0007-0549","authenticated-orcid":false,"given":"Fangwu","family":"Ma","sequence":"first","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}]},{"given":"Jinzhu","family":"Shi","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}]},{"given":"Yu","family":"Yang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}]},{"given":"Jinhang","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}]},{"given":"Kai","family":"Dai","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3129","DOI":"10.1109\/TITS.2018.2873101","article-title":"Synthesis of Robust Lane Keeping Systems: Impact of Controller and Design Parameters on System Performance","volume":"20","author":"Lee","year":"2018","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1016\/j.mechatronics.2010.06.002","article-title":"Kinematic parameter calibration of a car-like mobile robot to improve odometry accuracy","volume":"20","author":"Lee","year":"2010","journal-title":"Mechatronics"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Toledo, J., Pi\u00f1eiro, J., Arnay, R., Acosta, D., and Acosta, L. (2018). Improving odometric accuracy for an autonomous electric cart. Sensors, 18.","DOI":"10.3390\/s18010200"},{"key":"ref_4","unstructured":"Groves, P.D. (2013). Principles of GNSS, Inertial, and Multisensor Integrated Navigation Systems, Artech House."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xue, H., Fu, H., and Dai, B. (2019). IMU-Aided High-Frequency Lidar Odometry for Autonomous Driving. Appl. Sci., 9.","DOI":"10.3390\/app9071506"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhang, J., and Singh, S. (2015, January 26\u201330). Visual-lidar odometry and mapping: Low-drift, robust, and fast. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139486"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TPAMI.2017.2658577","article-title":"Direct sparse odometry","volume":"40","author":"Engel","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1109\/TRO.2016.2623335","article-title":"SVO: Semidirect visual odometry for monocular and multicamera systems","volume":"33","author":"Forster","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.robot.2017.09.010","article-title":"Keyframe-based monocular SLAM: Design, survey, and future directions","volume":"98","author":"Younes","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1897","DOI":"10.1186\/s40064-016-3573-7","article-title":"Review of visual odometry: Types, approaches, challenges, and applications","volume":"5","author":"Aqel","year":"2016","journal-title":"Springerplus"},{"key":"ref_12","first-page":"37","article-title":"Visual SLAM and Structure from Motion in Dynamic Environments: A Survey","volume":"51","author":"Saputra","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Aladem, M., and Rawashdeh, S. (2018). Lightweight visual odometry for autonomous mobile robots. Sensors, 18.","DOI":"10.3390\/s18092837"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Huang, G. (2019). Visual-inertial navigation: A concise review. arXiv.","DOI":"10.1109\/ICRA.2019.8793604"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1080\/01691864.2015.1057616","article-title":"A review of visual inertial odometry from filtering and optimisation perspectives","volume":"29","author":"Gui","year":"2015","journal-title":"Adv. Robot."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/LRA.2017.2653359","article-title":"Visual-Inertial Monocular SLAM with Map Reuse","volume":"2","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Von Stumberg, L., Usenko, V., and Cremers, D. (2018, January 21\u201325). Direct sparse visual-inertial odometry using dynamic marginalization. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8462905"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1109\/TIV.2018.2804168","article-title":"Vehicle Positioning in GNSS-Deprived Urban Areas by Stereo Visual-Inertial Odometry","volume":"3","author":"Ramezani","year":"2018","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A multi-state constraint Kalman filter for vision-aided inertial navigation. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lynen, S., Achtelik, M.W., Weiss, S., Chli, M., and Siegwart, R. (2013, January 3\u20137). A robust and modular multi-sensor fusion approach applied to MAV navigation. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696917"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bloesch, M., Omari, S., Hutter, M., and Siegwart, R. (October, January 28). Robust visual inertial odometry using a direct EKF-based approach. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353389"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1109\/LRA.2018.2793349","article-title":"Robust stereo visual inertial odometry for fast autonomous flight","volume":"3","author":"Sun","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Huai, Z., and Huang, G. (2018, January 1\u20135). Robocentric visual-inertial odometry. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593643"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual\u2013inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2014","journal-title":"Int. J. Robot. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, Y., Zhao, J., Guo, Y., He, W., and Yuan, K. (2018). Pl-vio: Tightly-coupled monocular visual-inertial odometry using point and line features. Sensors, 18.","DOI":"10.3390\/s18041159"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Usenko, V., Demmel, N., Schubert, D., St\u00fcckler, J., and Cremers, D. (2019). Visual-Inertial Mapping with Non-Linear Factor Recovery. arXiv.","DOI":"10.1109\/LRA.2019.2961227"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Delmerico, J., and Scaramuzza, D. (2018, January 21\u201325). A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460664"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3433","DOI":"10.1109\/TITS.2017.2685433","article-title":"A Framework for Fast and Robust Visual Odometry","volume":"18","author":"Wu","year":"2017","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_30","unstructured":"Wu, K.J., and Roumeliotis, S.I. (2016). Unobservable Directions of VINS under Special Motions, Department of Computer Science & Engineering, University of Minnesota."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shan, Z., Li, R., and Schwertfeger, S. (2019). RGBD-Inertial Trajectory Estimation and Mapping for Ground Robots. Sensors, 19.","DOI":"10.3390\/s19102251"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Schmid, K., Tomic, T., Ruess, F., Hirschm\u00fcller, H., and Suppa, M. (2013, January 3\u20137). Stereo vision based indoor\/outdoor navigation for flying robots. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696922"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9699","DOI":"10.1109\/JSEN.2018.2873055","article-title":"SE(2)-Constrained Visual Inertial Fusion for Ground Vehicles","volume":"18","author":"Zheng","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wu, K.J., Guo, C.X., Georgiou, G., and Roumeliotis, S.I. (June, January 29). Vins on wheels. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989603"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dang, Z., Wang, T., and Pang, F. (2018, January 12\u201315). Tightly-coupled Data Fusion of VINS and Odometer Based on Wheel Slip Estimation. Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ROBIO.2018.8665337"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hong, E., and Lim, J. (2018). Visual-Inertial Odometry with Robust Initialization and Online Scale Estimation. Sensors, 18.","DOI":"10.20944\/preprints201809.0509.v1"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Houseago, C., Bloesch, M., and Leutenegger, S. (2019, January 20\u201324). KO-Fusion: Dense Visual SLAM with Tightly-Coupled Kinematic and Odometric Tracking. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793471"},{"key":"ref_38","first-page":"419","article-title":"GeRoNa: Generic Robot Navigation","volume":"95","author":"Buck","year":"2018","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_39","unstructured":"(2019, October 08). ACK-MSCKF ROS Package. Available online: https:\/\/github.com\/qdensh\/ACK-MSCKF."},{"key":"ref_40","unstructured":"Breckenridge, W.G. (1999). Quaternions proposed standard conventions. Jet Propuls. Lab. Pasadena CA Interoff. Memo. IOM, 343\u2013379."},{"key":"ref_41","first-page":"2005","article-title":"Indirect Kalman filter for 3D attitude estimation","volume":"2","author":"Trawny","year":"2005","journal-title":"Univ. Minn. Dept. Comp. Sci. Eng. Tech. Rep"},{"key":"ref_42","unstructured":"Sola, J. (2017). Quaternion kinematics for the error-state Kalman filter. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Scaramuzza, D., Fraundorfer, F., Pollefeys, M., and Siegwart, R. (October, January 29). Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.","DOI":"10.1109\/ICCV.2009.5459294"},{"key":"ref_44","unstructured":"Kevin, M.L., and Park, F.C. (2017). Modern Robotics: Mechanics, Planning, and Control, Cambridge University Press."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rajamani, R. (2012). Vehicle Dynamics and Control, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4614-1433-9"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"14133","DOI":"10.3390\/s131014133","article-title":"Multi sensor fusion framework for indoor-outdoor localization of limited resource mobile robots","volume":"13","author":"Soriano","year":"2013","journal-title":"Sensors"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Furgale, P., Rehder, J., and Siegwart, R. (2013, January 3\u20137). Unified temporal and spatial calibration for multi-sensor systems. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696514"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1454","DOI":"10.1007\/s12555-014-0157-6","article-title":"Simplified epipolar geometry for real-time monocular visual odometry on roads","volume":"13","author":"Choi","year":"2015","journal-title":"Int. J. Control Autom. Syst."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/TASE.2016.2550621","article-title":"Monocular visual-inertial state estimation with online initialization and camera-IMU extrinsic calibration","volume":"14","author":"Yang","year":"2016","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.2514\/1.28949","article-title":"Averaging Quaternions","volume":"30","author":"Markley","year":"2007","journal-title":"J. Guid. Control Dyn."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1177\/0278364915620033","article-title":"The EuRoC micro aerial vehicle datasets","volume":"35","author":"Burri","year":"2016","journal-title":"Int. J. Robot. Res."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? the kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Pfrommer, B., Sanket, N., Daniilidis, K., and Cleveland, J. (June, January 29). Penncosyvio: A challenging visual inertial odometry benchmark. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989443"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Zhang, Z., and Scaramuzza, D. (2018, January 1\u20135). A tutorial on quantitative trajectory evaluation for visual (-inertial) odometry. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593941"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1365","DOI":"10.1109\/TRO.2008.2006706","article-title":"ISAM: Incremental Smoothing and Mapping","volume":"24","author":"Kaess","year":"2008","journal-title":"IEEE Trans. Robot."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1177\/0278364911430419","article-title":"ISAM2: Incremental smoothing and mapping using the Bayes tree","volume":"31","author":"Kaess","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Liu, H., Chen, M., Zhang, G., Bao, H., and Bao, Y. (2018, January 18\u201323). Ice-ba: Incremental, consistent and efficient bundle adjustment for visual-inertial slam. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00211"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/s40328-019-00254-8","article-title":"Comparative analysis of robust extended Kalman filter and incremental smoothing for UWB\/PDR fusion positioning in NLOS environments","volume":"54","author":"Li","year":"2019","journal-title":"Acta Geod. Geophys."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/21\/4816\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:32:05Z","timestamp":1760189525000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/21\/4816"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,5]]},"references-count":59,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2019,11]]}},"alternative-id":["s19214816"],"URL":"https:\/\/doi.org\/10.3390\/s19214816","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,5]]}}}