{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:21:39Z","timestamp":1760059299482,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,9]],"date-time":"2025-06-09T00:00:00Z","timestamp":1749427200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This paper presents a learning-assisted approach for state estimation of quadruped robots using observations of proprioceptive sensors, including multiple inertial measurement units (IMUs). Specifically, one body IMU and four additional IMUs attached to each calf link of the robot are used for sensing the dynamics of the body and legs, in addition to joint encoders. The extended Kalman filter (KF) is employed to fuse sensor data to estimate the robot\u2019s states in the world frame and enhance the convergence of the extended KF (EKF). To circumvent the requirements for the measurements from the motion capture (mocap) system or other vision systems, the right-invariant EKF (RI-EKF) is extended to employ the foot IMU measurements for enhanced state estimation, and a learning-based approach is presented to estimate the vision system measurements for the EKF. One-dimensional convolutional neural networks (CNN) are leveraged to estimate required measurements using only the available proprioception data. Experiments on real data from a quadruped robot demonstrate that proprioception can be sufficient for state estimation. The proposed learning-assisted approach, which does not rely on data from vision systems, achieves competitive accuracy compared to EKF using mocap measurements and lower estimation errors than RI-EKF using multi-IMU measurements.<\/jats:p>","DOI":"10.3390\/info16060479","type":"journal-article","created":{"date-parts":[[2025,6,9]],"date-time":"2025-06-09T06:46:01Z","timestamp":1749451561000},"page":"479","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Learning-Assisted Multi-IMU Proprioceptive State Estimation for Quadruped Robots"],"prefix":"10.3390","volume":"16","author":[{"given":"Xuanning","family":"Liu","sequence":"first","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8773-926X","authenticated-orcid":false,"given":"Yajie","family":"Bao","sequence":"additional","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4463-7751","authenticated-orcid":false,"given":"Peng","family":"Cheng","sequence":"additional","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]},{"given":"Dan","family":"Shen","sequence":"additional","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]},{"given":"Zhengyang","family":"Fan","sequence":"additional","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]},{"given":"Hao","family":"Xu","sequence":"additional","affiliation":[{"name":"Department of Electrical and Biomedical Engineering, University of Nevada, Reno, NV 89557, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2656-9380","authenticated-orcid":false,"given":"Genshe","family":"Chen","sequence":"additional","affiliation":[{"name":"Intelligent Fusion Technology, Inc., Germantown, MD 20874, USA"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1109\/TRO.2020.3003464","article-title":"Motion planning for quadrupedal locomotion: Coupled planning, terrain mapping, and whole-body control","volume":"36","author":"Mastalli","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_2","unstructured":"Bloesch, M. (2017). State Estimation for Legged Robots-Kinematics, Inertial Sensing, and Computer Vision. [Ph.D. Thesis, ETH Zurich]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s10033-020-00485-9","article-title":"Mechanism, actuation, perception, and control of highly dynamic multilegged robots: A review","volume":"33","author":"He","year":"2020","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nobili, S., Camurri, M., Barasuol, V., Focchi, M., Caldwell, D., Semini, C., and Fallon, M. (2017, January 12\u201316). Heterogeneous sensor fusion for accurate state estimation of dynamic legged robots. Proceedings of the Robotics: Science and System XIII, Cambridge, MA, USA.","DOI":"10.15607\/RSS.2017.XIII.007"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1177\/0278364917728574","article-title":"Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback","volume":"36","author":"Bloesch","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhao, S., Zhang, H., Wang, P., Nogueira, L., and Scherer, S. (October, January 27). Super odometry: Imu-centric lidar-visual-inertial estimator for challenging environments. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9635862"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Fink, G., and Semini, C. (2020\u201324, January 24). Proprioceptive Sensor Fusion for Quadruped Robot State Estimation. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341521"},{"key":"ref_8","unstructured":"Lin, T.Y., Li, T., Tong, W., and Ghaffari, M. (2023). Proprioceptive invariant robot state estimation. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1109\/TRO.2012.2228309","article-title":"Dead Reckoning in a Dynamic Quadruped Robot Based on Multimodal Proprioceptive Sensory Information","volume":"29","author":"Reinstein","year":"2013","journal-title":"IEEE Trans. Robot."},{"key":"ref_10","first-page":"3736","article-title":"Kalman and extended kalman filters: Concept, derivation and properties","volume":"43","author":"Ribeiro","year":"2004","journal-title":"Inst. Syst. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Camurri, M., Ramezani, M., Nobili, S., and Fallon, M. (2020). Pronto: A multi-sensor state estimator for legged robots in real-world scenarios. Front. Robot. AI, 7.","DOI":"10.3389\/frobt.2020.00068"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"17","DOI":"10.7551\/mitpress\/9816.003.0008","article-title":"State estimation for legged robots-consistent fusion of leg kinematics and IMU","volume":"17","author":"Bloesch","year":"2013","journal-title":"Robotics"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8178","DOI":"10.1109\/LRA.2022.3186501","article-title":"Online kinematic calibration for legged robots","volume":"7","author":"Yang","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yang, S., Zhang, Z., Bokser, B., and Manchester, Z. (2023, January 1\u20135). Multi-IMU Proprioceptive Odometry for Legged Robots. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.","DOI":"10.1109\/IROS55552.2023.10342061"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1797","DOI":"10.1109\/TAC.2016.2594085","article-title":"The invariant extended Kalman filter as a stable observer","volume":"62","author":"Barrau","year":"2016","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hartley, R., Jadidi, M.G., Grizzle, J.W., and Eustice, R.M. (2018). Contact-aided invariant extended Kalman filtering for legged robot state estimation. arXiv.","DOI":"10.15607\/RSS.2018.XIV.050"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xavier, F.E., Burger, G., P\u00e9triaux, M., Deschaud, J.E., and Goulette, F. (2023, January 1\u20135). Multi-IMU Proprioceptive State Estimator for Humanoid Robots. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.","DOI":"10.1109\/IROS55552.2023.10341849"},{"key":"ref_18","unstructured":"Lin, T.Y., Zhang, R., Yu, J., and Ghaffari, M. (2022, January 14\u201318). Legged Robot State Estimation using Invariant Kalman Filtering and Learned Contact Events. Proceedings of the Conference on Robot Learning, PMLR, Auckland, New Zealand."},{"key":"ref_19","unstructured":"Buchanan, R., Camurri, M., Dellaert, F., and Fallon, M. (2022, January 14\u201318). Learning inertial odometry for dynamic legged robot state estimation. Proceedings of the Conference on Robot Learning, PMLR, Auckland, New Zealand."},{"key":"ref_20","unstructured":"Youm, D., Oh, H., Choi, S., Kim, H., and Hwangbo, J. (2024). Legged Robot State Estimation With Invariant Extended Kalman Filter Using Neural Measurement Network. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, Y., Bao, Y., Cheng, P., Shen, D., Chen, G., and Xu, H. (2024, January 21\u201325). Enhanced robot state estimation using physics-informed neural networks and multimodal proprioceptive data. Proceedings of the Sensors and Systems for Space Applications XVII, SPIE, National Harbor, MD, USA.","DOI":"10.1117\/12.3022666"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, S., Zhang, Z., Fu, Z., and Manchester, Z. (June, January 29). Cerberus: Low-drift visual-inertial-leg odometry for agile locomotion. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160486"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Teng, S., Mueller, M.W., and Sreenath, K. (June, January 30). Legged robot state estimation in slippery environments using invariant extended kalman filter with velocity update. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561313"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"20200209","DOI":"10.1098\/rsta.2020.0209","article-title":"Time-series forecasting with deep learning: A survey","volume":"379","author":"Lim","year":"2021","journal-title":"Philos. Trans. R. Soc. A"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Fan, Z., Cheng, P., Chen, H., Bao, Y., Pham, K., Blasch, E., Xu, H., and Chen, G. (2024, January 16\u201320). STAN: Spatial-Temporal Attention Based Inertial Navigation Transformer. Proceedings of the 37th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2024), Baltimore, MD, USA.","DOI":"10.33012\/2024.19903"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5653","DOI":"10.1109\/LRA.2020.3007421","article-title":"TLIO: Tight Learned Inertial Odometry","volume":"5","author":"Liu","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"10506","DOI":"10.1109\/TITS.2024.3381161","article-title":"Deep learning for inertial positioning: A survey","volume":"25","author":"Chen","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hong, S., Xu, Y., Khare, A., Priambada, S., Maher, K., Aljiffry, A., Sun, J., and Tumanov, A. (2020, January 6\u201310). HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual.","DOI":"10.1145\/3394486.3403212"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/479\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:48:40Z","timestamp":1760032120000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/479"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,9]]},"references-count":29,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["info16060479"],"URL":"https:\/\/doi.org\/10.3390\/info16060479","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2025,6,9]]}}}