{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:11:36Z","timestamp":1760238696681,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2020,8,19]],"date-time":"2020-08-19T00:00:00Z","timestamp":1597795200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61603035"],"award-info":[{"award-number":["61603035"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>To achieve a high precision estimation of indoor robot motion, a tightly coupled RGB-D visual-inertial SLAM system is proposed herein based on multiple features. Most of the traditional visual SLAM methods only rely on points for feature matching and they often underperform in low textured scenes. Besides point features, line segments can also provide geometrical structure information of the environment. This paper utilized both points and lines in low-textured scenes to increase the robustness of RGB-D SLAM system. In addition, we implemented a fast initialization process based on the RGB-D camera to improve the real-time performance of the proposed system and designed a new backend nonlinear optimization framework. By minimizing the cost function formed by the pre-integrated IMU residuals and re-projection errors of points and lines in sliding windows, the state vector is optimized. The experiments evaluated on public datasets show that our system achieves higher accuracy and robustness on trajectories and in pose estimation compared with several state-of-the-art visual SLAM systems.<\/jats:p>","DOI":"10.3390\/s20174666","type":"journal-article","created":{"date-parts":[[2020,8,19]],"date-time":"2020-08-19T09:22:31Z","timestamp":1597828951000},"page":"4666","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multi-Feature Nonlinear Optimization Motion Estimation Based on RGB-D and Inertial Fusion"],"prefix":"10.3390","volume":"20","author":[{"given":"Xiongwei","family":"Zhao","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}]},{"given":"Cunxiao","family":"Miao","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}]},{"given":"He","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ma, S., Bai, X., Wang, Y., and Fang, R. (2019). Robust Stereo Visual-Inertial Odometry Using Nonlinear Optimization. Sensors, 19.","DOI":"10.3390\/s19173747"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ortega, L., Medina, D., Vil\u00e0-Valls, J., Vincent, F., and Chaumette, E. (2020). Positioning Performance Limits of GNSS Meta-Signals and HO-BOC Signals. Sensors, 20.","DOI":"10.3390\/s20123586"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hong, E., and Lim, J. (2018). Visual-Inertial Odometry with Robust Initialization and Online Scale Estimation. Sensors, 18.","DOI":"10.20944\/preprints201809.0509.v1"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Fang, W., Zheng, L., Deng, H., and Zhang, H. (2017). Real-time motion tracking for mobile augmented\/virtual reality using adaptive visual-inertial fusion. Sensors, 17.","DOI":"10.3390\/s17051037"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Falquez, J.M., Kasper, M., and Sibley, G. (2016, January 9\u201314). Inertial aided dense & semi-dense methods for robust direct visual odometry. Proceedings of the 29th IEEE\/RSJ International Conference on Intelligent Robots & Systems, Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759530"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Liu, H., Chen, M., Zhang, G., Bao, H., and Bao, Y. (2018, January 18\u201322). Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00211"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A Multi-State Constraint Kalman Filter for Vision-Aided Inertial Navigation. Proceedings of the IEEE International Conference on Robotics and Automation, Rome, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual-inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2016.2597321","article-title":"On-manifold preintegration for real-time visual-inertial odometry","volume":"33","author":"Forster","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"VINS-MONO: A robust and versatile monocular visual-inertial state estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ling, Y., Liu, H., Zhu, X., Jiang, J., and Liang, B. (2018, January 5\u20138). RGB-D inertial odometry for indoor robot via Keyframe-based nonlinear optimization. Proceedings of the IEEE International Conference on Mechatronics and Automation (ICMA), Changchun, China.","DOI":"10.1109\/ICMA.2018.8484687"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, C., Yuan, J., and Xie, L. (2017, January 10\u201312). Non-iterative SLAM. Proceedings of the 18th International Conference on Advanced Robotics (ICAR), Hong Kong, China.","DOI":"10.1109\/ICAR.2017.8023500"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"He, Y., Zhao, J., Guo, Y., He, W., and Yuan, K. (2018). PL-VIO: Tightly-coupled monocular visual\u2013inertial odometry using point and line features. Sensors, 18.","DOI":"10.3390\/s18041159"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fu, Q., Yu, H., Wang, J., Peng, X., Sun, W., Sun, M., and Lai, L. (2019). A Robust RGB-D SLAM System with Points and Lines for Low Texture Indoor Environments. IEEE Sens. J., 99.","DOI":"10.1109\/JSEN.2019.2927405"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, S., and Han, B. (2019). RGB-D visual odometry with point and line features in dynamic environment. J. Phys. Conf. Ser., 1303.","DOI":"10.1088\/1742-6596\/1303\/1\/012126"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lu, Y., and Song, D. (2015, January 7\u201313). Robust RGB-D odometry using point and line features. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.448"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1023\/B:VISI.0000013092.07433.82","article-title":"The 3D line motion matrix and alignment of line reconstructions","volume":"57","author":"Bartoli","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lu, X., Yao, J., Li, K., and Li, L. (2015, January 27\u201330). CannyLines: A parameter-free line segment detector. Proceedings of the IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.","DOI":"10.1109\/ICIP.2015.7350850"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"35","DOI":"10.5201\/ipol.2012.gjmr-lsd","article-title":"LSD: A line segment detector","volume":"2","author":"Jakubowicz","year":"2012","journal-title":"IPOL"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"794","DOI":"10.1016\/j.jvcir.2013.05.006","article-title":"An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency","volume":"24","author":"Zhang","year":"2013","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1109\/TRO.2015.2489498","article-title":"Building a 3-D line-based map using stereo SLAM","volume":"31","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"12816","DOI":"10.3390\/s150612816","article-title":"Tightly-coupled stereo visual-inertial navigation using point and line features","volume":"15","author":"Kong","year":"2015","journal-title":"Sensors"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1109\/TIP.2010.2068555","article-title":"A novel 3-D color histogram equalization method with uniform 1-D gray scale histogram","volume":"20","author":"Han","year":"2011","journal-title":"IEEE Trans. Image Process"},{"key":"ref_24","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, BC, Canada."},{"key":"ref_25","unstructured":"Shi, J., and Tomasi, C. (1994, January 21\u201323). Good features to track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Karthik, O.S., Varun, D., and Ramasangu, H. (2016, January 16\u201318). Localized Harris-FAST interest point detector. Proceedings of the IEEE Annual India Conference (INDICON), Bangalore, India.","DOI":"10.1109\/INDICON.2016.7839107"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shen, S., Michael, N., and Kumar, V. (2015, January 26\u201330). Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139939"},{"key":"ref_28","unstructured":"(2018, April 09). Ceres Solver Home Page. Available online: http:\/\/ceres-solver.org."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1002\/rob.20360","article-title":"Sliding window filter with application to planetary landing","volume":"27","author":"Sibley","year":"2010","journal-title":"J. Field Robot."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/TRO.2012.2197158","article-title":"Bags of binary words for fast place recognition in image sequences","volume":"28","author":"Tardos","year":"2012","journal-title":"IEEE Trans. Robot."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shan, Z., Li, R., and Schwertfeger, S. (2019). RGBD-inertial trajectory estimation and mapping for ground robots. Sensors, 19.","DOI":"10.3390\/s19102251"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Shi, X., Li, D., Zhao, P., Tian, Q., Tian, Y., Long, Q., Zhu, C., Song, J., Qiao, F., and Song, L. (2019). Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM. arXiv.","DOI":"10.1109\/ICRA40945.2020.9196638"},{"key":"ref_33","unstructured":"(2018, December 06). MichaelGrupp GitHub Repository. Available online: https:\/\/github.com\/MichaelGrupp\/evo."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4666\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:02:55Z","timestamp":1760176975000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4666"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,19]]},"references-count":33,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20174666"],"URL":"https:\/\/doi.org\/10.3390\/s20174666","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,8,19]]}}}