{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:17:11Z","timestamp":1761581831236,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2019,5,16]],"date-time":"2019-05-16T00:00:00Z","timestamp":1557964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Owing to the nonlinearity in visual-inertial state estimation, sufficiently accurate initial states, especially the spatial and temporal parameters between IMU (Inertial Measurement Unit) and camera, should be provided to avoid divergence. Moreover, these parameters are required to be calibrated online since they are likely to vary once the mechanical configuration slightly changes. Recently, direct approaches have gained popularity for their better performance than feature-based approaches in little-texture or low-illumination environments, taking advantage of tracking pixels directly. Based on these considerations, we perform a direct version of monocular VIO (Visual-inertial Odometry), and propose a novel approach to initialize the spatial-temporal parameters and estimate them with all other variables of interest (IMU pose, point inverse depth, etc.). We highlight that our approach is able to perform robust and accurate initialization and online calibration for the spatial and temporal parameters without utilizing any prior information, and also achieves high-precision estimates even when large temporal offset occurs. The performance of the proposed approach was verified through the public UAV (Unmanned Aerial Vehicle) dataset.<\/jats:p>","DOI":"10.3390\/s19102273","type":"journal-article","created":{"date-parts":[[2019,5,16]],"date-time":"2019-05-16T11:21:22Z","timestamp":1558005682000},"page":"2273","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Online Spatial and Temporal Calibration for Monocular Direct Visual-Inertial Odometry"],"prefix":"10.3390","volume":"19","author":[{"given":"Zheyu","family":"Feng","sequence":"first","affiliation":[{"name":"Information Engineering University, Zhengzhou 450001, China"}]},{"given":"Jianwen","family":"Li","sequence":"additional","affiliation":[{"name":"Information Engineering University, Zhengzhou 450001, China"}]},{"given":"Lundong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Information Engineering University, Zhengzhou 450001, China"}]},{"given":"Chen","family":"Chen","sequence":"additional","affiliation":[{"name":"Information Engineering University, Zhengzhou 450001, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation. Proceedings of the IEEE International Conference on Robotics and Automation, Roma, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Weiss, S., Achtelik, M.W., Lynen, S., Chli, M., and Siegwart, R. (2012, January 14\u201318). Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments. Proceedings of the IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225147"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1177\/0278364913481251","article-title":"High-precision, consistent EKF-based visual-inertial odometry","volume":"32","author":"Mingyang","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lynen, S., Achtelik, M.W., Weiss, S., and Chli, M. (2013, January 3\u20137). A robust and modular multi-sensor fusion approach applied to MAV navigation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696917"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1177\/0278364917728574","article-title":"Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback","volume":"36","author":"Bloesch","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_6","unstructured":"Shaojie, S., Michael, N., and Kumar, V. (2015, January 26\u201330). Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs. Proceedings of the IEEE International Conference on Robotics and Automation, Seattle, WA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Forster, C., Carlone, L., Dellaert, F., and Scaramuzza, D. (2015, January 17). IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation. Proceedings of the 2015 Robotics: Science & Systems Conference, Rome, Italy.","DOI":"10.15607\/RSS.2015.XI.006"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual-inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/LRA.2017.2653359","article-title":"Visual-Inertial Monocular SLAM With Map Reuse","volume":"2","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator","volume":"34","author":"Tong","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"von Stumberg, L., Usenko, V., and Cremers, D. (2018, January 21\u201325). Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization. Proceedings of the IEEE International Conference on Robotics and Automation, Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8462905"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1109\/TRO.2016.2623335","article-title":"SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems","volume":"33","author":"Forster","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TPAMI.2017.2658577","article-title":"Direct Sparse Odometry","volume":"40","author":"Engel","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1109\/TRO.2008.2004486","article-title":"A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation","volume":"24","author":"Mirzaei","year":"2008","journal-title":"IEEE Trans. Robot."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1177\/0278364910382802","article-title":"Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration","volume":"30","author":"Kelly","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2070","DOI":"10.1109\/LRA.2019.2893803","article-title":"Degenerate Motion Analysis for Aided INS with Online Spatial and Temporal Sensor Calibration","volume":"4","author":"Yulin","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Furgale, P., Rehder, J., and Siegwart, R. (2013, January 3\u20137). Unified temporal and spatial calibration for multi-sensor systems. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696514"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/TASE.2016.2550621","article-title":"Monocular Visual\u2013Inertial State Estimation with Online Initialization and Camera\u2013IMU Extrinsic Calibration","volume":"14","author":"Zhenfei","year":"2017","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_19","unstructured":"Weibo, H., and Hong, L. (2018, January 21\u201325). Online Initialization and Automatic Camera-IMU Extrinsic Calibration for Monocular Visual-Inertial SLAM. Proceedings of the IEEE International Conference on Robotics and Automation, Brisbane, Australia."},{"key":"ref_20","unstructured":"Zheyu, F., and Jianwen, L. (2018, January 22\u201323). Monocular Visual-Inertial State Estimation With Online Temporal Calibration. Proceedings of the Ubiquitous Positioning, Indoor Navigation and Location-Based Services, Wuhan, China."},{"key":"ref_21","unstructured":"Tong, Q., and Shaojie, S. (2018, January 1\u20135). Online Temporal Calibration for Monocular Visual-Inertial Systems. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1109\/TRO.2011.2170332","article-title":"Visual-Inertial-Aided Navigation for High-Dynamic Motion in Built Environments Without Initial Conditions","volume":"28","author":"Lupton","year":"2012","journal-title":"IEEE Trans. Robot."},{"key":"ref_23","first-page":"1147","article-title":"ORB-SLAM: A Versatile and Accurate Monocular SLAM System","volume":"31","author":"Montiel","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Lovegrove, S.J., and Davison, A.J. (2011, January 6\u201313). DTAM: Dense tracking and mapping in real-time. Proceedings of the International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126513"},{"key":"ref_25","unstructured":"Shi, J., and Tomasi, C. (1994, January 21\u201323). Good features to track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_26","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An Iterative Image Registration Technique with an Application to Stereo Vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence, Columbia, BC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hartley, R. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Schubert, D., Goll, T., Demmel, N., Usenko, V., Stueckler, J., and Cremers, D. (2018, January 1\u20135). The TUM VI Benchmark for Evaluating Visual-Inertial Odometry. Proceedings of the International\/RSJ Conference on Intelligent Robots and Systems, Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593419"},{"key":"ref_29","unstructured":"Tong, Q., and Shaojie, S. (2017, January 24\u201328). Robust initialization of monocular visual-inertial estimation on aerial robots. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1002\/rob.20360","article-title":"Sliding window filter with application to planetary landing","volume":"27","author":"Sibley","year":"2010","journal-title":"J. Field Robot."},{"key":"ref_31","unstructured":"Shaojie, S., Mulgaonkar, Y., Michael, N., and Kumar, V. (2016). Initialization-Free Monocular Visual-Inertial State Estimation with Application to Autonomous MAVs. Experimental Robotics: The 14th International Symposium on Experimental Robotics, Springer."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Huang, G., Mourikis, A.I., and Roumeliotis, S.I. (2008, January 19\u201323). Analysis and improvement of the consistency of extended Kalman filter based SLAM. Proceedings of the IEEE International Conference on Robotics and Automation, Pasadena, CA, USA.","DOI":"10.1109\/ROBOT.2008.4543252"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1177\/0278364915620033","article-title":"The EuRoC micro aerial vehicle datasets","volume":"35","author":"Burri","year":"2016","journal-title":"Int. J. Robot. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/10\/2273\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:52:33Z","timestamp":1760187153000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/10\/2273"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,16]]},"references-count":33,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,5]]}},"alternative-id":["s19102273"],"URL":"https:\/\/doi.org\/10.3390\/s19102273","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,5,16]]}}}