{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T18:14:08Z","timestamp":1761156848852,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,2,26]],"date-time":"2020-02-26T00:00:00Z","timestamp":1582675200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual inertial odometry (VIO) is the front-end of visual simultaneous localization and mapping (vSLAM) methods and has been actively studied in recent years. In this context, a time-of-flight (ToF) camera, with its high accuracy of depth measurement and strong resilience to ambient light of variable intensity, draws our interest. Thus, in this paper, we present a realtime visual inertial system based on a low cost ToF camera. The iterative closest point (ICP) methodology is adopted, incorporating salient point-selection criteria and a robustness-weighting function. In addition, an error-state Kalman filter is used and fused with inertial measurement unit (IMU) data. To test its capability, the ToF\u2013VIO system is mounted on an unmanned aerial vehicle (UAV) platform and operated in a variable light environment. The estimated flight trajectory is compared with the ground truth data captured by a motion capture system. Real flight experiments are also conducted in a dark indoor environment, demonstrating good agreement with estimated performance. The current system is thus shown to be accurate and efficient for use in UAV applications in dark and Global Navigation Satellite System (GNSS)-denied environments.<\/jats:p>","DOI":"10.3390\/s20051263","type":"journal-article","created":{"date-parts":[[2020,2,27]],"date-time":"2020-02-27T03:21:16Z","timestamp":1582773676000},"page":"1263","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Perception in the Dark; Development of a ToF Visual Inertial Odometry System"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1506-0615","authenticated-orcid":false,"given":"Shengyang","family":"Chen","sequence":"first","affiliation":[{"name":"Deptartment of Mechanical Engineering and Interdisciplinary Division of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7307-6039","authenticated-orcid":false,"given":"Ching-Wei","family":"Chang","sequence":"additional","affiliation":[{"name":"Deptartment of Mechanical Engineering and Interdisciplinary Division of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1181-8786","authenticated-orcid":false,"given":"Chih-Yung","family":"Wen","sequence":"additional","affiliation":[{"name":"Deptartment of Mechanical Engineering and Interdisciplinary Division of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,26]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel Tracking and Mapping for Small AR Workspaces. Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan.","key":"ref_1","DOI":"10.1109\/ISMAR.2007.4538852"},{"doi-asserted-by":"crossref","unstructured":"Forster, C., Pizzoli, M., and Scaramuzza, D. (June, January 31). SVO: Fast semi-direct monocular visual odometry. Proceedings of the 2014 IEEE international conference on robotics and automation (ICRA), Hong Kong, China.","key":"ref_2","DOI":"10.1109\/ICRA.2014.6906584"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1109\/TRO.2016.2623335","article-title":"SVO: Semidirect visual odometry for monocular and multicamera systems","volume":"33","author":"Forster","year":"2016","journal-title":"IEEE Trans. Robot."},{"doi-asserted-by":"crossref","unstructured":"Lynen, S., Achtelik, M.W., Weiss, S., Chli, M., and Siegwart, R. (2013, January 3\u20137). A robust and modular multi-sensor fusion approach applied to mav navigation. Proceedings of the 2013 IEEE\/RSJ international conference on intelligent robots and systems, Tokyo, Japan.","key":"ref_4","DOI":"10.1109\/IROS.2013.6696917"},{"doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A multi-state constraint Kalman filter for vision-aided inertial navigation. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.","key":"ref_5","DOI":"10.1109\/ROBOT.2007.364024"},{"doi-asserted-by":"crossref","unstructured":"Leutenegger, S., Furgale, P., Rabaud, V., Chli, M., Konolige, K., and Siegwart, R. (2013, January 24\u201328). Keyframe-based visual\u2013inertial slam using nonlinear optimization. Proceedings of the Robotis Science and Systems (RSS), Berlin, Germany.","key":"ref_6","DOI":"10.15607\/RSS.2013.IX.037"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"Vins-mono: A robust and versatile monocular visual\u2013inertial state estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"doi-asserted-by":"crossref","unstructured":"Shan, Z., Li, R., and Schwertfeger, S. (2019). RGBD-Inertial Trajectory Estimation and Mapping for Ground Robots. Sensors, 19.","key":"ref_8","DOI":"10.3390\/s19102251"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: a versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"doi-asserted-by":"crossref","unstructured":"Von Stumberg, L., Usenko, V., and Cremers, D. (2018, January 21\u201325). Direct sparse visual\u2013inertial odometry using dynamic marginalization. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","key":"ref_10","DOI":"10.1109\/ICRA.2018.8462905"},{"key":"ref_11","first-page":"127","article-title":"Kinectfusion: Real-time dense surface mapping and tracking","volume":"11","author":"Newcombe","year":"2011","journal-title":"ISMAR"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/LRA.2017.2653359","article-title":"Visual\u2013inertial monocular SLAM with map reuse","volume":"2","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1002\/rob.20204","article-title":"Scan registration for autonomous mining vehicles using 3D-NDT","volume":"24","author":"Magnusson","year":"2007","journal-title":"J. Field Robot."},{"doi-asserted-by":"crossref","unstructured":"Zhao, S., and Fang, Z. (2018). Direct depth SLAM: Sparse geometric feature enhanced direct depth SLAM system for low-texture environments. Sensors, 18.","key":"ref_14","DOI":"10.3390\/s18103339"},{"doi-asserted-by":"crossref","unstructured":"Kerl, C., Sturm, J., and Cremers, D. (2013, January 6\u201310). Robust odometry estimation for RGB-D cameras. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","key":"ref_15","DOI":"10.1109\/ICRA.2013.6631104"},{"doi-asserted-by":"crossref","unstructured":"Meilland, M., Comport, A.I., and Rives, P. (2010, January 18\u201322). A spherical robot-centered representation for urban navigation. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","key":"ref_16","DOI":"10.1109\/IROS.2010.5650380"},{"doi-asserted-by":"crossref","unstructured":"Delmerico, J., and Scaramuzza, D. (2018, January 21\u201325). A benchmark comparison of monocular visual\u2013inertial odometry algorithms for flying robots. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","key":"ref_17","DOI":"10.1109\/ICRA.2018.8460664"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/0262-8856(92)90066-C","article-title":"Object modelling by registration of multiple range images","volume":"10","author":"Chen","year":"1992","journal-title":"Image Vis. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1177\/0278364912460895","article-title":"Fast and accurate scan registration through minimization of the distance between compact 3D NDT representations","volume":"31","author":"Stoyanov","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1177\/0278364915620033","article-title":"The EuRoC micro aerial vehicle datasets","volume":"35","author":"Burri","year":"2016","journal-title":"Int. J. Robot. Res."},{"unstructured":"Li, L. (2014). Time-of-Flight Camera\u2013An Introduction, Texas Instruments. Technical White Paper.","key":"ref_21"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cviu.2015.05.006","article-title":"Kinect range sensing: Structured-light versus Time-of-Flight Kinect","volume":"139","author":"Sarbolandi","year":"2015","journal-title":"Comput. Vis. Image Underst."},{"unstructured":"Rusinkiewicz, S., and Levoy, M. (June, January 28). Efficient variants of the ICP algorithm. Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, QC, Canada.","key":"ref_23"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"992","DOI":"10.1109\/LRA.2016.2530164","article-title":"Fast visual odometry using intensity-assisted iterative closest point","volume":"1","author":"Li","year":"2016","journal-title":"IEEE Robot. Autom. Lett."},{"doi-asserted-by":"crossref","unstructured":"Tomono, M. (2009, January 12\u201317). Robust 3D SLAM with a stereo camera based on an edge-point ICP algorithm. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan.","key":"ref_25","DOI":"10.1109\/ROBOT.2009.5152529"},{"doi-asserted-by":"crossref","unstructured":"Santamaria-Navarro, A., Sol\u00e0, J., and Andrade-Cetto, J. (2019). Visual Guidance of Unmanned Aerial Manipulators, Springer.","key":"ref_26","DOI":"10.1007\/978-3-319-96580-2"},{"unstructured":"Grupp, M. (2020, February 25). evo: Python package for the evaluation of odometry and SLAM. Available online: https:\/\/github.com\/MichaelGrupp\/evo.","key":"ref_27"},{"doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A Benchmark for the Evaluation of RGB-D SLAM Systems. Proceedings of the International Conference on Intelligent Robot Systems (IROS), Vilamoura, Portugal.","key":"ref_28","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1109\/MRA.2015.2432331","article-title":"Registration with the point cloud library: A modular framework for aligning in 3-D","volume":"22","author":"Holz","year":"2015","journal-title":"IEEE Robot. Autom. Mag."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1263\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:01:48Z","timestamp":1760173308000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1263"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,26]]},"references-count":29,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20051263"],"URL":"https:\/\/doi.org\/10.3390\/s20051263","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,2,26]]}}}