{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T03:42:15Z","timestamp":1777520535427,"version":"3.51.4"},"reference-count":31,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T00:00:00Z","timestamp":1630540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a practical yet effective solution for integrating an RGB-D camera and an inertial sensor to handle the depth dropouts that frequently happen in outdoor environments, due to the short detection range and sunlight interference. In depth drop conditions, only the partial 5-degrees-of-freedom pose information (attitude and position with an unknown scale) is available from the RGB-D sensor. To enable continuous fusion with the inertial solutions, the scale ambiguous position is cast into a directional constraint of the vehicle motion, which is, in essence, an epipolar constraint in multi-view geometry. Unlike other visual navigation approaches, this can effectively reduce the drift in the inertial solutions without delay or under small parallax motion. If a depth image is available, a window-based feature map is maintained to compute the RGB-D odometry, which is then fused with inertial outputs in an extended Kalman filter framework. Flight results from the indoor and outdoor environments, as well as public datasets, demonstrate the improved navigation performance of the proposed approach.<\/jats:p>","DOI":"10.3390\/s21175913","type":"journal-article","created":{"date-parts":[[2021,9,6]],"date-time":"2021-09-06T13:18:26Z","timestamp":1630934306000},"page":"5913","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Depth-Camera-Aided Inertial Navigation Utilizing Directional Constraints"],"prefix":"10.3390","volume":"21","author":[{"given":"Usman","family":"Qayyum","sequence":"first","affiliation":[{"name":"Center of Excellence in Science & Applied Technology (CESAT), Islamabad 45550, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4505-1103","authenticated-orcid":false,"given":"Jonghyuk","family":"Kim","sequence":"additional","affiliation":[{"name":"Robotics Institute, University of Technology Sydney, Sydney, NSW 2006, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,2]]},"reference":[{"key":"ref_1","unstructured":"Zhang, H., and Ye, C. (January, January 24). DUI-VIO: Depth Uncertainty Incorporated Visual Inertial Odometry based on an RGB-D Camera. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, NV, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, H., Wen, I.D.X., Guo, H., and Yu, M. (2018). Research into Kinect\/Inertial Measurement Units Based on Indoor Robots. Sensors, 18.","DOI":"10.3390\/s18030839"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chai, W., and Chen, C. (2015, January 28\u201329). Enhanced Indoor Navigation Using Fusion of IMU and RGB-D Camera. Proceedings of the International Conference on Computer Information Systems and Industrial Applications (CISIA), Bangkok, Thailand.","DOI":"10.2991\/cisia-15.2015.149"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cho, H., Yeon, S., Choi, H., and Doh, N. (2018). Detection and Compensation of Degeneracy Cases for IMU-Kinect Integrated Continuous SLAM with Plane Features. Sensors, 18.","DOI":"10.3390\/s18040935"},{"key":"ref_5","unstructured":"Qayyum, U., and Kim, J. (2013, January 2\u20134). Inertial-Kinect Fusion for Outdoor 3D Navigation. Proceedings of the Australasian Conference on Robotics and Automation, Sydney, Australia."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Dai, X., Mao, Y., Huang, T., Li, B., and Huang, D. (2019, January 5\u20137). Navigation of Simultaneous Localization and Mapping by Fusing RGB-D Camera and IMU on UAV. Proceedings of the CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes, Xiamen, China.","DOI":"10.1109\/SAFEPROCESS45799.2019.9213339"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Diel, D.D., DeBitetto, P., and Teller, S. (2005, January 5\u20137). Epipolar Constraints for Vision-Aided Inertial Navigation. Proceedings of the Seventh IEEE Workshops on Applications of Computer Vision, Breckenridge, CO, USA.","DOI":"10.1109\/ACVMOT.2005.48"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.cja.2017.10.011","article-title":"Rapid and robust initialization for monocular visual inertial navigation within multi-state Kalman filter","volume":"31","author":"Fang","year":"2018","journal-title":"Chin. J. Aeronaut."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.robot.2017.03.019","article-title":"S-PTAM: Stereo Parallel Tracking and Mapping","volume":"93","author":"Pire","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_10","unstructured":"Huang, S., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., and Roy, N. (2011). Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera. Robotics Research, Proceedings of the 15th International Symposium on Robotics Research (ISRR), Flagstaff, AZ, USA, 28 August\u20131 September 2011, Springer."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., and Davison, A. (2011, January 16\u201319). KinectFusion: Real-Time 3D reconstruction and interaction using a moving depth camera. Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, USA.","DOI":"10.1145\/2047196.2047270"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dryanovski, I., Valenti, R., and Xiao, J. (2013, January 6\u201310). Fast Visual Odometry and Mapping from RGB-D Data. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630889"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Scherer, S.A., Dube, D., and Zell, A. (2012, January 14\u201318). Using depth in visual simultaneous localization and mapping. Proceedings of the Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224864"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Whelan, T., McDonald, J., Johannsson, H., Kaess, M., and Leonard, J. (2013, January 6\u201310). Robust Real-Time Visual Odometry for Dense RGB-D Mapping. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631400"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hu, G., Huang, S., Zhao, L., Alempijevic, A., and Dissanayake, G. (2012, January 7\u201312). A robust RGB-D SLAM algorithm. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386103"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Fu, D., Xia, H., and Qiao, Y. (2021). Monocular Visual-Inertial Navigation for Dynamic Environment. Remote Sens., 13.","DOI":"10.3390\/rs13091610"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, Y., Geneva, P., Zuo, X., Eckenhoff, K., Liu, Y., and Huang, G. (2019, January 20\u201324). Tightly-Coupled Aided Inertial Navigation with Point and Plane Features. Proceedings of the International Conference on Robotics and Automation, Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794078"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1177\/0278364910388963","article-title":"Visual-inertial navigation, mapping and localization: A scalable real-time causal approach","volume":"30","author":"Jones","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Mourikis, I., and Roumeliotis, S. (2007, January 10\u201314). A multistate constraint Kalman filter for vision-aided inertial navigation. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Rome, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_21","unstructured":"Konolige, K., Agrawal, M., and Sola, J. (2007, January 26\u201329). Large scale visual odometry for rough terrain. Proceedings of the International Symposium on Research in Robotics (ISRR), Hiroshima, Japan."},{"key":"ref_22","unstructured":"Bouvrie, B. (2011). Improving RGBD Indoor Mapping with IMU Data. [Master\u2019s Thesis, Delft University of Technology]."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ovren, H., Forssen, P., and Tornqvist, D. (2013, January 15\u201317). Why Would I Want a Gyroscope on my RGB-D Sensor?. Proceedings of the IEEE Winter Vision Meetings, Workshop on Robot Vision (WoRV13), Clearwater Beach, FL, USA.","DOI":"10.1109\/WORV.2013.6521916"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Weiss, S., and Siegwart, R. (2011, January 9\u201313). Real-Time Metric State Estimation for Modular Vision-Inertial Systems. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.","DOI":"10.1109\/ICRA.2011.5979982"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s10846-010-9490-z","article-title":"Fusion of IMU and Vision for Absolute Scale Estimation in Monocular SLAM","volume":"61","author":"Nuetzi","year":"2011","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1364\/JOSAA.4.000629","article-title":"Closed-form solution of absolute orientation using unit quaternions","volume":"4","author":"Horn","year":"1987","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2058","DOI":"10.1109\/TPAMI.2012.125","article-title":"Joint depth and color camera calibration with distortion correction","volume":"34","author":"Herrera","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1177\/0278364910382802","article-title":"Visual-inertial sensor fusion:localization mapping and sensor-to-sensor self-calibration","volume":"30","author":"Kelly","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Qayyum, U., and Kim, J. (2012, January 7\u201312). Seamless aiding of inertial-slam using Visual Directional Constraints from a monocular vision. Proceedings of the Intelligent Robot Systems (IROS), Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385830"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Khoshelham, K., and Elberink, S.O. (2012). Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications. Sensors, 12.","DOI":"10.3390\/s120201437"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5913\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:55:12Z","timestamp":1760165712000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5913"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,2]]},"references-count":31,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175913"],"URL":"https:\/\/doi.org\/10.3390\/s21175913","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,2]]}}}