{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:58:42Z","timestamp":1760241522084,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2018,5,1]],"date-time":"2018-05-01T00:00:00Z","timestamp":1525132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Grant from the Research Program of Shenzhen S&amp;T Innovation Committee","award":["JCYJ20170412105839839"],"award-info":[{"award-number":["JCYJ20170412105839839"]}]},{"name":"the open research Fund of State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing","award":["17E04"],"award-info":[{"award-number":["17E04"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Traditionally, visual-based RGB-D SLAM systems only use correspondences with valid depth values for camera tracking, thus ignoring the regions without 3D information. Due to the strict limitation on measurement distance and view angle, such systems adopt only short-range constraints which may introduce larger drift errors during long-distance unidirectional tracking. In this paper, we propose a novel geometric integration method that makes use of both 2D and 3D correspondences for RGB-D tracking. Our method handles the problem by exploring visual features both when depth information is available and when it is unknown. The system comprises two parts: coarse pose tracking with 3D correspondences, and geometric integration with hybrid correspondences. First, the coarse pose tracking generates the initial camera pose using 3D correspondences with frame-by-frame registration. The initial camera poses are then used as inputs for the geometric integration model, along with 3D correspondences, 2D-3D correspondences and 2D correspondences identified from frame pairs. The initial 3D location of the correspondence is determined in two ways, from depth image and by using the initial poses to triangulate. The model improves the camera poses and decreases drift error during long-distance RGB-D tracking iteratively. Experiments were conducted using data sequences collected by commercial Structure Sensors. The results verify that the geometric integration of hybrid correspondences effectively decreases the drift error and improves mapping accuracy. Furthermore, the model enables a comparative and synergistic use of datasets, including both 2D and 3D features.<\/jats:p>","DOI":"10.3390\/s18051385","type":"journal-article","created":{"date-parts":[[2018,5,3]],"date-time":"2018-05-03T03:20:27Z","timestamp":1525317627000},"page":"1385","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Geometric Integration of Hybrid Correspondences for RGB-D Unidirectional Tracking"],"prefix":"10.3390","volume":"18","author":[{"given":"Shengjun","family":"Tang","sequence":"first","affiliation":[{"name":"Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Land Surveying &amp; Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weixi","family":"Wang","sequence":"additional","affiliation":[{"name":"Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaoming","family":"Li","sequence":"additional","affiliation":[{"name":"Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0549-6052","authenticated-orcid":false,"given":"Walid","family":"Darwish","sequence":"additional","affiliation":[{"name":"Department of Land Surveying &amp; Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenbin","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Land Surveying &amp; Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhengdong","family":"Huang","sequence":"additional","affiliation":[{"name":"Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Land Surveying &amp; Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"},{"name":"Faculty of Geosciences and Environmental Engineering of Southwest Jiaotong University, Chengdu 611756, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Renzhong","family":"Guo","sequence":"additional","affiliation":[{"name":"Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,5,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1145\/504729.504754","article-title":"Probabilistic robotics","volume":"45","author":"Thrun","year":"2002","journal-title":"Commun. ACM"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., and Burgard, W. (2012, January 14\u201318). An evaluation of the RGB-D SLAM system. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225199"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., and Fitzgibbon, A. (2011, January 26\u201329). KinectFusion: Real-time dense surface mapping and tracking. Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Basel, Switzerland.","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.3390\/s120201437","article-title":"Accuracy and resolution of kinect depth data for indoor mapping applications","volume":"12","author":"Khoshelham","year":"2012","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Darwish, W., Tang, S., Li, W., and Chen, W. (2017). A New Calibration Method for Commercial RGB-D Sensors. Sensors, 17.","DOI":"10.3390\/s17061204"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ataer-Cansizoglu, E., Taguchi, Y., and Ramalingam, S. (2016, January 16\u201321). Pinpoint SLAM: A Hybrid of 2D and 3D Simultaneous Localization and Mapping for RGB-D Sensors. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487262"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1109\/LGRS.2015.2508880","article-title":"Mapping Indoor Spaces by Adaptive Coarse-to-Fine Registration of RGB-D Data","volume":"13","author":"Santos","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","unstructured":"Whelan, T., Kaess, M., Fallon, M., Johannsson, H., Leonard, J., and McDonald, J. (2018, April 27). Kintinuous: Spatially Extended Kinectfusion. Available online: http:\/\/www.robots.ox.ac.uk\/~mfallon\/publications\/12_whelan_tr.pdf."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1007\/978-3-642-34263-9_30","article-title":"A Memory-Efficient KinectFusion Using Octree","volume":"Volume 7633","author":"Hu","year":"2012","journal-title":"Computational Visual Media"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2508363.2508375","article-title":"Scalable real-time volumetric surface reconstruction","volume":"32","author":"Chen","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_11","first-page":"169","article-title":"Real-time 3D reconstruction at scale using voxel hashing","volume":"32","author":"Izadi","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Henry, P., Fox, D., Bhowmik, A., and Mongia, R. (July, January 29). Patch Volumes: Segmentation-Based Consistent Mapping with RGB-D Cameras. Proceedings of the 2013 International Conference on 3D Vision\u20143DV, Seattle, WA, USA.","DOI":"10.1109\/3DV.2013.59"},{"key":"ref_13","first-page":"9","article-title":"Real-Time Camera Tracking and 3D Reconstruction Using Signed Distance Functions","volume":"2","author":"Kahl","year":"2013","journal-title":"Robot. Sci. Syst."},{"key":"ref_14","unstructured":"Keller, M., Lefloch, D., Lambers, M., Izadi, S., Weyrich, T., and Kolb, A. (July, January 29). Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion. Proceedings of the International Conference on 3D Vision\u20143DV, Seattle, WA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Meilland, M., and Comport, A.I. (2013, January 3\u20137). On Unifying Key-Frame and Voxel-Based Dense Visual SLAM at Large Scales. Proceedings of the IEEE\/RSJ International Conference onIntelligent Robots and Systems (IROS), Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696881"},{"key":"ref_16","first-page":"1","article-title":"Dense scene reconstruction with points of interest","volume":"32","author":"Zhou","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.jvcir.2013.02.008","article-title":"Multi-resolution surfel maps for efficient dense 3D modeling and tracking","volume":"25","author":"Behnke","year":"2014","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1177\/0278364914551008","article-title":"Real-time large-scale dense RGB-D SLAM with volumetric fusion","volume":"34","author":"Whelan","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.cviu.2016.11.008","article-title":"Modeling large-scale indoor scenes with rigid fragments using RGB-D cameras","volume":"157","author":"Thomas","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_20","unstructured":"Engelhard, N., Endres, F., Hess, J., Sturm, J., and Burgard, W. (2011, January 8). Real-time 3D visual SLAM with a hand-held RGB-D camera. Proceedings of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, Vasteras, Sweden."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Christensen, I.H., and Khatib, O. (2017). Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera. Robotics Research: The 15th International Symposium ISRR, Flagstaff, Arizona, 9\u201312December 2011, Springer.","DOI":"10.1007\/978-3-319-29363-9"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Steinbrucker, F., Sturm, J., and Cremers, D. (2011, January 6\u201313). Real-time visual odometry from dense RGB-D images. Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130321"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1177\/0278364911434148","article-title":"RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments","volume":"31","author":"Henry","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_24","unstructured":"K\u00fcmmerle, R., Grisetti, G., Strasdat, H., Konolige, K., and Burgard, W. (2011, January 9\u201313). G2o: A general framework for graph optimization. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kerl, C., Sturm, J., and Cremers, D. (2013, January 6\u201310). Robust odometry estimation for RGB-D cameras. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631104"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Kerl, C., Sturm, J., and Cremers, D. (2013, January 3\u20137). Dense visual SLAM for RGB-D cameras. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696650"},{"key":"ref_27","first-page":"W2","article-title":"Generation and weighting of 3D point correspondences for improved registration of RGB-D data","volume":"5","author":"Khoshelham","year":"2013","journal-title":"Proc. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Steinbrucker, F., Kerl, C., Cremers, D., and Sturm, J. (2013, January 1\u20138). Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Darling Harbour, Sydney.","DOI":"10.1109\/ICCV.2013.405"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"247","DOI":"10.3390\/robotics3030247","article-title":"IMU and Multiple RGB-D Camera Fusion for Assisting Indoor Stop-and-Go 3D Terrestrial Laser Scanning","volume":"3","author":"Chow","year":"2014","journal-title":"Robotics"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","author":"Tardos","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hu, G., Shoudong, H., Liang, Z., Alempijevic, A., and Dissanayake, G. (2012, January 7\u201312). A robust RGB-D SLAM algorithm. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6386103"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, J., Kaess, M., and Singh, S. (2014, January 14\u201318). Real-time depth enhanced monocular odometry. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943269"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tang, S., Zhu, Q., Chen, W., Darwish, W., Wu, B., Hu, H., and Chen, M. (2016). Enhanced RGB-D Mapping Method for Detailed 3D Indoor and Outdoor Modeling. Sensors, 16.","DOI":"10.3390\/s16101589"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Galvez-Lopez, D., and Tardos, J.D. (2011, January 25\u201330). Real-time loop detection with bags of binary words. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6094885"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_36","unstructured":"Segal, A., Haehnel, D., and Thrun, S. (July, January 28). Generalized-ICP. Proceedings of the Robotics: Science and Systems, Seattle, WA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1177\/0278364912460895","article-title":"Fast and Accurate Scan Registration through Minimization of the Distance between Compact 3D NDT Representations","volume":"31","author":"Stoyanov","year":"2012","journal-title":"Int. J. Robot. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1385\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:02:46Z","timestamp":1760194966000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1385"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,5,1]]},"references-count":37,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,5]]}},"alternative-id":["s18051385"],"URL":"https:\/\/doi.org\/10.3390\/s18051385","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,5,1]]}}}