{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T12:54:52Z","timestamp":1778676892527,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2016,9,27]],"date-time":"2016-09-27T00:00:00Z","timestamp":1474934400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41501492"],"award-info":[{"award-number":["41501492"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the Hong Kong Research Grants Council (RGC) Competitive Earmarked Research Grant","award":["152023\/14E"],"award-info":[{"award-number":["152023\/14E"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>RGB-D sensors (sensors with RGB camera and Depth camera) are novel sensing systems that capture RGB images along with pixel-wise depth information. Although they are widely used in various applications, RGB-D sensors have significant drawbacks including limited measurement ranges (e.g., within 3 m) and errors in depth measurement increase with distance from the sensor with respect to 3D dense mapping. In this paper, we present a novel approach to geometrically integrate the depth scene and RGB scene to enlarge the measurement distance of RGB-D sensors and enrich the details of model generated from depth images. First, precise calibration for RGB-D Sensors is introduced. In addition to the calibration of internal and external parameters for both, IR camera and RGB camera, the relative pose between RGB camera and IR camera is also calibrated. Second, to ensure poses accuracy of RGB images, a refined false features matches rejection method is introduced by combining the depth information and initial camera poses between frames of the RGB-D sensor. Then, a global optimization model is used to improve the accuracy of the camera pose, decreasing the inconsistencies between the depth frames in advance. In order to eliminate the geometric inconsistencies between RGB scene and depth scene, the scale ambiguity problem encountered during the pose estimation with RGB image sequences can be resolved by integrating the depth and visual information and a robust rigid-transformation recovery method is developed to register RGB scene to depth scene. The benefit of the proposed joint optimization method is firstly evaluated with the publicly available benchmark datasets collected with Kinect. Then, the proposed method is examined by tests with two sets of datasets collected in both outside and inside environments. The experimental results demonstrate the feasibility and robustness of the proposed method.<\/jats:p>","DOI":"10.3390\/s16101589","type":"journal-article","created":{"date-parts":[[2016,9,27]],"date-time":"2016-09-27T10:14:11Z","timestamp":1474971251000},"page":"1589","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Enhanced RGB-D Mapping Method for Detailed 3D Indoor and Outdoor Modeling"],"prefix":"10.3390","volume":"16","author":[{"given":"Shengjun","family":"Tang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"},{"name":"State-Province Joint Engineering Laboratory of Spatial Information Technology for High Speed Railway Safety, Chengdu 610031, China"},{"name":"Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China"},{"name":"Collaborative Innovation Center for Geospatial Techneology, 129 Luoyu Road, Wuhan 430079, China"},{"name":"Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qing","family":"Zhu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"},{"name":"State-Province Joint Engineering Laboratory of Spatial Information Technology for High Speed Railway Safety, Chengdu 610031, China"},{"name":"Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China"},{"name":"Collaborative Innovation Center for Geospatial Techneology, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Walid","family":"Darwish","sequence":"additional","affiliation":[{"name":"Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han","family":"Hu","sequence":"additional","affiliation":[{"name":"Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Min","family":"Chen","sequence":"additional","affiliation":[{"name":"Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2016,9,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1007\/978-3-642-28572-1_33","article-title":"RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments","volume":"Volume 79","author":"Khatib","year":"2014","journal-title":"Experimental Robotics"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1109\/TPAMI.2004.17","article-title":"An efficient solution to the five-point relative pose problem","volume":"26","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1109\/TRO.2008.2004832","article-title":"Frameslam: From bundle adjustment to real-time visual mapping","volume":"24","author":"Konolige","year":"2008","journal-title":"IEEE Trans. Robot."},{"key":"ref_4","unstructured":"Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. (2010, January 18\u201321). RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. Proceedings of the 12th International Symposium on Experimental Robotics (ISER), Agra, India."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kerl, C., Sturm, J., and Cremers, D. (2013, January 3\u20137). Dense visual slam for RGB-D cameras. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Deajeon, Korea.","DOI":"10.1109\/IROS.2013.6696650"},{"key":"ref_6","first-page":"298","article-title":"Bundle adjustment\u2014A modern synthesis","volume":"Volume 1883","author":"Triggs","year":"2000","journal-title":"Vision Algorithms: Theory and Practice"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dryanovski, I., Valenti, R.G., and Xiao, J. (2013, January 6\u201310). Fast visual odometry and mapping from RGB-D data. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630889"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hu, G., Huang, S., Liang, Z., Alempijevic, A., and Dissanayake, G. (2012, January 7\u201312). A robust RGB-D slam algorithm. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386103"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Whelan, T., Johannsson, H., Kaess, M., Leonard, J.J., and McDonald, J. (2013, January 6\u201310). Robust real-time visual odometry for dense RGB-D mapping. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631400"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1177\/0278364914551008","article-title":"Real-time large-scale dense RGB-D slam with volumetric fusion","volume":"34","author":"Whelan","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.3390\/s120201437","article-title":"Accuracy and resolution of kinect depth data for indoor mapping applications","volume":"12","author":"Khoshelham","year":"2012","journal-title":"Sensors"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4981","DOI":"10.3390\/s140304981","article-title":"Enhanced monocular visual odometry integrated with laser distance meter for astronaut navigation","volume":"14","author":"Wu","year":"2014","journal-title":"Sensors"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Steinbrucker, F., Kerl, C., Cremers, D., and Sturm, J. (2013, January 1\u20138). Large-scale multi-resolution surface reconstruction from RGB-D sequences. Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.405"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1023\/A:1008854305733","article-title":"Globally consistent range scan alignment for environment mapping","volume":"4","author":"Lu","year":"1997","journal-title":"Autonom. Robots"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., and Fitzgibbon, A. (2011, January 26\u201329). Kinectfusion: Real-time dense surface mapping and tracking. Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Basel, Switzerland.","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Du, H., Henry, P., Ren, X., Cheng, M., Goldman, D.B., Seitz, S.M., and Fox, D. (2011, January 17\u201321). Interactive 3D modeling of indoor environments with a consumer depth camera. Proceedings of the 13th International Conference on Ubiquitous Computing, Beijing, China.","DOI":"10.1145\/2030112.2030123"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1177\/0278364911434148","article-title":"RGB-D mapping: Using kinect-style depth cameras for dense 3D modeling of indoor environments","volume":"31","author":"Henry","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_18","unstructured":"Engelhard, N., Endres, F., Hess, J., Sturm, J., and Burgard, W. (2011, January 8). Real-time 3D visual slam with a hand-held RGB-D camera. Proceedings of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, Vasteras, Sweden."},{"key":"ref_19","first-page":"W2","article-title":"Generation and weighting of 3D point correspondences for improved registration of RGB-D data","volume":"5","author":"Khoshelham","year":"2013","journal-title":"Proc. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1109\/LGRS.2015.2508880","article-title":"Mapping indoor spaces by adaptive coarse-to-fine registration of RGB-D data","volume":"13","author":"Basso","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1109\/TRO.2013.2279412","article-title":"3-D mapping with an RGB-D camera","volume":"30","author":"Endres","year":"2014","journal-title":"IEEE Trans. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Stuckler, J., and Behnke, S. (2012, January 13\u201315). Integrating depth and color cues for dense multi-resolution scene mapping using RGB-D cameras. Proceedings of the 2012 IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Hamburg, Germany.","DOI":"10.1109\/MFI.2012.6343050"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"14639","DOI":"10.3390\/s150614639","article-title":"Loop closing detection in RGB-D slam combining appearance and geometric constraints","volume":"15","author":"Zhang","year":"2015","journal-title":"Sensors"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"247","DOI":"10.3390\/robotics3030247","article-title":"Imu and multiple RGB-D camera fusion for assisting indoor stop-and-go 3D terrestrial laser scanning","volume":"3","author":"Chow","year":"2014","journal-title":"Robotics"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.culher.2012.12.003","article-title":"Multi-image 3D reconstruction data evaluation","volume":"15","author":"Koutsoudis","year":"2014","journal-title":"J. Cult. Herit."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1023\/B:VISI.0000025798.50602.3a","article-title":"Visual modeling with a hand-held camera","volume":"59","author":"Pollefeys","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1145\/1141911.1141964","article-title":"Photo tourism: Exploring photo collections in 3D","volume":"25","author":"Snavely","year":"2006","journal-title":"ACM Trans. Graph."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ikeuchi, K. (2014). Computer Vision: A Reference Guide, Springer.","DOI":"10.1007\/978-0-387-31439-6"},{"key":"ref_29","first-page":"855","article-title":"Close-range camera calibration","volume":"37","author":"Duane","year":"1971","journal-title":"Photogramm. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1007\/3-540-45053-X_47","article-title":"3-D motion and structure from 2-D motion causally integrated over time: Implementation","volume":"Volume 1843","author":"Vernon","year":"2000","journal-title":"Computer Vision\u2014ECCV 2000"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s11263-007-0107-3","article-title":"Modeling the world from internet photo collections","volume":"80","author":"Snavely","year":"2008","journal-title":"Int. J. Comput. Vis."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Ziegler, G., Tevs, A., Theobalt, C., and Seidel, H.-P. GPU Point List Generation through Histogram Pyramids. Available online: http:\/\/pubman.mpdl.mpg.de\/pubman\/item\/escidoc:1819167:4\/component\/escidoc:1840866\/MPI-I-2006-4-002.pdf."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1145\/293347.293348","article-title":"An optimal algorithm for approximate nearest neighbor searching fixed dimensions","volume":"45","author":"Arya","year":"1998","journal-title":"J. ACM"},{"key":"ref_36","unstructured":"Ahmed, M.T., Dailey, M.N., Landabaso, J.L., and Herrero, N. (2010, January 17\u201321). Robust key frame extraction for 3D reconstruction from video streams. Proceedings of the VISAPP 2010, Angers, France."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"053201","DOI":"10.1117\/1.2919801","article-title":"Optimal keyframe selection algorithm for three-dimensional reconstruction in uncalibrated multiple images","volume":"47","author":"Seo","year":"2008","journal-title":"Opt. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_39","unstructured":"Nocedal, J., and Wright, S. (2006). Numerical Optimization, Springer."},{"key":"ref_40","unstructured":"Besl, P.J., and McKay, N.D. (1992). Robotics-DL Tentative, International Society for Optics and Photonics."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jancosek, M., and Pajdla, T. (2011, January 20\u201325). Multi-view reconstruction preserving weakly-supported surfaces. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995693"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D slam systems. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1177\/0278364912460895","article-title":"Fast and accurate scan registration through minimization of the distance between compact 3D NDT representations","volume":"31","author":"Stoyanov","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Steinbrucker, F., Sturm, J., and Cremers, D. (2011, January 6\u201313). Real-time visual odometry from dense RGB-D images. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130321"},{"key":"ref_45","unstructured":"Huang, A.S., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., and Roy, N. (September, January 28). Visual odometry and mapping for autonomous flight using an RGB-D camera. Proceedings of the International Symposium on Robotics Research (ISRR), Flagstaff, AZ, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1589\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:31:54Z","timestamp":1760211114000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1589"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,9,27]]},"references-count":45,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2016,10]]}},"alternative-id":["s16101589"],"URL":"https:\/\/doi.org\/10.3390\/s16101589","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,9,27]]}}}