{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T21:30:25Z","timestamp":1781299825115,"version":"3.54.1"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,29]],"date-time":"2023-01-29T00:00:00Z","timestamp":1674950400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["52172381"],"award-info":[{"award-number":["52172381"]}]},{"name":"National Natural Science Foundation of China","award":["cstc2021jcyj-msxmX1121"],"award-info":[{"award-number":["cstc2021jcyj-msxmX1121"]}]},{"DOI":"10.13039\/501100005230","name":"Natural Science Foundation of Chongqing, China","doi-asserted-by":"publisher","award":["52172381"],"award-info":[{"award-number":["52172381"]}],"id":[{"id":"10.13039\/501100005230","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005230","name":"Natural Science Foundation of Chongqing, China","doi-asserted-by":"publisher","award":["cstc2021jcyj-msxmX1121"],"award-info":[{"award-number":["cstc2021jcyj-msxmX1121"]}],"id":[{"id":"10.13039\/501100005230","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Monocular camera and Lidar are the two most commonly used sensors in unmanned vehicles. Combining the advantages of the two is the current research focus of SLAM and semantic analysis. In this paper, we propose an improved SLAM and semantic reconstruction method based on the fusion of Lidar and monocular vision. We fuse the semantic image with the low-resolution 3D Lidar point clouds and generate dense semantic depth maps. Through visual odometry, ORB feature points with depth information are selected to improve positioning accuracy. Our method uses parallel threads to aggregate 3D semantic point clouds while positioning the unmanned vehicle. Experiments are conducted on the public CityScapes and KITTI Visual Odometry datasets, and the results show that compared with the ORB-SLAM2 and DynaSLAM, our positioning error is approximately reduced by 87%; compared with the DEMO and DVL-SLAM, our positioning accuracy improves in most sequences. Our 3D reconstruction quality is better than DynSLAM and contains semantic information. The proposed method has engineering application value in the unmanned vehicles field.<\/jats:p>","DOI":"10.3390\/s23031502","type":"journal-article","created":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T02:28:34Z","timestamp":1675045714000},"page":"1502","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["SLAM and 3D Semantic Reconstruction Based on the Fusion of Lidar and Monocular Vision"],"prefix":"10.3390","volume":"23","author":[{"given":"Lu","family":"Lou","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing 400074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yitian","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Chongqing Jiaotong University, Chongqing 400074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Zhang","sequence":"additional","affiliation":[{"name":"Guangdong Haoxing Technology Co., Ltd, Foshan 528300, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanbing","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Mechatronics and Vehicle Engineering, Chongqing Jiaotong University, Chongqing 400074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jia, G., Li, X., Zhang, D., Xu, W., Lv, H., Shi, Y., and Cai, M. (2022). Visual-SLAM Classical framework and key Techniques: A review. Sensors, 22.","DOI":"10.3390\/s22124582"},{"key":"ref_2","unstructured":"Huang, B., Zhao, J., and Liu, J. (2019). A survey of simultaneous localization and mapping. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Debeunne, C., and Vivet, D. (2020). A review of visual-LiDAR fusion based simultaneous localization and mapping. Sensors, 20.","DOI":"10.3390\/s20072068"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jiao, J. (2018, January 23\u201327). Machine learning assisted high-definition map creation. Proceedings of the 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Tokyo, Japan.","DOI":"10.1109\/COMPSAC.2018.00058"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4076","DOI":"10.1109\/LRA.2018.2860039","article-title":"DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes","volume":"3","author":"Bescos","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"95301","DOI":"10.1109\/ACCESS.2020.2994348","article-title":"SDF-SLAM: Semantic depth filter SLAM for dynamic environments","volume":"8","author":"Cui","year":"2020","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Shan, T., and Englot, B. (2018, January 1\u20135). Lego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594299"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"\u0106wian, K., Nowicki, M.R., Wietrzykowski, J., and Skrzypczy\u0144ski, P. (2021). Large-scale LiDAR SLAM with factor graph optimization on high-level geometric features. Sensors, 21.","DOI":"10.3390\/s21103445"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"Vins-mono: A robust and versatile monocular visual-inertial state estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1109\/TRO.2021.3075644","article-title":"Orb-slam3: An accurate open-source library for visual, visual\u2013inertial, and multimap slam","volume":"37","author":"Campos","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ku, J., Harakeh, A., and Waslander, S.L. (2018, January 8\u201310). In defense of classical image processing: Fast depth completion on the cpu. Proceedings of the 2018 15th Conference on Computer and Robot Vision (CRV), Toronto, ON, Canada.","DOI":"10.1109\/CRV.2018.00013"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Graeter, J., Wilczynski, A., and Lauer, M. (2018, January 1\u20135). Limo: Lidar-monocular visual odometry. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594394"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"De Silva, V., Roche, J., and Kondoz, A. (2018). Robust fusion of LiDAR and wide-angle camera data for autonomous mobile robots. Sensors, 18.","DOI":"10.3390\/s18082730"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/s10514-015-9525-1","article-title":"A real-time method for depth enhanced visual odometry","volume":"41","author":"Zhang","year":"2017","journal-title":"Auton. Robot."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s10514-019-09881-0","article-title":"DVL-SLAM: Sparse depth enhanced direct visual-LiDAR SLAM","volume":"44","author":"Shin","year":"2020","journal-title":"Auton. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"McCormac, J., Clark, R., Bloesch, M., Davison, A., and Leutenegger, S. (2018, January 12\u201316). Fusion++: Volumetric object-level slam. Proceedings of the 2018 International Conference on 3D Vision (3DV), Prague, Czech Republic.","DOI":"10.1109\/3DV.2018.00015"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Runz, M., Buffier, M., and Agapito, L. (2018, January 16\u201320). Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2018.00024"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"B\u00e2rsan, I.A., Liu, P., Pollefeys, M., and Geiger, A. (2018, January 21\u201325). Robust dense mapping for large-scale dynamic environments. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8462974"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Milioto, A., Vizzo, I., Behley, J., and Stachniss, C. (2019, January 3\u20138). Rangenet++: Fast and accurate lidar semantic segmentation. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), The Venetian Macao, Macau.","DOI":"10.1109\/IROS40897.2019.8967762"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, X., Milioto, A., Palazzolo, E., Giguere, P., Behley, J., and Stachniss, C. (2019, January 3\u20138). Suma++: Efficient lidar-based semantic slam. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), The Venetian Macao, Macau.","DOI":"10.1109\/IROS40897.2019.8967704"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3051","DOI":"10.1007\/s11263-021-01515-2","article-title":"Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation","volume":"129","author":"Yu","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s11263-008-0152-6","article-title":"Epnp: An accurate o (n) solution to the pnp problem","volume":"81","author":"Lepetit","year":"2009","journal-title":"Int. J. Comput. Vis."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Triggs, B., McLauchlan, P.F., Hartley, R.I., and Fitzgibbon, A.W. (1999, January 21\u201322). Bundle adjustment\u2014A modern synthesis. Proceedings of the International Workshop on Vision Algorithms, Corfu, Greece.","DOI":"10.1007\/3-540-44480-7_21"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Huber, P.J. (2011). International Encyclopedia of Statistical Science, Springer.","DOI":"10.1007\/978-3-642-04898-2_594"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_30","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014). Computer Vision\u2014ECCV 2014, Springer International Publishing."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1109\/TPAMI.2007.1049","article-title":"MonoSLAM: Real-time single camera SLAM","volume":"29","author":"Davison","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? the kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_33","unstructured":"Cignoni, P., Callieri, M., Corsini, M., Dellepiane, M., Ganovelli, F., and Ranzuglia, G. (2008, January 2\u20134). Meshlab: An open-source mesh processing tool. Proceedings of the Eurographics Italian Chapter Conference, Salerno, Italy."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.rse.2017.06.031","article-title":"Google Earth Engine: Planetary-scale geospatial analysis for everyone","volume":"202","author":"Gorelick","year":"2017","journal-title":"Remote. Sens. Environ."},{"key":"ref_35","unstructured":"Maturana, D., Chou, P.W., Uenoyama, M., and Scherer, S. (2018). Field and Service Robotics, Springer International Publishing."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Paz, D., Zhang, H., Li, Q., Xiang, H., and Christensen, H.I. (2020, January 25\u201329). Probabilistic semantic mapping for urban autonomous driving applications. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341738"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1502\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:19:05Z","timestamp":1760120345000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1502"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,29]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031502"],"URL":"https:\/\/doi.org\/10.3390\/s23031502","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,29]]}}}