{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:21:07Z","timestamp":1760239267972,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,11,3]],"date-time":"2020-11-03T00:00:00Z","timestamp":1604361600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Jiangsu 303 Province, China","award":["No.BK2011563"],"award-info":[{"award-number":["No.BK2011563"]}]},{"name":"Scientifific and Technical Supporting Programs of Jiangsu 304 Province, China","award":["No.BE2011169"],"award-info":[{"award-number":["No.BE2011169"]}]},{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61201425","61271231","61300157","61100111"],"award-info":[{"award-number":["61201425","61271231","61300157","61100111"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We present a real-time Truncated Signed Distance Field (TSDF)-based three-dimensional (3D) semantic reconstruction for LiDAR point cloud, which achieves incremental surface reconstruction and highly accurate semantic segmentation. The high-precise 3D semantic reconstruction in real time on LiDAR data is important but challenging. Lighting Detection and Ranging (LiDAR) data with high accuracy is massive for 3D reconstruction. We so propose a line-of-sight algorithm to update implicit surface incrementally. Meanwhile, in order to use more semantic information effectively, an online attention-based spatial and temporal feature fusion method is proposed, which is well integrated into the reconstruction system. We implement parallel computation in the reconstruction and semantic fusion process, which achieves real-time performance. We demonstrate our approach on the CARLA dataset, Apollo dataset, and our dataset. When compared with the state-of-art mapping methods, our method has a great advantage in terms of both quality and speed, which meets the needs of robotic mapping and navigation.<\/jats:p>","DOI":"10.3390\/s20216264","type":"journal-article","created":{"date-parts":[[2020,11,3]],"date-time":"2020-11-03T09:09:32Z","timestamp":1604394572000},"page":"6264","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Reconstruction of High-Precision Semantic Map"],"prefix":"10.3390","volume":"20","author":[{"given":"Xinyuan","family":"Tu","sequence":"first","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Runhao","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingji","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sidan","family":"Du","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineer, Nanjing University, Nanjing 21000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,3]]},"reference":[{"key":"ref_1","unstructured":"Verma, V., Kumar, R., and Hsu, S. (2006, January 17\u201322). 3D building detection and modeling from aerial lidar data. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhou, Q.Y., and Neumann, U. (2010, January 5\u201311). 2.5 D dual contouring: A robust approach to creating building models from aerial lidar point clouds. Proceedings of the European conference on computer vision, Crete, Greece.","DOI":"10.1007\/978-3-642-15558-1_9"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Poullis, C., and You, S. (2009, January 20\u201325). Automatic reconstruction of cities from remote sensor data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206562"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s10514-012-9321-0","article-title":"OctoMap: An efficient probabilistic 3D mapping framework based on octrees","volume":"34","author":"Hornung","year":"2013","journal-title":"Auton. Robot."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2487228.2487237","article-title":"Screened poisson surface reconstruction","volume":"32","author":"Kazhdan","year":"2013","journal-title":"Acm Trans. Graph."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1111\/cgf.12189","article-title":"Noise-adaptive shape reconstruction from raw point sets","volume":"32","author":"Giraudot","year":"2013","journal-title":"Graph. Forum"},{"key":"ref_7","first-page":"1","article-title":"Semantic decomposition and reconstruction of residential scenes from LiDAR data","volume":"32","author":"Lin","year":"2013","journal-title":"Acm Trans. Graph."},{"key":"ref_8","unstructured":"Lovi, D., Birkbeck, N., Cobzas, D., and Jagersand, M. (2010, January 17\u201320). Incremental free-space carving for real-time 3d reconstruction. Proceedings of the Fifth International Symposium on 3D Data Processing Visualization and Transmission, Paris, France."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hoppe, C., Klopschitz, M., Donoser, M., and Bischof, H. (2013, January 9\u201313). Incremental Surface Extraction from Sparse Structure-from-Motion Point Clouds. Proceedings of the British Machine Vision Conference, Bristol, UK.","DOI":"10.5244\/C.27.94"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yang, M., Yu, K., Zhang, C., Li, Z., and Yang, K. (2018, January 18\u201322). Denseaspp for semantic segmentation in street scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA.","DOI":"10.1109\/CVPR.2018.00388"},{"key":"ref_13","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhao, H., Qi, X., Shen, X., Shi, J., and Jia, J. (2018, January 8\u201314). Icnet for real-time semantic segmentation on high-resolution images. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_25"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8\u201314). Bisenet: Bilateral segmentation network for real-time semantic segmentation. Proceedings of the European conference on computer vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01261-8_20"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Su, H., Nie\u00dfner, M., Dai, A., Yan, M., and Guibas, L.J. (2016, January 27\u201330). Volumetric and multi-view cnns for object classification on 3d data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.609"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tchapmi, L., Choy, C., Armeni, I., Gwak, J., and Savarese, S. (2017, January 10\u201312). Segcloud: Semantic segmentation of 3d point clouds. Proceedings of the International Conference on 3D Vision, Qingdao, China.","DOI":"10.1109\/3DV.2017.00067"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Graham, B. (2015). Sparse 3D convolutional neural networks. arXiv.","DOI":"10.5244\/C.29.150"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Graham, B., Engelcke, M., and Van Der Maaten, L. (2018, January 18\u201322). 3d semantic segmentation with submanifold sparse convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA.","DOI":"10.1109\/CVPR.2018.00961"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5432","DOI":"10.1109\/LRA.2020.3007440","article-title":"3D-MiniNet: Learning a 2D Representation From Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation","volume":"5","author":"Alonso","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, C., Wu, B., Wang, Z., Zhan, W., Vajda, P., Keutzer, K., and Tomizuka, M. (2020). Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation. arXiv.","DOI":"10.1007\/978-3-030-58604-1_1"},{"key":"ref_23","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_24","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4\u20139). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the Advances in Neural Information Processing Systems 30, Long Beach, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Riegler, G., Osman Ulusoy, A., and Geiger, A. (2017, January 21\u201326). Octnet: Learning deep 3d representations at high resolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.701"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xu, Q., Sun, X., Wu, C.Y., Wang, P., and Neumann, U. (2020, January 16\u201318). Grid-GCN for Fast and Scalable Point Cloud Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00570"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lei, H., Akhtar, N., and Mian, A. (2020, January 16\u201318). SegGCN: Efficient 3D Point Cloud Segmentation With Fuzzy Spherical Kernel. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01163"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Choy, C., Gwak, J., and Savarese, S. (2019, January 16\u201320). 4D spatio-temporal convnets: Minkowski convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00319"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xiang, Y., and Fox, D. (2017). Da-rnn: Semantic mapping with data associated recurrent neural networks. arXiv.","DOI":"10.15607\/RSS.2017.XIII.013"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Pham, Q.H., Hua, B.S., Nguyen, T., and Yeung, S.K. (2019, January 7\u201311). Real-time progressive 3D semantic segmentation for indoor scenes. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Honolulu, HI, USA.","DOI":"10.1109\/WACV.2019.00121"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cavallari, T., and Di Stefano, L. (2016). Semanticfusion: Joint labeling, tracking and mapping. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-49409-8_55"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, X., and Belaroussi, R. (2016). Semi-dense 3d semantic mapping from monocular slam. arXiv.","DOI":"10.1109\/ITSC.2017.8317942"},{"key":"ref_33","unstructured":"McCormac, J., Handa, A., Davison, A., and Leutenegger, S. (June, January 29). Semanticfusion: Dense 3d semantic mapping with convolutional neural networks. Proceedings of the IEEE International Conference on Robotics and Automation, Singapore."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jeong, J., Yoon, T.S., and Park, J.B. (2018). Towards a meaningful 3D map using a 3D lidar and a camera. Sensors, 18.","DOI":"10.3390\/s18082571"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.neucom.2020.06.004","article-title":"Building and optimization of 3D semantic map based on Lidar and camera fusion","volume":"409","author":"Li","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_36","unstructured":"Yue, Y., Zhao, C., Li, R., Yang, C., Zhang, J., Wen, M., Wang, Y., and Wang, D. (August, January 31). A Hierarchical Framework for Collaborative Probabilistic Semantic Mapping. Proceedings of the IEEE International Conference on Robotics and Automation, Paris, France."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., and Fitzgibbon, A. (2011, January 26\u201329). KinectFusion: Real-time dense surface mapping and tracking. Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland.","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Vineet, V., Miksik, O., Lidegaard, M., Nie\u00dfner, M., Golodetz, S., Prisacariu, V.A., K\u00e4hler, O., Murray, D.W., Izadi, S., and P\u00e9rez, P. (2015, January 26\u201330). Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction. Proceedings of the IEEE International Conference on Robotics and Automation, Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7138983"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel tracking and mapping for small AR workspaces. Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan.","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Curless, B., and Levoy, M. (1996, January 4\u20139). A volumetric method for building complex models from range images. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/237170.237269"},{"key":"ref_41","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017). CARLA: An open urban driving simulator. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Huang, X., Cheng, X., Geng, Q., Cao, B., Zhou, D., Wang, P., Lin, Y., and Yang, R. (2018, January 18\u201322). The apolloscape dataset for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake, UT, USA.","DOI":"10.1109\/CVPRW.2018.00141"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sengupta, S., Greveson, E., Shahrokni, A., and Torr, P.H. (2013, January 6\u201310). Urban 3d semantic modelling using stereo vision. Proceedings of the IEEE International Conference on robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630632"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6264\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:28:35Z","timestamp":1760178515000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6264"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,3]]},"references-count":43,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216264"],"URL":"https:\/\/doi.org\/10.3390\/s20216264","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,11,3]]}}}