{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T23:30:39Z","timestamp":1777505439739,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T00:00:00Z","timestamp":1628812800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the China Scholarship Council","award":["201608510073"],"award-info":[{"award-number":["201608510073"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>An accurate understanding of urban objects is critical for urban modeling, intelligent infrastructure planning and city management. The semantic segmentation of light detection and ranging (LiDAR) point clouds is a fundamental approach for urban scene analysis. Over the last years, several methods have been developed to segment urban furniture with point clouds. However, the traditional processing of large amounts of spatial data has become increasingly costly, both time-wise and financially. Recently, deep learning (DL) techniques have been increasingly used for 3D segmentation tasks. Yet, most of these deep neural networks (DNNs) were conducted on benchmarks. It is, therefore, arguable whether DL approaches can achieve the state-of-the-art performance of 3D point clouds segmentation in real-life scenarios. In this research, we apply an adapted DNN (ARandLA-Net) to directly process large-scale point clouds. In particular, we develop a new paradigm for training and validation, which presents a typical urban scene in central Europe (Munzingen, Freiburg, Baden-W\u00fcrttemberg, Germany). Our dataset consists of nearly 390 million dense points acquired by Mobile Laser Scanning (MLS), which has a rather larger quantity of sample points in comparison to existing datasets and includes meaningful object categories that are particular to applications for smart cities and urban planning. We further assess the DNN on our dataset and investigate a number of key challenges from varying aspects, such as data preparation strategies, the advantage of color information and the unbalanced class distribution in the real world. The final segmentation model achieved a mean Intersection-over-Union (mIoU) score of 54.4% and an overall accuracy score of 83.9%. Our experiments indicated that different data preparation strategies influenced the model performance. Additional RGB information yielded an approximately 4% higher mIoU score. Our results also demonstrate that the use of weighted cross-entropy with inverse square root frequency loss led to better segmentation performance than when other losses were considered.<\/jats:p>","DOI":"10.3390\/rs13163220","type":"journal-article","created":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T09:22:38Z","timestamp":1628846558000},"page":"3220","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Towards Urban Scene Semantic Segmentation with Deep Learning from LiDAR Point Clouds: A Case Study in Baden-W\u00fcrttemberg, Germany"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1975-3235","authenticated-orcid":false,"given":"Yanling","family":"Zou","sequence":"first","affiliation":[{"name":"Chair of Remote Sensing and Landscape Information Systems, University of Freiburg, Tennenbacherstr. 4, 79106 Freiburg, Germany"},{"name":"College of Control Engineering, Chengdu University of Information Technology, Chengdu 610225, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Holger","family":"Weinacker","sequence":"additional","affiliation":[{"name":"Chair of Remote Sensing and Landscape Information Systems, University of Freiburg, Tennenbacherstr. 4, 79106 Freiburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Barbara","family":"Koch","sequence":"additional","affiliation":[{"name":"Chair of Remote Sensing and Landscape Information Systems, University of Freiburg, Tennenbacherstr. 4, 79106 Freiburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.isprsjprs.2019.02.015","article-title":"Pairwise coarse registration of point clouds in urban scenes using voxel-based 4-planes congruent sets","volume":"151","author":"Xu","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.isprsjprs.2020.10.002","article-title":"Unsupervised scene adaptation for semantic segmentation of urban mobile laser scanning point clouds","volume":"169","author":"Luo","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"91","DOI":"10.5194\/isprs-annals-IV-1-W1-91-2017","article-title":"SEMANTIC3D.NET: A new large-scale point cloud classification benchmark","volume":"IV-1\u2013W1","author":"Hackel","year":"2017","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.isprsjprs.2017.10.001","article-title":"Pairwise registration of TLS point clouds using covariance descriptors and a non-cooperative game","volume":"134","author":"Zai","year":"2017","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.isprsjprs.2014.06.015","article-title":"Keypoint-based 4-Points Congruent Sets\u2014Automated marker-less registration of laser scans","volume":"96","author":"Theiler","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.isprsjprs.2015.08.007","article-title":"Globally consistent registration of terrestrial laser scans via graph optimization","volume":"109","author":"Theiler","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., and Markham, A. (2019, January 16\u201320). RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR42600.2020.01112"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Munoz, D., Bagnell, J.A., Vandapel, N., and Hebert, M. (2009, January 20\u201325). Contextual Classification with Functional Max-Margin Markov Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206590"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1177\/0278364918767506","article-title":"Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification","volume":"37","author":"Roynard","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tan, W., Qin, N., Ma, L., Li, Y., Du, J., Cai, G., Yang, K., and Li, J. (2020, January 14\u201319). Toronto-3D: A large-scale mobile lidar dataset for semantic segmentation of urban roadways. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00109"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., and Bennamoun, M. (2020). Deep learning for 3D point clouds: A survey. IEEE Trans. Pattern Anal. Mach. Intell., in press.","DOI":"10.1109\/TPAMI.2020.3005434"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Griffiths, D., and Boehm, J. (2019). A Review on Deep Learning Techniques for 3D Sensed Data Classification. Remote Sens., 11.","DOI":"10.3390\/rs11121499"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Graham, B., Engelcke, M., and van der Maaten, L. (2018, January 18\u201322). 3D Semantic Segmentation With Submanifold Sparse Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00961"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Choy, C., Gwak, J., and Savarese, S. (2019, January 16\u201320). 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00319"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Le, T., and Duan, Y. (2018, January 18\u201322). PointGrid: A Deep Network for 3D Shape Understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00959"},{"key":"ref_16","unstructured":"Liu, Z., Tang, H., Lin, Y., and Han, S. (2019, January 8\u201314). Point-Voxel CNN for Efficient 3D Deep Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Meng, H.Y., Gao, L., Lai, Y.K., and Manocha, D. (2019, January 16\u201320). VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00859"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhou, Z., David, P., Yue, X., Xi, Z., Gong, B., and Foroosh, H. (2020, January 14\u201319). PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00962"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/MGRS.2019.2937630","article-title":"Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation","volume":"8","author":"Xie","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lyu, Y., Huang, X., and Zhang, Z. (2020, January 14\u201319). Learning to Segment 3D Point Clouds in 2D Image Space. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01227"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cortinhal, T., Tzelepis, G., and Aksoy, E.E. (2020). SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving. arXiv.","DOI":"10.1007\/978-3-030-64559-5_16"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, C., Wu, B., Wang, Z., Zhan, W., Vajda, P., Keutzer, K., and Tomizuka, M. (2020, January 23\u201328). Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58604-1_1"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"04020026","DOI":"10.1061\/(ASCE)ME.1943-5479.0000774","article-title":"Architecting Smart City Digital Twins: Combined Semantic Model and Machine Learning Approach","volume":"36","author":"Austin","year":"2020","journal-title":"J. Manag. Eng."},{"key":"ref_24","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2016). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. arXiv."},{"key":"ref_25","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. arXiv."},{"key":"ref_26","first-page":"820","article-title":"PointCNN: Convolution on Xtransformed points","volume":"31","author":"Li","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tatarchenko, M., Park, J., Koltun, V., and Zhou, Q.Y. (2018, January 18\u201322). Tangent Convolutions for Dense Prediction in 3D. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00409"},{"key":"ref_28","first-page":"1","article-title":"Dynamic Graph CNN for Learning on Point Clouds","volume":"38","author":"Wang","year":"2019","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., and Guibas, L.J. (2019, January 27\u201328). KPConv: Flexible and Deformable Convolution for Point Clouds. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00651"},{"key":"ref_30","unstructured":"Loic, L., and Martin, S. (2018, January 18\u201322). Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Huang, Q., Wang, W., and Neumann, U. (2018). Recurrent Slice Networks for 3D Segmentation on Point Clouds. arXiv.","DOI":"10.1109\/CVPR.2018.00278"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Hua, B.S., and Yeung, S.K. (2019, January 27\u201328). ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00169"},{"key":"ref_33","first-page":"90","article-title":"TREESVIS: A software system for simultaneous ED-real-time visualisation of DTM, DSM, laser raw data, multispectral data, simple tree and building models","volume":"36","author":"Weinacker","year":"2004","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_34","unstructured":"Girardeau-Montaut, D. (2021, June 22). CloudCompare. Available online: https:\/\/www.danielgm.net\/cc\/."},{"key":"ref_35","unstructured":"Rosu, R.A., Sch\u00fctt, P., Quenzel, J., and Behnke, S. (2020, January 12\u201316). LatticeNet: Fast point cloud segmentation using permutohedral lattices. Proceedings of the Robotics: Science and Systems (RSS), Online."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Berman, M., Rannen Triki, A., and Blaschko, M.B. (2018, January 18\u201322). The Lov\u00e1sz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00464"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Varney, N., Asari, V., and Graehling, Q. (2020, January 14\u201319). DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00101"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., and Markham, A. (2021, January 19\u201324). Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LO, USA.","DOI":"10.1109\/CVPR46437.2021.00494"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"3716","DOI":"10.3390\/rs6053716","article-title":"Automatic Segmentation of Raw LIDAR Data for Extraction of Building Roofs","volume":"6","author":"Awrangjeb","year":"2014","journal-title":"Remote Sens."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.isprsjprs.2015.01.011","article-title":"Octree-based region growing for point cloud segmentation","volume":"104","author":"Vo","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1404","DOI":"10.1016\/j.patcog.2014.10.014","article-title":"Outlier Detection and Robust Normal-Curvature Estimation in Mobile Laser Scanning 3D Point Cloud Data","volume":"48","author":"Nurunnabi","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_42","unstructured":"Kingma, D., and Ba, J. (2014, January 14\u201316). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_43","unstructured":"Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., and Shelhamer, E. (2014). cuDNN: Efficient Primitives for Deep Learning. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lang, I., Manor, A., and Avidan, S. (2020, January 14\u201319). SampleNet: Differentiable Point Cloud Sampling. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00760"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Dovrat, O., Lang, I., and Avidan, S. (2018, January 18\u201322). Learning to Sample. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Lake City, UT, USA.","DOI":"10.1109\/CVPR.2019.00287"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Xu, Q., Sun, X., Wu, C.Y., Wang, P., and Neumann, U. (2020, January 14\u201319). Grid-GCN for Fast and Scalable Point Cloud Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00570"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Yan, X., Zheng, C., Li, Z., Wang, S., and Cui, S. (2020, January 14\u201319). PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00563"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/16\/3220\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:45:35Z","timestamp":1760165135000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/16\/3220"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,13]]},"references-count":47,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["rs13163220"],"URL":"https:\/\/doi.org\/10.3390\/rs13163220","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,13]]}}}