{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T04:27:07Z","timestamp":1769747227915,"version":"3.49.0"},"reference-count":57,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T00:00:00Z","timestamp":1680566400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bavarian Ministry of Economic Affairs, Regional Development and Energy"},{"name":"Bayern Innovativ\u2014Bavarian Society"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Advances in deep learning techniques for remote sensing as well as the increased availability of high-resolution data enable the extraction of more detailed information from aerial images. One promising task is the semantic segmentation of roof segments and their orientation. However, the lack of annotated data is a major barrier for deploying respective models on a large scale. Previous research demonstrated the viability of the deep learning approach for the task, but currently, published datasets are small-scale, manually labeled, and rare. Therefore, this paper extends the state of the art by presenting a novel method for the automated generation of large-scale datasets based on semantic 3D city models. Furthermore, we train a model on a dataset 50 times larger than existing datasets and achieve superior performance while applying it to a wider variety of buildings. We evaluate the approach by comparing networks trained on four dataset configurations, including an existing dataset and our novel large-scale dataset. The results show that the network performance measured as intersection over union can be increased from 0.60 for the existing dataset to 0.70 when the large-scale model is applied on the same region. The large-scale model performs superiorly even when applied to more diverse test samples, achieving 0.635. The novel approach contributes to solving the dataset bottleneck and consequently to improving semantic segmentation of roof segments. The resulting remotely sensed information is crucial for applications such as solar potential analysis or urban planning.<\/jats:p>","DOI":"10.3390\/rs15071931","type":"journal-article","created":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T04:09:50Z","timestamp":1680581390000},"page":"1931","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Improving Semantic Segmentation of Roof Segments Using Large-Scale Datasets Derived from 3D City Models and High-Resolution Aerial Imagery"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-3246-8548","authenticated-orcid":false,"given":"Florian L.","family":"Faltermeier","sequence":"first","affiliation":[{"name":"Chair of Geoinformatics, TUM School of Engineering and Design, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7866-1998","authenticated-orcid":false,"given":"Sebastian","family":"Krapf","sequence":"additional","affiliation":[{"name":"Institute of Automotive Technology, Department of Mechanical Engineering, TUM School of Engineering and Design, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7121-5525","authenticated-orcid":false,"given":"Bruno","family":"Willenborg","sequence":"additional","affiliation":[{"name":"Chair of Geoinformatics, TUM School of Engineering and Design, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1456-0423","authenticated-orcid":false,"given":"Thomas H.","family":"Kolbe","sequence":"additional","affiliation":[{"name":"Chair of Geoinformatics, TUM School of Engineering and Design, Technical University of Munich, 80333 Munich, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.JRS.11.042609","article-title":"Comprehensive survey of deep learning in remote sensing: Theories, tools, and challenges for the community","volume":"11","author":"Ball","year":"2017","journal-title":"J. Appl. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Neupane, B., Horanont, T., and Aryal, J. (2021). Deep Learning-Based Semantic Segmentation of Urban Features in Satellite Images: A Review and Meta-Analysis. Remote Sens., 13.","DOI":"10.3390\/rs13040808"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"114417","DOI":"10.1016\/j.eswa.2020.114417","article-title":"A review of deep learning methods for semantic segmentation of remote sensing imagery","volume":"169","author":"Yuan","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.isprsjprs.2020.01.013","article-title":"ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data","volume":"162","author":"Diakogiannis","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_5","unstructured":"Gouet-Brunet, V., Khokhlova, M., Kosti, R., Chen, L., and Yin, X.C. (2020, January 12\u201316). PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training. Proceedings of the 2nd Workshop on Structuring and Understanding of Multimedia heritAge Contents, Seattle, WA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10548","DOI":"10.1109\/JSTARS.2021.3119286","article-title":"PiCoCo: Pixelwise Contrast and Consistency Learning for Semisupervised Building Footprint Segmentation","volume":"14","author":"Kang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/LGRS.2022.3188257","article-title":"Semantic Segmentation of Remote Sensing Images With Sparse Annotations","volume":"19","author":"Hua","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6054","DOI":"10.1109\/TGRS.2017.2719738","article-title":"Learning Aerial Image Segmentation From Online Maps","volume":"55","author":"Kaiser","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TGRS.2016.2612821","article-title":"Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification","volume":"55","author":"Maggiori","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2793","DOI":"10.1109\/TPAMI.2017.2750680","article-title":"Learning Building Extraction in Aerial Scenes with Convolutional Networks","volume":"40","author":"Yuan","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, W., He, C., Fang, J., Zheng, J., Fu, H., and Yu, L. (2019). Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data. Remote Sens., 11.","DOI":"10.3390\/rs11040403"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2135","DOI":"10.1109\/LGRS.2019.2963065","article-title":"GIS-Supervised Building Extraction With Label Noise-Adaptive Fully Convolutional Neural Network","volume":"17","author":"Zhang","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Touzani, S., and Granderson, J. (2021). Open Data and Deep Semantic Segmentation for Automated Extraction of Building Footprints. Remote Sens., 13.","DOI":"10.3390\/rs13132578"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1016\/j.solener.2011.02.022","article-title":"Scalable methodology for the photovoltaic solar energy potential assessment based on available roof surface area: Application to Piedmont Region (Italy)","volume":"85","author":"Bergamasco","year":"2011","journal-title":"Sol. Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1016\/j.solener.2017.06.065","article-title":"Assessment of rooftop photovoltaic potentials at the urban level using publicly available geodata and image recognition techniques","volume":"155","author":"Mainzer","year":"2017","journal-title":"Sol. Energy"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lee, S., Iyengar, S., Feng, M., Shenoy, P., and Maji, S. (2019, January 4\u20138). DeepRoof: A Data-Driven Approach For Solar Potential Estimation Using Rooftop Imagery. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330741"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Krapf, S., Kemmerzell, N., Khawaja Haseeb Uddin, S., Hack V\u00e1zquez, M., Netzler, F., and Lienkamp, M. (2021). Towards Scalable Economic Photovoltaic Potential Analysis Using Aerial Images and Deep Learning. Energies, 14.","DOI":"10.3390\/en14133800"},{"key":"ref_18","first-page":"103098","article-title":"SolarNet: A convolutional neural network-based framework for rooftop solar potential estimation from aerial imagery","volume":"116","author":"Li","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Krapf, S., Bogenrieder, L., Netzler, F., Balke, G., and Lienkamp, M. (2022). RID\u2014Roof Information Dataset for Computer Vision-Based Photovoltaic Potential Assessment. Remote Sens., 14.","DOI":"10.3390\/rs14102299"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Ling, H., Gao, J., Yin, K., Lafleche, J.F., Barriuso, A., Torralba, A., and Fidler, S. (2021, January 20\u201325). DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01001"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lee, J., and Zlatanova, S. (2009). 3D Geo-Information Sciences, Springer. Lecture Notes in Geoinformation and Cartography.","DOI":"10.1007\/978-3-540-87395-2"},{"key":"ref_22","unstructured":"Open Geospatial Consortium (2012). OGC City Geography Markup Language (CityGML) Encoding Standard Version 2.0.0, Open Geospatial Consortium."},{"key":"ref_23","unstructured":"Wysocki, O., Schwab, B., and Willenborg, B. (2022). OloOcki\/awesome-citygml: Release. Zenodo."},{"key":"ref_24","first-page":"234","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","volume":"Volume 9351","author":"Navab","year":"2015","journal-title":"Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015"},{"key":"ref_25","unstructured":"(2023, January 30). Landesamt f\u00fcr Digitalisierung, Breitband und Vermessung. Available online: https:\/\/www.ldbv.bayern.de\/."},{"key":"ref_26","first-page":"W4","article-title":"3D building reconstruction from LiDAR based on a cell decomposition approach","volume":"38","author":"Kada","year":"2009","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/s40965-018-0046-7","article-title":"3DCityDB\u2014A 3D geodatabase solution for the management, analysis, and visualization of semantic 3D city models based on CityGML","volume":"3","author":"Yao","year":"2018","journal-title":"Open Geospat. Data Softw. Stand."},{"key":"ref_28","unstructured":"(2023, January 30). 3D City Database: The Open Source CityGML Database. Available online: https:\/\/github.com\/3dcitydb."},{"key":"ref_29","unstructured":"Faltermeier, F.L. (2023). tum-gis\/citygml-roof-segment-labels: Generate datasets of roof segment labels for aerial imagery derived from CityGML semantic 3D city models for semantic segmentation. Zenodo."},{"key":"ref_30","unstructured":"Bundesministerium f\u00fcr Digitales und Verkehr (BMDV) (2018). Regionalstatistische Raumtypologie (RegioStaR) des BMVI f\u00fcr die Mobilit\u00e4ts- und Verkehrsforschung: Arbeitspapier Version V1.1 (06.06.2018), Bundesministerium f\u00fcr Digitales und Verkehr (BMDV)."},{"key":"ref_31","unstructured":"Bayerische Vermessungsverwaltung (2018). Kundeninformation LoD2 Geb\u00e4udemodelle: Stand 3\/2018, Bayerische Vermessungsverwaltung."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_33","unstructured":"Yakubovskiy, P. (2022, November 14). Segmentation Models. Available online: https:\/\/github.com\/qubvel\/segmentation_models."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jadon, S. (2020, January 27\u201329). A survey of loss functions for semantic segmentation. Proceedings of the 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), Via del Mar, Chile.","DOI":"10.1109\/CIBCB48159.2020.9277638"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sugino, T., Kawase, T., Onogi, S., Kin, T., Saito, N., and Nakajima, Y. (2021). Loss Weightings for Improving Imbalanced Brain Structure Segmentation Using Fully Convolutional Networks. Healthcare, 9.","DOI":"10.3390\/healthcare9080938"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inf. Process. Manag."},{"key":"ref_38","unstructured":"Swiss Confederation Federal Office of Topography Swisstopo (2023, February 10). swissBUILDINGS3D 2.0: 3D Building Models of Switzerland. Available online: https:\/\/www.swisstopo.admin.ch\/en\/geodata\/landscape\/buildings3d2.html."},{"key":"ref_39","unstructured":"Kofler, F., Ezhov, I., Isensee, F., Balsiger, F., Berger, C., Koerner, M., Paetzold, J., Li, H., Shit, S., and McKinley, R. (2021). Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient. arXiv."},{"key":"ref_40","unstructured":"Reinke, A., Tizabi, M.D., Sudre, C.H., Eisenmann, M., R\u00e4dsch, T., Baumgartner, M., Acion, L., Antonelli, M., Arbel, T., and Bakas, S. (2021). Common Limitations of Image Processing Metrics: A Picture Story. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15\u201320). Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00075"},{"key":"ref_42","unstructured":"de Marsico, M., Di Sanniti Baja, G., and Fred, A. (2019, January 19\u201321). Deep Neural Networks with Intersection over Union Loss for Binary Image Segmentation. Proceedings of the ICPRAM 2019, Prague, Czech Republic."},{"key":"ref_43","unstructured":"Meila, M., and Zhang, T. (2021, January 18\u201324). Learning Generalized Intersection Over Union for Dense Pixelwise Prediction. Proceedings of the 38th International Conference on Machine Learning, Virtual Event."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bischke, B., Helber, P., Folz, J., Borth, D., and Dengel, A. (2019, January 22\u201325). Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8803050"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"209517","DOI":"10.1109\/ACCESS.2020.3038225","article-title":"Building Footprint Extraction from High Resolution Aerial Images Using Generative Adversarial Network (GAN) Architecture","volume":"8","author":"Abdollahi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1080\/2150704X.2021.1895444","article-title":"Semantic Segmentation of High Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing","volume":"12","author":"Collier","year":"2021","journal-title":"Remote Sens. Lett."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, C., Li, S., Xiong, D., Fang, P., and Liao, M. (2020). Remote Sensing Image Semantic Segmentation Based on Edge Information Guidance. Remote Sens., 12.","DOI":"10.3390\/rs12091501"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2021.3108781","article-title":"Boundary Enhancement Semantic Segmentation for Building Extraction From Remote Sensed Image","volume":"60","author":"Jung","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Li, Z., Xin, Q., Sun, Y., and Cao, M. (2021). A Deep Learning-Based Framework for Automated Extraction of Building Footprint Polygons from Very High-Resolution Aerial Imagery. Remote Sens., 13.","DOI":"10.3390\/rs13183630"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"4595","DOI":"10.1109\/JSTARS.2021.3073994","article-title":"E-D-Net: Automatic Building Extraction From High-Resolution Aerial Images With Boundary Information","volume":"14","author":"Zhu","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2178","DOI":"10.1109\/TGRS.2019.2954461","article-title":"Toward Automatic Building Footprint Delineation From Aerial Images Using CNN and Regularization","volume":"58","author":"Wei","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"107404","DOI":"10.1016\/j.patcog.2020.107404","article-title":"U2-Net: Going deeper with nested U-structure for salient object detection","volume":"106","author":"Qin","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Wei, X., Li, X., Liu, W., Zhang, L., Cheng, D., Ji, H., Zhang, W., and Yuan, K. (2021). Building Outline Extraction Directly Using the U2-Net Semantic Segmentation Model from High-Resolution Aerial Images and a Comparison Study. Remote Sens., 13.","DOI":"10.3390\/rs13163187"},{"key":"ref_54","unstructured":"Mortensen, E. (2021, January 10\u201317). Segmenter: Transformer for Semantic Segmentation. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada."},{"key":"ref_55","first-page":"12077","article-title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","volume":"Volume 34","author":"Ranzato","year":"2021","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20\u201325). Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_57","unstructured":"Bayerische Vermessungsverwaltung (2023, January 30). Kostenfreie Geodaten (OpenData). Available online: https:\/\/geodaten.bayern.de\/opengeodata\/."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/7\/1931\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:09:37Z","timestamp":1760123377000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/7\/1931"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,4]]},"references-count":57,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["rs15071931"],"URL":"https:\/\/doi.org\/10.3390\/rs15071931","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,4]]}}}