{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T16:02:05Z","timestamp":1782403325376,"version":"3.54.5"},"reference-count":44,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T00:00:00Z","timestamp":1646179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to the inconsistent spatiotemporal spectral scales, a remote sensing dataset over a large-scale area and over long-term time series will have large variations and large statistical distribution features, which will lead to a performance drop of the deep learning model that is only trained on the source domain. For building an extraction task, deep learning methods perform weak generalization from the source domain to the other domain. To solve the problem, we propose a Capsule\u2013Encoder\u2013Decoder model. We use a vector named capsule to store the characteristics of the building and its parts. In our work, the encoder extracts capsules from remote sensing images. Capsules contain the information of the buildings\u2019 parts. Additionally, the decoder calculates the relationship between the target building and its parts. The decoder corrects the buildings\u2019 distribution and up-samples them to extract target buildings. Using remote sensing images in the lower Yellow River as the source dataset, building extraction experiments were trained on both our method and the mainstream methods. Compared with the mainstream methods on the source dataset, our method achieves convergence faster, and our method shows higher accuracy. Significantly, without fine tuning, our method can reduce the error rates of building extraction results on an almost unfamiliar dataset. The building parts\u2019 distribution in capsules has high-level semantic information, and capsules can describe the characteristics of buildings more comprehensively, which are more explanatory. The results prove that our method can not only effectively extract buildings but also perform great generalization from the source remote sensing dataset to another.<\/jats:p>","DOI":"10.3390\/rs14051235","type":"journal-article","created":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T22:53:25Z","timestamp":1646261605000},"page":"1235","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Capsule\u2013Encoder\u2013Decoder: A Method for Generalizable Building Extraction from Remote Sensing Images"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4258-1301","authenticated-orcid":false,"given":"Zhenchao","family":"Tang","sequence":"first","affiliation":[{"name":"School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China"},{"name":"School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen 518107, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9213-9832","authenticated-orcid":false,"given":"Calvin Yu-Chian","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen 518107, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengzhen","family":"Jiang","sequence":"additional","affiliation":[{"name":"Center of Geographic Information, Yellow River Engineering Consulting Co., Ltd., Zhengzhou 450003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2488-7243","authenticated-orcid":false,"given":"Dongying","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiran","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Water Science and Engineering, Zhengzhou University, Zhengzhou 450001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7544-2928","authenticated-orcid":false,"given":"Zhiming","family":"Hong","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4873-0697","authenticated-orcid":false,"given":"Huaiwei","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Xu, Y., Wu, L., Xie, Z., and Chen, Z. (2018). Building extraction in very high resolution remote sensing imagery using deep learning and guided filters. Remote Sens., 10.","DOI":"10.3390\/rs10010144"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Guo, M., Liu, H., Xu, Y., and Huang, Y. (2020). Building extraction based on U-Net with an attention block and multiple losses. Remote Sens., 12.","DOI":"10.3390\/rs12091400"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TGRS.2018.2858817","article-title":"Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set","volume":"57","author":"Ji","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, K., Fu, K., Gao, X., Yan, M., Sun, X., and Zhang, H. (2017, January 23\u201328). Building extraction from remote sensing images with deep learning in a supervised manner. Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA.","DOI":"10.1109\/IGARSS.2017.8127295"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, M., Wu, J., Liu, L., Zhao, W., Tian, F., Shen, Q., Zhao, B., and Du, R. (2021). DR-Net: An improved network for building extraction from high resolution remote sensing image. Remote Sens., 13.","DOI":"10.3390\/rs13020294"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1109\/TNNLS.2020.3041646","article-title":"A Local-Global Dual-Stream Network for Building Extraction From Very-High-Resolution Remote Sensing Images","volume":"33","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2611","DOI":"10.1109\/JSTARS.2021.3058097","article-title":"Attention-Gate-Based Encoder\u2013Decoder Network for Automatical Building Extraction","volume":"14","author":"Deng","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4595","DOI":"10.1109\/JSTARS.2021.3073994","article-title":"ED-Net: Automatic Building Extraction From High-Resolution Aerial Images With Boundary Information","volume":"14","author":"Zhu","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Shao, Z., Tang, P., Wang, Z., Saleem, N., Yam, S., and Sommai, C. (2020). BRRNet: A fully convolutional neural network for automatic building extraction from high-resolution remote sensing images. Remote Sens., 12.","DOI":"10.3390\/rs12061050"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zou, Z., Shi, T., Li, W., Zhang, Z., and Shi, Z. (2020). Do game data generalize well for remote sensing image segmentation?. Remote Sens., 12.","DOI":"10.3390\/rs12020275"},{"key":"ref_11","unstructured":"Zhang, R. (2019, January 10\u201315). Making convolutional networks shift-invariant again. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2793","DOI":"10.1109\/TPAMI.2017.2750680","article-title":"Learning building extraction in aerial scenes with convolutional networks","volume":"40","author":"Yuan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10468","DOI":"10.1109\/JSTARS.2021.3119001","article-title":"ReSFlow: A Remote Sensing Imagery Data-Flow for Improved Model Generalization","volume":"14","author":"Lunga","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sheng, H., Chen, X., Su, J., Rajagopal, R., and Ng, A. (2020, January 14\u201319). Effective data fusion with generalized vegetation index: Evidence from land cover segmentation in agriculture. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00038"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, B., Cao, F., and Ye, H. (2022). A Novel Method for Hyperspectral Image Classification: Deep Network with Adaptive Graph Structure Integration. IEEE Trans. Geosci. Remote Sens.","DOI":"10.1109\/TGRS.2022.3150349"},{"key":"ref_16","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fu, X., and Qu, H. (2018, January 3\u20136). Research on semantic segmentation of high-resolution remote sensing image based on full convolutional neural network. Proceedings of the 2018 12th International Symposium on Antennas, Propagation and EM Theory (ISAPE), Hangzhou, China.","DOI":"10.1109\/ISAPE.2018.8634106"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zheng, X., and Chen, T. (October, January 26). Segmentation of High Spatial Resolution Remote Sensing Image based On U-Net Convolutional Networks. Proceedings of the IGARSS 2020\u20142020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA.","DOI":"10.1109\/IGARSS39084.2020.9324600"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ye, Z., Fu, Y., Gan, M., Deng, J., Comber, A., and Wang, K. (2019). Building extraction from very high resolution aerial imagery using joint attention deep neural network. Remote Sens., 11.","DOI":"10.3390\/rs11242970"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22\u201329). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","volume":"40","author":"Chen","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","first-page":"718","article-title":"A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields","volume":"48","author":"Zuo","year":"2019","journal-title":"Acta Geod. Cartogr. Sin"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3252","DOI":"10.1109\/JSTARS.2018.2860989","article-title":"Semantic segmentation for high spatial resolution remote sensing images based on convolution neural network and pyramid pooling module","volume":"11","author":"Yu","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_29","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lin, Y., Xu, D., Wang, N., Shi, Z., and Chen, Q. (2020). Road extraction from very-high-resolution remote sensing images via a nested SE-Deeplab model. Remote Sens., 12.","DOI":"10.3390\/rs12182985"},{"key":"ref_31","unstructured":"Sabour, S., Frosst, N., and Hinton, G.E. (2017). Dynamic routing between capsules. arXiv."},{"key":"ref_32","unstructured":"Hinton, G.E., Sabour, S., and Frosst, N. (2018). Matrix capsules with EM routing. International Conference on Learning Representations, Available online: https:\/\/openreview.net\/forum?id=HJWLfGWRb&noteId=rk5MadsMf&noteId=rk5MadsMf."},{"key":"ref_33","unstructured":"Kosiorek, A.R., Sabour, S., Teh, Y.W., and Hinton, G.E. (2019). Stacked capsule autoencoders. arXiv."},{"key":"ref_34","unstructured":"Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., and Teh, Y.W. (2019, January 17\u201319). Set transformer: A framework for attention-based permutation-invariant neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Nagoya, Japan."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1109\/LGRS.2020.2986380","article-title":"Capsule feature pyramid network for building footprint extraction from high-resolution aerial imagery","volume":"18","author":"Yu","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_36","first-page":"1","article-title":"Building Extraction From Remote Sensing Imagery With a High-Resolution Capsule Network","volume":"19","author":"Yu","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Xiao, T., Liu, Y., Zhou, B., Jiang, Y., and Sun, J. (2018, January 8\u201314). Unified perceptual parsing for scene understanding. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01228-1_26"},{"key":"ref_38","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_39","unstructured":"Chen, C., Li, O., Tao, C., Barnett, A.J., Su, J., and Rudin, C. (2018). This looks like that: Deep learning for interpretable image recognition. arXiv."},{"key":"ref_40","unstructured":"Mnih, V. (2013). Machine Learning for Aerial Image Labeling. [Ph.D. Thesis, University of Toronto (Canada)]."},{"key":"ref_41","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_42","first-page":"279","article-title":"Land cover classification using sematic image segmentation with deep learning","volume":"35","author":"Lee","year":"2019","journal-title":"Korean J. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","volume":"37","author":"He","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","unstructured":"Wu, Z., Shen, C., and Hengel, A.v.d. (2016). Bridging category-level and instance-level semantic image segmentation. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/5\/1235\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:31:01Z","timestamp":1760135461000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/5\/1235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,2]]},"references-count":44,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["rs14051235"],"URL":"https:\/\/doi.org\/10.3390\/rs14051235","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,2]]}}}