{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:17:49Z","timestamp":1778811469677,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,13]],"date-time":"2023-01-13T00:00:00Z","timestamp":1673568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Science and Engineering Research Council of Canada (NSERC)","doi-asserted-by":"publisher","award":["RGPIN-2022-05051"],"award-info":[{"award-number":["RGPIN-2022-05051"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the aftermath of a natural hazard, rapid and accurate building damage assessment from remote sensing imagery is crucial for disaster response and rescue operations. Although recent deep learning-based studies have made considerable improvements in assessing building damage, most state-of-the-art works focus on pixel-based, multi-stage approaches, which are more complicated and suffer from partial damage recognition issues at the building-instance level. In the meantime, it is usually time-consuming to acquire sufficient labeled samples for deep learning applications, making a conventional supervised learning pipeline with vast annotation data unsuitable in time-critical disaster cases. In this study, we present an end-to-end building damage assessment framework integrating multitask semantic segmentation with semi-supervised learning to tackle these issues. Specifically, a multitask-based Siamese network followed by object-based post-processing is first constructed to solve the semantic inconsistency problem by refining damage classification results with building extraction results. Moreover, to alleviate labeled data scarcity, a consistency regularization-based semi-supervised semantic segmentation scheme with iteratively perturbed dual mean teachers is specially designed, which can significantly reinforce the network perturbations to improve model performance while maintaining high training efficiency. Furthermore, a confidence weighting strategy is embedded into the semi-supervised pipeline to focus on convincing samples and reduce the influence of noisy pseudo-labels. The comprehensive experiments on three benchmark datasets suggest that the proposed method is competitive and effective in building damage assessment under the circumstance of insufficient labels, which offers a potential artificial intelligence-based solution to respond to the urgent need for timeliness and accuracy in disaster events.<\/jats:p>","DOI":"10.3390\/rs15020478","type":"journal-article","created":{"date-parts":[[2023,1,13]],"date-time":"2023-01-13T05:09:32Z","timestamp":1673586572000},"page":"478","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["MS4D-Net: Multitask-Based Semi-Supervised Semantic Segmentation Framework with Perturbed Dual Mean Teachers for Building Damage Assessment from High-Resolution Remote Sensing Imagery"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1656-560X","authenticated-orcid":false,"given":"Yongjun","family":"He","sequence":"first","affiliation":[{"name":"Department of Geography and Environment, The University of Western Ontario, London, ON N6A 5C2, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8404-0530","authenticated-orcid":false,"given":"Jinfei","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Geography and Environment, The University of Western Ontario, London, ON N6A 5C2, Canada"},{"name":"Institute for Earth and Space Exploration, The University of Western Ontario, London, ON N6A 3K7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chunhua","family":"Liao","sequence":"additional","affiliation":[{"name":"School of Geospatial Engineering and Science, Sun Yat-sen University, Zhuhai 519082, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1321-510X","authenticated-orcid":false,"given":"Xin","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Geography and Environment, The University of Western Ontario, London, ON N6A 5C2, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Shan","sequence":"additional","affiliation":[{"name":"Department of Geography and Environment, The University of Western Ontario, London, ON N6A 5C2, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ci, T., Liu, Z., and Wang, Y. (2019). Assessment of the Degree of Building Damage Caused by Disaster Using Convolutional Neural Networks in Combination with Ordinal Regression. Remote Sens., 11.","DOI":"10.3390\/rs11232858"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1007\/s10518-006-9005-2","article-title":"Classification of Collapsed Buildings for Fast Damage and Loss Assessment","volume":"4","author":"Schweier","year":"2006","journal-title":"Bull. Earthq. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liao, C., Wang, J., Xie, Q., Baz, A.A., Huang, X., Shang, J., and He, Y. (2020). Synergistic Use of Multi-Temporal RADARSAT-2 and VEN\u00b5S Data for Crop Classification Based on 1D Convolutional Neural Network. Remote Sens., 12.","DOI":"10.3390\/rs12050832"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"112636","DOI":"10.1016\/j.rse.2021.112636","article-title":"Building Damage Assessment for Rapid Disaster Response with a Deep Object-Based Semantic Change Detection Framework: From Natural Disasters to Man-Made Disasters","volume":"265","author":"Zheng","year":"2021","journal-title":"Remote Sens. Environ."},{"key":"ref_5","unstructured":"Gupta, R., Goodman, B., Patel, N., Hosfelt, R., Sajeev, S., Heim, E., Doshi, J., Lucas, K., Choset, H., and Gaston, M. (2019, January 16\u201317). Creating XBD: A Dataset for Assessing Building Damage from Satellite Imagery. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA."},{"key":"ref_6","first-page":"1","article-title":"BDANet: Multiscale Convolutional Neural Network with Cross-Directional Attention for Building Damage Assessment from Satellite Images","volume":"60","author":"Shen","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1080\/22797254.2018.1527662","article-title":"Building Damage Assessment Scale Tailored to Remote Sensing Vertical Imagery","volume":"51","author":"Cotrufo","year":"2018","journal-title":"Eur. J. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chen, X., Yuan, Y., Zeng, G., and Wang, J. (2021, January 20\u201325). Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00264"},{"key":"ref_9","first-page":"102456","article-title":"A Review on Deep Learning in UAV Remote Sensing","volume":"102","author":"Osco","year":"2021","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Oludare, V., Kezebou, L., Panetta, K., and Agaian, S. (2021, January 12\u201316). Semi-Supervised Learning for Improved Post-Disaster Damage Assessment from Satellite Imagery. Proceedings of the Multimodal Image Exploitation and Learning 2021, SPIE Conference Proceedings, Online.","DOI":"10.1117\/12.2586232"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"81","DOI":"10.14419\/ijet.v7i1.8.9977","article-title":"Semi-Supervised Learning: A Brief Review","volume":"7","author":"Reddy","year":"2018","journal-title":"Int. J. Eng. Technol."},{"key":"ref_12","unstructured":"Hu, H., Wei, F., Hu, H., Ye, Q., Cui, J., and Wang, L. (2021, January 6\u201314). Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, Y., Tian, Y., Chen, Y., Liu, F., Belagiannis, V., and Carneiro, G. (2022, January 19\u201324). Perturbed and Strict Mean Teachers for Semi-Supervised Semantic Segmentation. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00422"},{"key":"ref_14","unstructured":"Laine, S., and Aila, T. (2017, January 24\u201326). Temporal Ensembling for Semi-Supervised Learning. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_15","unstructured":"Tarvainen, A., and Valpola, H. (2017, January 4\u20139). Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. Proceedings of the 31 Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ke, Z., Wang, D., Yan, Q., Ren, J., and Lau, R. (November, January 27). Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00683"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ouali, Y., Hudelot, C., and Tami, M. (2020, January 13\u201319). Semi-Supervised Semantic Segmentation With Cross-Consistency Training. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01269"},{"key":"ref_18","unstructured":"Zou, Y., Zhang, Z., Zhang, H., Li, C.-L., Bian, X., Huang, J.-B., and Pfister, T. (2021, January 3\u20137). PseudoSeg: Designing Pseudo Labels for Semantic Segmentation. Proceedings of the International Conference on Learning Representations, Online."},{"key":"ref_19","unstructured":"French, G., Laine, S., Aila, T., Mackiewicz, M., and Finlayson, G. (2020, January 7\u201310). Semi-Supervised Semantic Segmentation Needs Strong, Varied Perturbations. Proceedings of the 31st British Machine Vision Conference, Online."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ke, Z., Qiu, D., Li, K., Yan, Q., and Lau, R.W.H. (2020, January 23\u201328). Guided Collaborative Training for Pixel-Wise Semi-Supervised Learning. Proceedings of the 16th IEEE European Conference Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58601-0_26"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, Y., Wang, J., Liao, C., Shan, B., and Zhou, X. (2022). ClassHyPer: ClassMix-Based Hybrid Perturbations for Deep Semi-Supervised Semantic Segmentation of Remote Sensing Imagery. Remote Sens., 14.","DOI":"10.3390\/rs14040879"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10548","DOI":"10.1109\/JSTARS.2021.3119286","article-title":"PiCoCo: Pixelwise Contrast and Consistency Learning for Semisupervised Building Footprint Segmentation","volume":"14","author":"Kang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"9086","DOI":"10.1109\/JSTARS.2022.3215730","article-title":"Enhancement of Urban Floodwater Mapping From Aerial Imagery With Dense Shadows via Semi-Supervised Learning","volume":"15","author":"He","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_24","first-page":"102881","article-title":"Semi-Supervised Semantic Segmentation Framework with Pseudo Supervisions for Land-Use\/Land-Cover Mapping in Coastal Areas","volume":"112","author":"Chen","year":"2022","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, B., Zhang, Y., Li, Y., Wan, Y., Guo, H., Zheng, Z., and Yang, K. (2022). Semi-Supervised Deep Learning via Transformation Consistency Regularization for Remote Sensing Image Semantic Segmentation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 1\u201315.","DOI":"10.1109\/JSTARS.2022.3203750"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sun, C., Wu, J., Chen, H., and Du, C. (2022). SemiSANet: A Semi-Supervised High-Resolution Remote Sensing Image Change Detection Model Using Siamese Networks with Graph Attention. Remote Sens., 14.","DOI":"10.3390\/rs14122801"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"112589","DOI":"10.1016\/j.rse.2021.112589","article-title":"Deep Building Footprint Update Network: A Semi-Supervised Method for Updating Existing Building Footprint from Bi-Temporal Remote Sensing Images","volume":"264","author":"Guo","year":"2021","journal-title":"Remote Sens. Environ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5891","DOI":"10.1109\/TGRS.2020.3011913","article-title":"SemiCDNet: A Semisupervised Convolutional Neural Network for Change Detection in High Resolution Remote-Sensing Images","volume":"59","author":"Peng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5570","DOI":"10.1109\/JSTARS.2022.3188025","article-title":"FMWDCT: Foreground Mixup Into Weighted Dual-Network Cross Training for Semisupervised Remote Sensing Road Extraction","volume":"15","author":"You","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_30","first-page":"5623217","article-title":"Semi-Supervised Building Footprint Generation With Feature and Output Consistency Training","volume":"60","author":"Li","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"045008","DOI":"10.1117\/1.JRS.12.045008","article-title":"Semisupervised Classification of Hurricane Damage from Postevent Aerial Imagery Using Deep Learning","volume":"12","author":"Li","year":"2018","journal-title":"J. Appl. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1193\/1.1650865","article-title":"Using High-Resolution Satellite Images for Post-Earthquake Building Damage Assessment: A Study Following the 26 January 2001 Gujarat Earthquake","volume":"20","author":"Saito","year":"2004","journal-title":"Earthq. Spectra"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.isprsjprs.2013.06.011","article-title":"A Comprehensive Review of Earthquake-Induced Building Damage Detection with Remote Sensing Techniques","volume":"84","author":"Dong","year":"2013","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lucks, L., Bulatov, D., Th\u00f6nnessen, U., and B\u00f6ge, M. (2019, January 25\u201327). Superpixel-Wise Assessment of Building Damage from Aerial Images. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Prague, Czech Republic.","DOI":"10.5220\/0007253800002108"},{"key":"ref_35","first-page":"V-112","article-title":"Urban Building Damage Detection from Very High Resolution Imagery Using One-Class SVM and Spatial Relations","volume":"5","author":"Li","year":"2009","journal-title":"Int. Geosci. Remote Sens. Symp. (IGARSS)"},{"key":"ref_36","first-page":"102899","article-title":"Operational Earthquake-Induced Building Damage Assessment Using CNN-Based Direct Remote Sensing Change Detection on Superpixel Level","volume":"112","author":"Qing","year":"2022","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_37","unstructured":"Lee, J., Xu, J.Z., Sohn, K., Lu, W., Berthelot, D., Gur, I., Khaitan, P., Ke-Wei, H., Koupparis, K., and Kowatsch, B. (2020). Assessing Post-Disaster Damage from Satellite Imagery Using Semi-Supervised Learning Techniques. arXiv."},{"key":"ref_38","unstructured":"Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C. (2019). MixMatch: A Holistic Approach to Semi-Supervised Learning. arXiv."},{"key":"ref_39","unstructured":"Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E.D., Kurakin, A., Zhang, H., and Raffel, C. (2020, January 6\u201312). FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wu, K., Otoo, E., and Shoshani, A. (2005, January 12\u201317). Optimizing Connected Component Labeling Algorithms. Proceedings of the Medical Imaging 2005: Image Processing, San Diego, CA, USA.","DOI":"10.1117\/12.596105"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Yun, S., Han, D., Chun, S., Oh, S.J., Yoo, Y., and Choe, J. (November, January 27). CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00612"},{"key":"ref_42","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Smith, L.N. (2017, January 24\u201331). Cyclical Learning Rates for Training Neural Networks. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.58"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"101631","DOI":"10.1016\/j.media.2019.101631","article-title":"Siam-U-Net: Encoder-Decoder Siamese Network for Knee Cartilage Tracking in Ultrasound Images","volume":"60","author":"Dunnhofer","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.isprsjprs.2017.03.001","article-title":"Disaster Damage Detection through Synergistic Use of Deep Learning and 3D Point Cloud Features Derived from Very High Resolution Oblique Aerial Images, and Multiple-Kernel-Learning","volume":"140","author":"Vetrivel","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/2\/478\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:05:19Z","timestamp":1760119519000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/2\/478"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,13]]},"references-count":45,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["rs15020478"],"URL":"https:\/\/doi.org\/10.3390\/rs15020478","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,13]]}}}