{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T17:31:22Z","timestamp":1773768682061,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,2,18]],"date-time":"2022-02-18T00:00:00Z","timestamp":1645142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Remote sensing is an effective method of evaluating building damage after a large-scale natural disaster, such as an earthquake or a typhoon. In recent years, with the development of computer vision technology, deep learning algorithms have been used for damage assessment from aerial images. In April 2016, a series of earthquakes hit the Kyushu region, Japan, and caused severe damage in the Kumamoto and Oita Prefectures. Numerous buildings collapsed because of the strong and continuous shaking. In this study, a deep learning model called Mask R-CNN was modified to extract residential buildings and estimate their damage levels from post-event aerial images. Our Mask R-CNN model employs an improved feature pyramid network and online hard example mining. Furthermore, a non-maximum suppression algorithm across multiple classes was also applied to improve prediction. The aerial images captured on 29 April 2016 (two weeks after the main shock) in Mashiki Town, Kumamoto Prefecture, were used as the training and test sets. Compared with the field survey results, our model achieved approximately 95% accuracy for building extraction and over 92% accuracy for the detection of severely damaged buildings. The overall classification accuracy for the four damage classes was approximately 88%, demonstrating acceptable performance.<\/jats:p>","DOI":"10.3390\/rs14041002","type":"journal-article","created":{"date-parts":[[2022,2,21]],"date-time":"2022-02-21T08:23:29Z","timestamp":1645431809000},"page":"1002","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Damaged Building Extraction Using Modified Mask R-CNN Model Using Post-Event Aerial Images of the 2016 Kumamoto Earthquake"],"prefix":"10.3390","volume":"14","author":[{"given":"Yihao","family":"Zhan","sequence":"first","affiliation":[{"name":"Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0655-4114","authenticated-orcid":false,"given":"Wen","family":"Liu","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan"},{"name":"Research Institute of Disaster Medicine, Chiba University, Chiba 263-8522, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8320-7207","authenticated-orcid":false,"given":"Yoshihisa","family":"Maruyama","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,18]]},"reference":[{"key":"ref_1","unstructured":"(2021, October 27). The Human Cost of Disasters: An Overview of the Last 20 Years (2000\u20132019). Available online: https:\/\/www.undrr.org\/publication\/human-cost-disasters-overview-last-20-years-2000-2019."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4870","DOI":"10.3390\/rs6064870","article-title":"Rapid damage assessment by means of multi-temporal SAR\u2014A comprehensive review and outlook to Sentinel-1","volume":"6","author":"Plank","year":"2014","journal-title":"Remote Sens."},{"key":"ref_3","first-page":"135","article-title":"Techniques of remote sensing and GIS for flood monitoring and damage assessment: A case study of Sindh Province, Pakistan","volume":"15","author":"Haq","year":"2012","journal-title":"Egypt. J. Remote Sens. Space Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.agrformet.2009.09.009","article-title":"Post-hurricane forest damage assessment using satellite remote sensing","volume":"150","author":"Wang","year":"2009","journal-title":"Agric. For. Meteorol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"906","DOI":"10.1080\/19475705.2020.1760360","article-title":"Rapid urban flood damage assessment using high resolution remote sensing data and an object-based approach","volume":"11","author":"Bustamante","year":"2020","journal-title":"Geomat. Nat. Hazards Risk"},{"key":"ref_6","unstructured":"Matsuoka, M., and Yamazaki, F. (February, January 30). Interferometric Characterization of Areas Damaged by the 1995 Kobe Earthquake Using Satellite SAR Images. Proceedings of the 12th World Conference on Earthquake Engineering, Auckland, New Zealand."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"S319","DOI":"10.1193\/1.2101127","article-title":"Detection and animation of damage using very high-resolution satellite data following the 2003 Bam, Iran earthquake","volume":"21","author":"Vu","year":"2005","journal-title":"Earthq. Spectra"},{"key":"ref_8","unstructured":"Matsuoka, M., and Yamazaki, F. (2000, January 12\u201315). Use of interferometric satellite SAR for earthquake damage detection. Proceedings of the 6th International Conference on Seismic Zonation, Palm Springs, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.14358\/PERS.77.10.1011","article-title":"Building extraction and rubble mapping for city Port-au-Prince post-2010 earthquake with GeoEye-1 imagery and lidar data","volume":"77","author":"Hussain","year":"2011","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5024930","DOI":"10.1155\/2018\/5024930","article-title":"Development of heavy rain damage prediction model using machine learning based on big data","volume":"2018","author":"Choi","year":"2018","journal-title":"Adv. Meteorol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kim, H.I., and Han, K.Y. (2020). Linking hydraulic modeling with a machine learning approach for extreme flood prediction and response. Atmosphere, 11.","DOI":"10.3390\/atmos11090987"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Harirchian, E., Kumari, V., Jadhav, K., Rasulzade, S., Lahmer, T., and Raj Das, R. (2021). A synthesized study based on machine learning approaches for rapid classifying earthquake damage grades to RC buildings. Appl. Sci., 11.","DOI":"10.3390\/app11167540"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Castorrini, A., Venturini, P., Gerboni, F., Corsini, A., and Rispoli, F. (2021, January 7\u201311). Machine learning aided prediction of rain erosion damage on wind turbine blade sections. Proceedings of the ASME Turbo Expo 2021: Turbomachinery Technical Conference and Exposition, Rotterdam, The Netherlands.","DOI":"10.1115\/GT2021-59156"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1016\/j.eswa.2016.03.024","article-title":"Building detection from orthophotos using a machine learning approach: An empirical study on image segmentation and descriptors","volume":"58","author":"Dornaika","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1177\/8755293019901309","article-title":"Building-damage detection method based on machine learning utilizing aerial photographs of the Kumamoto Earthquake","volume":"36","author":"Naito","year":"2020","journal-title":"Earthq. Spectra"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3614","DOI":"10.1080\/01431161.2019.1706781","article-title":"Learning a robust CNN-based rotation insensitive model for ship detection in VHR remote sensing images","volume":"41","author":"Dong","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.3130\/aijs.83.1391","article-title":"Recognition of damaged building using deep learning based on aerial and local photos taken after the 1995 Kobe Earthquake","volume":"83","author":"Ishii","year":"2018","journal-title":"J. Struct. Constr. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"164507","DOI":"10.1109\/ACCESS.2019.2952946","article-title":"Classification for remote sensing data with improved CNN-SVM method","volume":"7","author":"Sun","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6195","DOI":"10.1109\/TGRS.2019.2904868","article-title":"CDnet: CNN-based cloud detection for remote sensing imagery","volume":"57","author":"Yang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, C., Wei, S., Ji, S., and Lu, M. (2019). Detecting large-scale urban land cover changes from very high resolution remote sensing images using CNN-based classification. ISPRS Int. J. Geo.-Inf., 8.","DOI":"10.3390\/ijgi8040189"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1080\/2150704X.2018.1528398","article-title":"Learning a two-stage CNN model for multi-sized building detection in remote sensing images","volume":"10","author":"Chaoyue","year":"2019","journal-title":"Remote Sens. Lett."},{"key":"ref_22","unstructured":"Krizhevsky, A., and Hinton, G. (2012, January 3\u20136). Image net classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 8\u201316). Identity mappings in deep residual networks. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","unstructured":"Jiang, H., and Learned-Miller, E. (June, January 30). Face detection with the Faster R-CNN. Proceedings of the 12th IEEE International Conference on Automatic Face & Gesture Recognition, Washington, DC, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1007\/978-3-030-22871-2_69","article-title":"Detection of distal radius fractures trained by a small set of X-Ray images and Faster R-CNN","volume":"997","author":"Yahalomi","year":"2019","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1007\/978-3-319-95168-3_22","article-title":"Building extraction in high spatial resolution images using deep learning techniques","volume":"10962","author":"Shetty","year":"2018","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3799","DOI":"10.1109\/JSTARS.2021.3064981","article-title":"A new Mask R-CNN-based method for improved landslide detection","volume":"14","author":"Ullo","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Stiller, D., Stark, T., Wurm, M., Dech, S., and Taubenb\u00f6ck, H. (2019, January 22\u201324). Large-scale building extraction in very high-resolution aerial imagery using Mask R-CNN. Proceedings of the 2019 Joint Urban Remote Sensing Event (JURSE), Vannes, France.","DOI":"10.1109\/JURSE.2019.8808977"},{"key":"ref_36","unstructured":"(2021, October 28). Kyushu District Administrative Evaluation Bureau, Ministry of Internal Affairs and Communications: Survey on the Issuance of Disaster Victim Certificates during Large-Scale Disasters\u2014Focusing on the 2016 Kumamoto Earthquake. (In Japanese)."},{"key":"ref_37","unstructured":"(2021, October 27). Geological Survey of Japan (GSJ), National Institute of Advanced Industrial Science and Technology (AIST), 2016. Quick Estimation System for Earthquake Map Triggered by Observed Records (QuiQuake). Available online: https:\/\/gbank.gsj.jp\/QuiQuake\/QuakeMap\/."},{"key":"ref_38","unstructured":"Cabinet Office of Japan (2021, October 27). Summary of Damage Situation in the Kumamoto Earthquake Sequence, Available online: http:\/\/www.bousai.go.jp\/updates\/h280414jishin\/index.html."},{"key":"ref_39","unstructured":"(2021, October 27). Record of the Mashiki Town Earthquake of 2008: Summary Version. Available online: https:\/\/www.town.mashiki.lg.jp\/kiji0033823\/3_3823_5428_up_pi3lhhyu.pdf."},{"key":"ref_40","unstructured":"Microsoft Photogrammetry (2021, October 28). UltraCam-X Technical Specifications. Available online: https:\/\/www.sfsaviation.ch\/files\/177\/SFS%20UCX.pdf."},{"key":"ref_41","unstructured":"Wada, K. (2021, October 28). Labelme: Image Polygonal Annotation with Python. Available online: https:\/\/github.com\/wkentaro\/labelme."},{"key":"ref_42","unstructured":"Ministry of Land, Infrastructure, Transport and Tourism (MLIT) (2021, October 28). Report of the Committee to Analyze the Causes of Building Damage in the Kumamoto Earthquake, Available online: https:\/\/www.mlit.go.jp\/common\/001147923.pdf."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"65","DOI":"10.3130\/aijs.64.65_5","article-title":"Classifications of structural types and damage patterns of buildings for earthquake field investigation","volume":"64","author":"Okada","year":"1999","journal-title":"J. Struct. Constr. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"67","DOI":"10.3130\/aijs.66.67_4","article-title":"Classifications of damage patterns of reinforced concrete buildings for earthquake field investigation","volume":"66","author":"Takai","year":"2001","journal-title":"J. Struct. Constr. Eng."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18\u201322). Path aggregation network for instance segmentation. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00913"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shrivastava, A., Gupta, A., and Girshick, R. (2016, January 27\u201330). Training region-based object detectors with online hard example mining. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.89"},{"key":"ref_47","unstructured":"Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., and Xu, J. (2019). MMDetection: Open mmlab detection toolbox and benchmark. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Jadon, S. (2020). A survey of loss functions for semantic segmentation. arXiv.","DOI":"10.1109\/CIBCB48159.2020.9277638"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Saito, T., and Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0118432"},{"key":"ref_50","unstructured":"Powers, D. (2015). What the F-measure doesn\u2019t measure: Features, flaws, fallacies and fixes. arXiv."},{"key":"ref_51","first-page":"547","article-title":"Etude de la distribution florale dans une portion des Alpes et du Jura","volume":"37","author":"Paul","year":"1901","journal-title":"Bull. Soc. Vaud. Sci. Nat."},{"key":"ref_52","unstructured":"(2021, November 01). mAP (mean Average Precision) for Object Detection. Available online: https:\/\/jonathan-hui.medium.com\/map-mean-average-precision-for-object-detection-45c121a31173."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_54","unstructured":"Statistics Bureau of Japan, Ministry of Internal Affairs and Communications (2022, February 01). Standard Grid Square and Grid Square Code Used for the Statistics, Available online: https:\/\/www.stat.go.jp\/english\/data\/mesh\/02.html."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Hosang, J., Benenson, R., and Schiele, B. (2017). Learning non-maximum suppression. arXiv.","DOI":"10.1109\/CVPR.2017.685"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"241","DOI":"10.20965\/jdr.2017.p0241","article-title":"Extraction of collapsed buildings in the 2016 Kumamoto earthquake using multi-temporal PALSAR-2 data","volume":"12","author":"Liu","year":"2017","journal-title":"J. Disaster Res."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/4\/1002\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:22:27Z","timestamp":1760134947000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/4\/1002"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,18]]},"references-count":56,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["rs14041002"],"URL":"https:\/\/doi.org\/10.3390\/rs14041002","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,18]]}}}