{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T15:45:33Z","timestamp":1778168733339,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,18]],"date-time":"2022-08-18T00:00:00Z","timestamp":1660780800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"government of Ontario"},{"name":"Natural Sciences and Engineering Research Council (NSERC)\u2019s Discovery grant"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The deterioration of infrastructure\u2019s health has become more predominant on a global scale during the 21st century. Aging infrastructure as well as those structures damaged by natural disasters have prompted the research community to improve state-of-the-art methodologies for conducting Structural Health Monitoring (SHM). The necessity for efficient SHM arises from the hazards damaged infrastructure imposes, often resulting in structural collapse, leading to economic loss and human fatalities. Furthermore, day-to-day operations in these affected areas are limited until an inspection is performed to assess the level of damage experienced by the structure and the required rehabilitation determined. However, human-based inspections are often labor-intensive, inefficient, subjective, and restricted to accessible site locations, which ultimately negatively impact our ability to collect large amounts of data from inspection sites. Though Deep-Learning (DL) methods have been heavily explored in the past decade to rectify the limitations of traditional methods and automate structural inspection, data scarcity continues to remain prevalent within the field of SHM. The absence of sufficiently large, balanced, and generalized databases to train DL-based models often results in inaccurate and biased damage predictions. Recently, Generative Adversarial Networks (GANs) have received attention from the SHM community as a data augmentation tool by which a training dataset can be expanded to improve the damage classification. However, there are no existing studies within the SHM field which investigate the performance of DL-based multiclass damage identification using synthetic data generated from GANs. Therefore, this paper investigates the performance of a convolutional neural network architecture using synthetic images generated from a GAN for multiclass damage detection of concrete surfaces. Through this study, it was determined the average classification performance of the proposed CNN on hybrid datasets decreased by 10.6% and 7.4% for validation and testing datasets when compared to the same model trained entirely on real samples. Moreover, each model\u2019s performance decreased on average by 1.6% when comparing a singular model trained with real samples and the same model trained with both real and synthetic samples for a given training configuration. The correlation between classification accuracy and the amount and diversity of synthetic data used for data augmentation is quantified and the effect of using limited data to train existing GAN architectures is investigated. It was observed that the diversity of the samples decreases and correlation increases with the increase in the number of synthetic samples.<\/jats:p>","DOI":"10.3390\/s22166193","type":"journal-article","created":{"date-parts":[[2022,8,18]],"date-time":"2022-08-18T23:28:41Z","timestamp":1660865321000},"page":"6193","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Data Augmentation for Deep-Learning-Based Multiclass Structural Damage Detection Using Limited Information"],"prefix":"10.3390","volume":"22","author":[{"given":"Kyle","family":"Dunphy","sequence":"first","affiliation":[{"name":"Department of Civil and Environmental Engineering, Western University, London, ON N6A 3K7, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8079-7117","authenticated-orcid":false,"given":"Mohammad Navid","family":"Fekri","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Western University, London, ON N6A 3K7, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0062-8212","authenticated-orcid":false,"given":"Katarina","family":"Grolinger","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Western University, London, ON N6A 3K7, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5685-7087","authenticated-orcid":false,"given":"Ayan","family":"Sadhu","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, Western University, London, ON N6A 3K7, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,18]]},"reference":[{"key":"ref_1","first-page":"567","article-title":"A review of deep learning-based Structural Health Monitoring of civil infrastructure","volume":"24","author":"Ye","year":"2019","journal-title":"Smart Struct. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e2321","DOI":"10.1002\/stc.2321","article-title":"A literature review of next-generation smart sensing technology in Structural Health Monitoring","volume":"26","author":"Sony","year":"2019","journal-title":"Struct. Control Health Monit."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"103198","DOI":"10.1016\/j.autcon.2020.103198","article-title":"Image augmentation to improve construction resource detection using Generative Adversarial Networks, cut-and-past, and image transformation","volume":"115","author":"Bang","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1111\/mice.12561","article-title":"Generative Adversarial Network for road damage detection","volume":"36","author":"Maeda","year":"2020","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1111\/mice.12741","article-title":"Balanced semisupervised Generative Adversarial Network for damage assessment from low-data imbalanced-class regime","volume":"36","author":"Gao","year":"2021","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2069","DOI":"10.1177\/1475921720959226","article-title":"Lost data reconstruction for Structural Health Monitoring using deep convolutional Generative Adversarial Networks","volume":"20","author":"Lei","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"104139","DOI":"10.1016\/j.autcon.2022.104139","article-title":"Road damage detection using super-resolution and semi-supervised learning with Generative Adversarial Network","volume":"135","author":"Shim","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1007\/s12205-022-0518-2","article-title":"Crack Detection Based on Generative Adversarial Networks and Deep Learning","volume":"26","author":"Chen","year":"2022","journal-title":"KSCE J. Civ. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1111\/mice.12458","article-title":"Deep leaf-bootstrapping Generative Adversarial Network for structural image data augmentation","volume":"34","author":"Gao","year":"2019","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"04020073","DOI":"10.1061\/(ASCE)ST.1943-541X.0002535","article-title":"Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection","volume":"146","author":"Sun","year":"2020","journal-title":"J. Struct. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1177\/1475921720935585","article-title":"A review of computer vision-based Structural Health Monitoring at local and global levels","volume":"20","author":"Dong","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2621","DOI":"10.1007\/s11831-020-09471-9","article-title":"Machine Learning Algorithms in Civil Structural Health Monitoring: A Systematic Review","volume":"28","author":"Flah","year":"2021","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"111347","DOI":"10.1016\/j.engstruct.2020.111347","article-title":"A systematic review of convolutional neural network-based structural condition assessment techniques","volume":"226","author":"Sony","year":"2021","journal-title":"Eng. Struct."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"9289","DOI":"10.1007\/s00521-021-05690-8","article-title":"Surface crack detection using deep learning with shallow CNN architecture for enhanced computation","volume":"33","author":"Kim","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1111\/mice.12334","article-title":"Autonomous Structural Visual Inspection using Region-Based Deep Learning for Detecting Multiple Damage Types","volume":"33","author":"Cha","year":"2017","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Slonski, M., and Tekieli, M. (2020). 2D Digitial Image Correlation and Region-Based Covnolutional Neural Network in Monitoring and Evaluation of Surface Cracks in Concrete Structural Elements. Materials, 13.","DOI":"10.3390\/ma13163527"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103291","DOI":"10.1016\/j.autcon.2020.103291","article-title":"Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds suing deep learning","volume":"118","author":"Kang","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2199","DOI":"10.1177\/1475921720985437","article-title":"Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model","volume":"20","author":"Zhang","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chen, C., Seo, H., and Zhao, Y. (2021). A novel pavement transverse cracks detection model using WT-CNN and STFT-CNN for smartphone data analysis. Int. J. Pavement Eng., 1\u201313.","DOI":"10.1080\/10298436.2021.1945056"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3274","DOI":"10.1080\/10298436.2021.1888092","article-title":"Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM","volume":"23","author":"Chen","year":"2022","journal-title":"Int. J. Pavement Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ranyal, E., Sadhu, A., and Jain, K. (2022). Road Condition Monitoring Using Smart Sensing and Artificial Intelligence: A Review. Sensors, 22.","DOI":"10.3390\/s22083044"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"103225","DOI":"10.1016\/j.compind.2020.103225","article-title":"A Novel Approach to Data Augmentation for Pavement Distress Segmentation","volume":"121","author":"Mazzini","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"104376","DOI":"10.1016\/j.engappai.2021.104376","article-title":"Virtual generation of pavement crack images based on improved deep convolutional Generative Adversarial Network","volume":"104","author":"Pei","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Shin, H., Ahn, Y., Tae, S., Gil, H., Song, M., and Lee, S. (2021). Enhancement of Multi-class Structural Defect Recognition Using Generative Adversarial Network. Sustainability, 13.","DOI":"10.3390\/su132212682"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"59498","DOI":"10.1109\/ACCESS.2021.3073915","article-title":"Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier","volume":"9","author":"Situ","year":"2021","journal-title":"IEEE Access"},{"key":"ref_26","first-page":"1167","article-title":"A general framework for supervised Structural Health Monitoring and sensor output validation mitigating data imbalance with Generative Adversarial Networks-generated high-dimensional features","volume":"21","author":"Sarlo","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2054","DOI":"10.1177\/1475921721996238","article-title":"Investigation on the data augmenation using machine learning algorithms in Structural Health Monitoring information","volume":"20","author":"Tan","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_28","first-page":"40","article-title":"Improved Image Based Super Resolution and Concrete Crack Prediction Using Pre-Trained Deep Learning Models","volume":"4","author":"Sathya","year":"2020","journal-title":"J. Soft Comput. Civ. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.autcon.2019.02.013","article-title":"A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks","volume":"102","author":"Dung","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kim, B., and Cho, S. (2018). Automated vision-based detection of cracks on concrete surface using a deep-learning technique. Senosrs, 18.","DOI":"10.3390\/s18103452"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1007\/s12559-021-09922-w","article-title":"A New GAN-Based Approach to Data Augmentation and Image Segmentation for Crack Detection in Thermal Imaging Tests","volume":"13","author":"Tian","year":"2021","journal-title":"Cogn. Comput."},{"key":"ref_32","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_33","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv."},{"key":"ref_34","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein Generative Adversarial Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_35","unstructured":"Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C. (2017, January 4\u20139). Improved Training of Wasserstein GANs. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., and Aila, T. (2019, January 15\u201320). A Style-Based Generator Architecture for Generative Adversarial Network. Proceeding of 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00453"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1177\/14759217211021942","article-title":"Continuous missing data imputation with incomplete dataset by Generative Adversarial Networks-based unsupervised learning for long-term bridge health monitoring","volume":"21","author":"Jiang","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1609","DOI":"10.1177\/1475921720924601","article-title":"Toward data anomaly detection for automated Structural Health Monitoring: Exploiting generative adversarial nets and autoencoders","volume":"20","author":"Mao","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_39","first-page":"1","article-title":"Generalizating from a Few Examples: A Survey on Few-shot Learning","volume":"53","author":"Wang","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kim, I., Jeon, H., Baek, S., Hong, W., and Jung, H. (2018). Application of Crack Identification Techniques for an Aging Concrete Bridge inspection Using an Unmanned Aerial Vehicle. Sensors, 18.","DOI":"10.3390\/s18061881"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Bhrowmick, S., Nagarajaiah, S., and Veeraraghavan, A. (2020). Vision and Deep Learning-based Algorithms to Detect and Quantify Cracks on Concrete Surfaces from UAV Videos. Sensors, 20.","DOI":"10.3390\/s20216299"},{"key":"ref_43","unstructured":"Liu, B., Zhu, Y., Song, K., and Elgammal, A. (2021, January 3\u20137). Towards Faster and Stabalized GAN Training for High-Fidelity Few-Shot Image Synthesis. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s43065-022-00051-8","article-title":"Acoustic emission-based damage localization using wavelet-assisted deep learning","volume":"3","author":"Barbosh","year":"2022","journal-title":"J. Infrastruct. Preserv. Resil."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1664","DOI":"10.1016\/j.dib.2018.11.015","article-title":"SDNET2018: An annotated image dataset for non-contact concrete crack detection using deep convolutional neural network","volume":"21","author":"Dorafshan","year":"2018","journal-title":"Data Brief"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6193\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:11:41Z","timestamp":1760141501000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,18]]},"references-count":45,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22166193"],"URL":"https:\/\/doi.org\/10.3390\/s22166193","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,18]]}}}