{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T11:42:08Z","timestamp":1782301328312,"version":"3.54.5"},"reference-count":38,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T00:00:00Z","timestamp":1675987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003562","name":"Korea Ministry of Environment","doi-asserted-by":"publisher","award":["2021002280004"],"award-info":[{"award-number":["2021002280004"]}],"id":[{"id":"10.13039\/501100003562","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Animal sound classification (ASC) refers to the automatic identification of animal categories by sound, and is useful for monitoring rare or elusive wildlife. Thus far, deep-learning-based models have shown good performance in ASC when training data is sufficient, but suffer from severe performance degradation if not. Recently, generative adversarial networks (GANs) have shown the potential to solve this problem by generating virtual data. However, in a multi-class environment, existing GAN-based methods need to construct separate generative models for each class. Additionally, they only consider the waveform or spectrogram of sound, resulting in poor quality of the generated sound. To overcome these shortcomings, we propose a two-step sound augmentation scheme using a class-conditional GAN. First, common features are learned from all classes of animal sounds, and multiple classes of animal sounds are generated based on the features that consider both waveforms and spectrograms using class-conditional GAN. Second, we select data from the generated data based on the confidence of the pretrained ASC model to improve classification performance. Through experiments, we show that the proposed method improves the accuracy of the basic ASC model by up to 18.3%, which corresponds to a performance improvement of 13.4% compared to the second-best augmentation method.<\/jats:p>","DOI":"10.3390\/s23042024","type":"journal-article","created":{"date-parts":[[2023,2,13]],"date-time":"2023-02-13T02:14:11Z","timestamp":1676254451000},"page":"2024","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["DualDiscWaveGAN-Based Data Augmentation Scheme for Animal Sound Classification"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5535-0655","authenticated-orcid":false,"given":"Eunbeen","family":"Kim","sequence":"first","affiliation":[{"name":"School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9835-360X","authenticated-orcid":false,"given":"Jaeuk","family":"Moon","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9738-1038","authenticated-orcid":false,"given":"Jonghwa","family":"Shim","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0418-4092","authenticated-orcid":false,"given":"Eenjun","family":"Hwang","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.apacoust.2014.01.001","article-title":"Automatic bird sound detection in long real-field recordings: Applications and tools","volume":"80","author":"Potamitis","year":"2014","journal-title":"Appl. Acoust."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.ecoinf.2019.05.007","article-title":"Handcrafted features and late fusion with deep learning for bird sound classification","volume":"52","author":"Xie","year":"2019","journal-title":"Ecol. Inform."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, F., Zhang, L., Chen, H., and Xie, J. (2021). Bird Species Identification Using Spectrogram Based on Multi-Channel Fusion of DCNNs. Entropy, 23.","DOI":"10.3390\/e23111507"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"205327","DOI":"10.1109\/ACCESS.2020.3037063","article-title":"Conditional tabular GAN-based two-stage data generation scheme for short-term load forecasting","volume":"8","author":"Moon","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"101084","DOI":"10.1016\/j.ecoinf.2020.101084","article-title":"Data augmentation approaches for improving animal audio classification","volume":"57","author":"Nanni","year":"2020","journal-title":"Ecol. Inform."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Commun. ACM"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., and Xu, H. (2021, January 19\u201326). Time series data augmentation for deep learning: A survey. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, QC, Canada.","DOI":"10.24963\/ijcai.2021\/631"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1577778","DOI":"10.1155\/2022\/1577778","article-title":"Deep Learning-Based Data Augmentation and Model Fusion for Automatic Arrhythmia Identification and Classification Algorithms","volume":"2022","author":"Ma","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1109\/LSP.2022.3150258","article-title":"CNN-RNN and Data Augmentation Using Deep Convolutional Generative Adversarial Network for Environmental Sound Classification","volume":"29","author":"Bahmei","year":"2022","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s13636-020-00175-3","article-title":"Ensemble of convolutional neural networks to improve animal audio classification","volume":"2020","author":"Nanni","year":"2020","journal-title":"EURASIP J. Audio Speech Music Process."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, Y., Zhu, L., Yan, Y., and Yang, Y. (2019\u20131, January 29). Dual attention matching for audio-visual event localization. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00639"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"101331","DOI":"10.1016\/j.ecoinf.2021.101331","article-title":"On loss functions and CNNs for improved bioacoustic signal classification","volume":"64","author":"Xie","year":"2021","journal-title":"Ecol. Inform."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Abayomi-Alli, O.O., Dama\u0161evi\u010dius, R., Qazi, A., Adedoyin-Olowe, M., and Misra, S. (2022). Data Augmentation and Deep Learning Methods in Sound Classification: A Systematic Review. Electronics, 11.","DOI":"10.3390\/electronics11223795"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/LSP.2017.2657381","article-title":"Deep convolutional neural networks and data augmentation for environmental sound classification","volume":"24","author":"Salamon","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"107389","DOI":"10.1016\/j.apacoust.2020.107389","article-title":"Environmental sound classification using a regularized deep convolutional neural network with data augmentation","volume":"167","author":"Mushtaq","year":"2020","journal-title":"Appl. Acoust."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Park, D.S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019, January 15\u201319). Specaugment: A simple data augmentation method for automatic speech recognition. Proceedings of the Interspeech, Graz, Austria.","DOI":"10.21437\/Interspeech.2019-2680"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"105912","DOI":"10.1016\/j.asoc.2019.105912","article-title":"Unsupervised feature learning for environmental sound classification using weighted cycle-consistent generative adversarial network","volume":"86","author":"Esmaeilpour","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6301","DOI":"10.1007\/s10462-022-10153-0","article-title":"EnvGAN: A GAN-based augmentation to improve environmental sound classification","volume":"55","author":"Madhu","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_20","unstructured":"Donahue, C., McAuley, J., and Puckette, M. (2018). Adversarial audio synthesis. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"104930","DOI":"10.1016\/j.compbiomed.2021.104930","article-title":"Conditional GAN based augmentation for predictive modeling of respiratory signals","volume":"138","author":"Jayalakshmy","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_22","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Seibold, M., Hoch, A., Farshad, M., Navab, N., and F\u00fcrnstahl, P. (2022). Conditional Generative Data Augmentation for Clinical Audio Datasets. arXiv.","DOI":"10.1007\/978-3-031-16449-1_33"},{"key":"ref_24","first-page":"5769","article-title":"Improved training of Wasserstein GANs","volume":"30","author":"Gulrajani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.compind.2019.01.001","article-title":"Generative adversarial networks for data augmentation in machine fault diagnosis","volume":"106","author":"Shao","year":"2019","journal-title":"Comput. Ind."},{"key":"ref_26","unstructured":"Odena, A., Olah, C., and Shlens, J. (2017, January 6\u201311). Conditional image synthesis with auxiliary classifier GANs. Proceedings of the 34th International Conference on Machine Learning, International Convention Centre, Sydney, Australia."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.ecoinf.2017.04.003","article-title":"Automated bird acoustic event detection and robust species classification","volume":"39","author":"Zhao","year":"2017","journal-title":"Ecol. Inform."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"146040","DOI":"10.1155\/2014\/146040","article-title":"Voice activity detection in noisy environments based on double-combined Fourier transform and line fitting","volume":"2014","author":"Park","year":"2014","journal-title":"Sci. World J."},{"key":"ref_29","unstructured":"Okamoto, Y., Imoto, K., Komatsu, T., Takamichi, S., Yagyu, T., Yamanishi, R., and Yamashita, Y. (2019). Overview of tasks and investigation of subjective evaluation methods in environmental sound synthesis and conversion. arXiv."},{"key":"ref_30","first-page":"6629","article-title":"GANs trained by a two time-scale update rule converge to a local Nash equilibrium","volume":"30","author":"Heusel","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_31","unstructured":"Engel, J., Agrawal, K.K., Chen, S., Gulrajani, I., Donahue, C., and Roberts, A. (2019). Gansynth: Adversarial neural audio synthesis. arXiv."},{"key":"ref_32","unstructured":"Miyato, T., and Koyama, M. (2018). cGANs with projection discriminator. arXiv."},{"key":"ref_33","first-page":"5852","article-title":"On GANs and GMMs","volume":"31","author":"Richardson","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, X., Iqbal, T., Zhao, J., Huang, Q., Plumbley, M.D., and Wang, W. (2021, January 25\u201328). Conditional sound generation using neural discrete time-frequency representation learning. Proceedings of the IEEE 31st International Workshop on Machine Learning for Signal Processing, Gold Coast, Australia.","DOI":"10.1109\/MLSP52302.2021.9596430"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.aci.2018.08.003","article-title":"Classification assessment methods","volume":"17","author":"Tharwat","year":"2020","journal-title":"Appl. Comput. Inform."},{"key":"ref_37","unstructured":"Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2022, November 11). Automatic Differentiation in PyTorch. NIPS Autodiff Workshop. Available online: https:\/\/openreview.net\/pdf?id=BJJsrmfCZ."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.cviu.2018.10.009","article-title":"Pros and cons of gan evaluation measures","volume":"179","author":"Borji","year":"2019","journal-title":"Comput. Vis. Image Underst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2024\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:30:44Z","timestamp":1760121044000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2024"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,10]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23042024"],"URL":"https:\/\/doi.org\/10.3390\/s23042024","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,10]]}}}