{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:12:32Z","timestamp":1750219952215,"version":"3.41.0"},"reference-count":22,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2023,9,22]],"date-time":"2023-09-22T00:00:00Z","timestamp":1695340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2023,9,30]]},"abstract":"<jats:p>Vocoding is a sub-process of text-to-speech task, which aims at generating audios from intermediate representations between text and audio. Several recent works have shown that generative adversarial network\u2013 (GAN) based vocoders can generate audios with high quality. While GAN-based neural vocoders have shown higher efficiency in generating speed than autoregressive vocoders, the audio fidelity still cannot compete with ground-truth samples. One major cause of the degradation in audio quality and spectrogram vague comes from the average pooling layers in discriminator. As the multi-scale discriminator commonly used by recent GAN-based vocoders applies several average pooling layers to capture different-frequency bands, we believe it is crucial to prevent the high-frequency information from leakage in the average pooling process. This article proposes MSCGAN, which solves the above-mentioned problem and achieves higher-fidelity speech synthesis. We demonstrate that substituting the average pooling process with a multi-scale convolution architecture effectively retains high-frequency features and thus forces the generator to recover audio details in time and frequency domain. Compared with other state-of-the-art GAN-based vocoders, MSCGAN can produce competitive audio with a higher spectrogram clarity and mean opinion score score in subjective human evaluation.<\/jats:p>","DOI":"10.1145\/3610532","type":"journal-article","created":{"date-parts":[[2023,8,16]],"date-time":"2023-08-16T12:12:17Z","timestamp":1692187937000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving Generative Adversarial Network-based Vocoding through Multi-scale Convolution"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8101-5567","authenticated-orcid":false,"given":"Wanting","family":"Li","sequence":"first","affiliation":[{"name":"Harbin Institute of Technology (Shenzhen), Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6748-6832","authenticated-orcid":false,"given":"Yiting","family":"Chen","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology (Shenzhen), Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0271-8246","authenticated-orcid":false,"given":"Buzhou","family":"Tang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology (Shenzhen), Shenzhen, China and Pengcheng Laboratory, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,22]]},"reference":[{"key":"e_1_3_1_2_1","first-page":"2415","volume-title":"Proceedings of the 35th International Conference on Machine Learning (ICML\u201918)","author":"Kalchbrenner Nal","year":"2018","unstructured":"Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, A\u00e4ron van den Oord, Sander Dieleman, and Koray Kavukcuoglu. 2018. Efficient neural audio synthesis. In Proceedings of the 35th International Conference on Machine Learning (ICML\u201918). PMLR, 2415\u20132424."},{"key":"e_1_3_1_3_1","doi-asserted-by":"publisher","DOI":"10.1250\/ast.27.349"},{"key":"e_1_3_1_4_1","first-page":"2197","volume-title":"Proceedings of the 22nd Annual Conference of the International Speech Communication Association (INTERSPEECH\u201921)","author":"Kim Ji-Hoon","year":"2021","unstructured":"Ji-Hoon Kim, Sang-Hoon Lee, Ji-Hyun Lee, and Seong-Whan Lee. 2021. Fre-GAN: Adversarial frequency-consistent audio synthesis. In Proceedings of the 22nd Annual Conference of the International Speech Communication Association (INTERSPEECH\u201921). ISCA, 2197\u20132201."},{"key":"e_1_3_1_5_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Kong Jungil","year":"2020","unstructured":"Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)."},{"key":"e_1_3_1_6_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201919)","author":"Kumar Kundan","year":"2019","unstructured":"Kundan Kumar, Rithesh Kumar, Thibault de Boissi\u00e8re, Lucas Gestin, Wei Zhen Teoh, Jose M. R. Sotelo, Alexandre de Br\u00e9bisson, Yoshua Bengio, and Aaron C. Courville. 2019. MelGAN: Generative adversarial networks for conditional waveform synthesis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS\u201919). 14881\u201314892."},{"key":"e_1_3_1_7_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2021-2173"},{"key":"e_1_3_1_8_1","doi-asserted-by":"publisher","DOI":"10.1587\/transinf.2015EDP7457"},{"key":"e_1_3_1_9_1","first-page":"1","volume-title":"Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics","author":"Perraudin Nathana\u00ebl","year":"2013","unstructured":"Nathana\u00ebl Perraudin, Peter Balazs, and Peter L. S\u00f8ndergaard. 2013. A fast Griffin-Lim algorithm. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. IEEE, 1\u20134."},{"key":"e_1_3_1_10_1","first-page":"7706","volume-title":"Proceedings of the 37th International Conference on Machine Learning (ICML\u201920)","author":"Ping Wei","year":"2020","unstructured":"Wei Ping, Kainan Peng, Kexin Zhao, and Zhao Song. 2020. WaveFlow: A compact flow-based model for raw audio. In Proceedings of the 37th International Conference on Machine Learning (ICML\u201920). PMLR, 7706\u20137716."},{"key":"e_1_3_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8683143"},{"key":"e_1_3_1_12_1","article-title":"TFGAN: Time and frequency domain based generative adversarial network for high-fidelity speech synthesis","author":"Tian Qiao","year":"2020","unstructured":"Qiao Tian, Yi Chen, Zewang Zhang, Heng Lu, Ling-Hui Chen, Lei Xie, and Shan Liu. 2020. TFGAN: Time and frequency domain based generative adversarial network for high-fidelity speech synthesis (unpublished).","journal-title":"(unpublished)"},{"key":"e_1_3_1_13_1","doi-asserted-by":"crossref","unstructured":"Jean-Marc Valin and Jan Skoglund. 2019. LPCNET: Improving Neural Speech Synthesis through Linear Prediction. In IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP\u201919) Brighton United Kingdom May 12-17 2019 IEEE 5891\u20135895.","DOI":"10.1109\/ICASSP.2019.8682804"},{"key":"e_1_3_1_14_1","first-page":"125","volume-title":"Proceedings of the 9th ISCA Speech Synthesis Workshop","author":"Oord A\u00e4ron van den","year":"2016","unstructured":"A\u00e4ron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. 2016. WaveNet: A generative model for raw audio. In Proceedings of the 9th ISCA Speech Synthesis Workshop. ISCA, 125."},{"key":"e_1_3_1_15_1","doi-asserted-by":"publisher","DOI":"10.7488\/ds\/2645"},{"key":"e_1_3_1_16_1","doi-asserted-by":"crossref","unstructured":"Ravichander Vipperla Sangjun Park Kihyun Choo Samin Ishtiaq Kyoungbo Min Sourav Bhattacharya Abhinav Mehrotra Alberto Gil C. P. Ramos and Nicholas D. Lane. 2020. Bunched LPCNet: Vocoder for low-cost neural text-to-speech systems. In Interspeech 2020 21st Annual Conference of the International Speech Communication Association (Virtual Event Shanghai China 25-29 October 2020 ISCA) . 3565\u20133569.","DOI":"10.21437\/Interspeech.2020-2041"},{"key":"e_1_3_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2021.3051765"},{"key":"e_1_3_1_18_1","first-page":"6199","article-title":"Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram","author":"Yamamoto Ryuichi","year":"2020","unstructured":"Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. 2020. Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201920), 6199\u20136203.","journal-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201920)"},{"key":"e_1_3_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/SLT48900.2021.9383551"},{"key":"e_1_3_1_20_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2020-1238"},{"key":"e_1_3_1_21_1","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Multimedia and Expo (ICME\u201922)","author":"Zhao Xianbing","year":"2022","unstructured":"Xianbing Zhao, Yixin Chen, Yiting Chen, Sicen Liu, and Buzhou Tang. 2022a. HMAI-BERT: Hierarchical multimodal alignment and interaction network-enhanced BERT for multimodal sentiment analysis. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME\u201922). IEEE, 1\u20136."},{"key":"e_1_3_1_22_1","first-page":"4753","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201922)","author":"Zhao Xianbing","year":"2022","unstructured":"Xianbing Zhao, Yixin Chen, Wanting Li, Lei Gao, and Buzhou Tang. 2022b. MAG+: An extended multimodal adaptation gate for multimodal sentiment analysis. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201922). IEEE, 4753\u20134757."},{"key":"e_1_3_1_23_1","first-page":"96","volume-title":"Proceedings of the 22nd Annual Conference of the International Speech Communication Association (INTERSPEECH\u201921)","author":"Zhu Ge","year":"2021","unstructured":"Ge Zhu, Fei Jiang, and Zhiyao Duan. 2021. Y-Vector: Multiscale waveform encoder for speaker embedding. In Proceedings of the 22nd Annual Conference of the International Speech Communication Association (INTERSPEECH\u201921). ISCA, 96\u2013100."}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610532","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610532","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:03Z","timestamp":1750182543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610532"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,22]]},"references-count":22,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,9,30]]}},"alternative-id":["10.1145\/3610532"],"URL":"https:\/\/doi.org\/10.1145\/3610532","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2023,9,22]]},"assertion":[{"value":"2022-08-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-13","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}