{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:53:47Z","timestamp":1792788827668,"version":"4.5.0"},"reference-count":34,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Industry-university Cooperation and Collaborative Education Project of the Ministry of Education","award":["231003221260825"],"award-info":[{"award-number":["231003221260825"]}]},{"name":"The 2024 Graduate Education Reform Project of Shanghai Dianji University","award":["A102252401102029"],"award-info":[{"award-number":["A102252401102029"]}],"id":[{"id":"https:\/\/ror.org\/055fene14","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"\n Deep learning-based zero-shot speech synthesis has achieved substantial progress in speaker generalization, but stable modeling remains challenging in fine-grained emotional scenarios. Existing systems often process textual and emotional conditions through shared or closely coupled pathways, which may introduce interference between semantic content and emotional expression. In addition, uniformly averaging per-sample Conditional Flow Matching (CFM) losses may provide insufficient optimization emphasis to high-loss emotional samples. This study proposes a CFM-based zero-shot emotional speech synthesis method. Emotion\u2013Text Decoupling Attention (ETDA) processes semantic and emotional conditions through parallel cross-attention streams and combines them through adaptive gated fusion, allowing the two conditions to retain their respective information before fusion. A 16-class fine-grained emotion space is constructed through classifier filtering and K-Means clustering based on pitch and energy features, and the resulting labels are mapped to continuous representations using a trainable lookup embedding. During training, Adaptive Loss-Threshold Reweighting estimates a threshold from mini-batch loss statistics and assigns larger weights to samples whose individual CFM losses exceed that threshold. Under speaker-disjoint evaluation on ESD, the proposed method achieves a word error rate of\n \n \n \n \n \n 3.62<\/mml:mn>\n \u00b1<\/mml:mo>\n 0.06<\/mml:mn>\n %<\/mml:mi>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n , a mel-cepstral distortion of\n \n \n \n \n \n 4.45<\/mml:mn>\n \u00b1<\/mml:mo>\n 0.05<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n dB, and an emotional expressiveness mean opinion score of\n \n \n \n \n \n 4.64<\/mml:mn>\n \u00b1<\/mml:mo>\n 0.04<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n . Controlled text-length expansion and analyses on a fixed subset of high-loss emotional samples further indicate that the proposed method maintains linguistic content and emotional expression more consistently under the evaluated conditions. These results suggest that the proposed method can, to some extent, improve content preservation, emotional expression, and acoustic reconstruction in fine-grained zero-shot emotional speech synthesis.\n <\/jats:p>","DOI":"10.1371\/journal.pone.0358777","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:30:44Z","timestamp":1790184644000},"page":"e0358777","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Zero-shot emotional speech synthesis based on feature decoupling and adaptive loss-threshold reweighting"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-0222-7717","authenticated-orcid":true,"given":"Yan","family":"Zhu","sequence":"first","affiliation":[{"name":"Shanghai Dianji University","place":["Shanghai, China"],"department":["School of Design and Art"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3939-9245","authenticated-orcid":true,"given":"Yu","family":"Wang","sequence":"additional","affiliation":[{"name":"Shanghai Dianji University","place":["Shanghai, China"],"department":["School of Design and Art"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yijin","family":"Zhou","sequence":"additional","affiliation":[{"name":"Shanghai Dianji University","place":["Shanghai, China"],"department":["School of Design and Art"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bomin","family":"Liu","sequence":"additional","affiliation":[{"name":"Shanghai Dianji University","place":["Shanghai, China"],"department":["School of Design and Art"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0528-883X","authenticated-orcid":true,"given":"Rui","family":"Zhou","sequence":"additional","affiliation":[{"name":"Nanjing University","place":["Nanjing, China"],"department":["School of Computer Science"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"pone.0358777.ref001","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1109\/TASLPRO.2025.3530270","article-title":"Neural codec language models are zero-shot text to speech synthesizers","volume":"33","author":"S Chen","year":"2025","journal-title":"IEEE Trans Audio, Speech Lang Process"},{"key":"pone.0358777.ref002","unstructured":"Shen K, Ju Z, Tan X, Liu Y, Leng Y, He L, et al. NaturalSpeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. In: International Conference on Learning Representations (ICLR); 2024. https:\/\/doi.org\/10.48550\/arXiv.2304.09116"},{"key":"pone.0358777.ref003","article-title":"CosyVoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens.","author":"Z Du","year":"2024","journal-title":"arXiv"},{"key":"pone.0358777.ref004","doi-asserted-by":"crossref","unstructured":"Hang T, Gu S, Li C, Bao J, Chen D, Hu H, et al. Efficient Diffusion Training via Min-SNR Weighting Strategy. In: 2023 IEEE\/CVF International Conference on Computer Vision (ICCV); 2023. p. 7407\u201317. http:\/\/dx.doi.org\/10.1109\/iccv51070.2023.00684","DOI":"10.1109\/ICCV51070.2023.00684"},{"key":"pone.0358777.ref005","unstructured":"MacQueen J. Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; 1967. p. 281\u201397."},{"key":"pone.0358777.ref006","doi-asserted-by":"crossref","unstructured":"\u0141a\u0144cucki A. FastPitch: Parallel text-to-speech with pitch prediction. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2021. p. 6588\u201392. https:\/\/doi.org\/10.1109\/ICASSP39728.2021.9413889","DOI":"10.1109\/ICASSP39728.2021.9413889"},{"key":"pone.0358777.ref007","unstructured":"Ren Y, Hu C, Tan X, Qin T, Zhao S, Zhao Z. FastSpeech 2: Fast and high-quality end-to-end text to speech. In: International Conference on Learning Representations (ICLR); 2021. https:\/\/doi.org\/10.48550\/arXiv.2006.04558"},{"key":"pone.0358777.ref008","doi-asserted-by":"crossref","unstructured":"Casanova E, Davis K, G\u00f6lge E, G\u00f6knar G, Gulea I, Hart L. XTTS: A massively multilingual zero-shot text-to-speech model. In: Proceedings of Interspeech; 2024. p. 201\u20135. https:\/\/doi.org\/10.21437\/interspeech.2024-2016","DOI":"10.21437\/Interspeech.2024-2016"},{"key":"pone.0358777.ref009","article-title":"OpenVoice: Versatile instant voice cloning","author":"Z Qin","year":"2023","journal-title":"arXiv"},{"key":"pone.0358777.ref010","unstructured":"Ju Z, Wang Y, Shen K, Tan X, Xin D, Yang D, et al. NaturalSpeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. In: Proceedings of the International Conference on Machine Learning (ICML); 2024. https:\/\/doi.org\/10.48550\/arXiv.2403.03100"},{"key":"pone.0358777.ref011","doi-asserted-by":"crossref","unstructured":"Guo Z, Leng Y, Wu Y, Zhao S, Tan X. Prompttts: Controllable Text-To-Speech With Text Descriptions. In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2023. p. 1\u20135. https:\/\/doi.org\/10.1109\/icassp49357.2023.10096285","DOI":"10.1109\/ICASSP49357.2023.10096285"},{"key":"pone.0358777.ref012","doi-asserted-by":"crossref","unstructured":"Guo Y, Du C, Chen X, Yu K. Emodiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance. In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2023. p. 1\u20135. https:\/\/doi.org\/10.1109\/icassp49357.2023.10095621","DOI":"10.1109\/ICASSP49357.2023.10095621"},{"key":"pone.0358777.ref013","unstructured":"Chen D, Zhang X, Wang Y, Dai K, Ma L, Wu Z. FlexiVoice: Enabling flexible style control in zero-shot TTS with natural language instructions. In: International Conference on Learning Representations (ICLR); 2026. https:\/\/doi.org\/10.48550\/arXiv.2601.04656"},{"key":"pone.0358777.ref014","doi-asserted-by":"crossref","unstructured":"Inoue S, Zhou K, Wang S, Li H. Fine-Grained Quantitative Emotion Editing for Speech Generation. In: 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC); 2024. p. 1\u20136. http:\/\/dx.doi.org\/10.1109\/apsipaasc63619.2025.10848721","DOI":"10.1109\/APSIPAASC63619.2025.10848721"},{"key":"pone.0358777.ref015","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"J Ho","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"pone.0358777.ref016","unstructured":"Popov V, Vovk I, Gogoryan V, Sadekova T, Kudinov M. Grad-TTS: A diffusion probabilistic model for text-to-speech. In: Proceedings of the International Conference on Machine Learning (ICML); 2021. https:\/\/doi.org\/10.48550\/arXiv.2105.06337"},{"key":"pone.0358777.ref017","unstructured":"Lipman Y, Chen RT, Ben-Hamu H, Nickel M, Le M. Flow matching for generative modeling. In: International Conference on Learning Representations (ICLR); 2023. https:\/\/doi.org\/10.48550\/arXiv.2210.02747"},{"key":"pone.0358777.ref018","doi-asserted-by":"crossref","unstructured":"Le M, Vyas A, Shi B, Karrer B, Sari L, Moritz R, et al. Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale. In: Advances in Neural Information Processing Systems 36; 2023. p. 14005\u201334. http:\/\/dx.doi.org\/10.52202\/075280-0618","DOI":"10.52202\/075280-0618"},{"key":"pone.0358777.ref019","doi-asserted-by":"crossref","unstructured":"Mehta S, Tu R, Beskow J, Sz\u00e9kely \u00c9, Henter GE. Matcha-TTS: A Fast TTS Architecture with Conditional Flow Matching. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2024. p. 11341\u20135. http:\/\/dx.doi.org\/10.1109\/icassp48485.2024.10448291","DOI":"10.1109\/ICASSP48485.2024.10448291"},{"key":"pone.0358777.ref020","doi-asserted-by":"crossref","unstructured":"Hieu NHN, Nguyen NS, Dang HN, Vo T, Hy T-S, Nguyen V. OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2025. p. 21500\u201317. https:\/\/doi.org\/10.18653\/v1\/2025.acl-long.1043","DOI":"10.18653\/v1\/2025.acl-long.1043"},{"key":"pone.0358777.ref021","doi-asserted-by":"crossref","first-page":"19594","DOI":"10.52202\/075280-0860","article-title":"StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models","volume":"36","author":"YA Li","year":"2023","journal-title":"Adv Neural Inf Process Syst"},{"key":"pone.0358777.ref022","first-page":"12449","article-title":"wav2vec 2.0: A framework for self-supervised learning of speech representations","volume":"33","author":"A Baevski","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"pone.0358777.ref023","first-page":"17022","article-title":"HiFi-GAN: Generative adversarial networks for efficient and high-fidelity speech synthesis","volume":"33","author":"J Kong","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pone.0358777.ref024","unstructured":"Ito K, Johnson L. The LJ Speech Dataset. 2017. Available from: https:\/\/keithito.com\/LJ-Speech-Dataset\/"},{"key":"pone.0358777.ref025","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.specom.2021.11.006","article-title":"Emotional voice conversion: Theory, databases and ESD","volume":"137","author":"K Zhou","year":"2022","journal-title":"Speech Communication"},{"key":"pone.0358777.ref026","doi-asserted-by":"crossref","unstructured":"McAuliffe M, Socolof M, Mihuc S, Wagner M, Sonderegger M. Montreal Forced Aligner: Trainable text-to-speech alignment using Kaldi. In: Proceedings of Interspeech; 2017. p. 498\u2013502. https:\/\/doi.org\/10.21437\/Interspeech.2017-1386","DOI":"10.21437\/Interspeech.2017-1386"},{"key":"pone.0358777.ref027","unstructured":"Loshchilov I, Hutter F. Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR); 2019. https:\/\/doi.org\/10.48550\/arXiv.1711.05101"},{"key":"pone.0358777.ref028","unstructured":"Loshchilov I, Hutter F. SGDR: Stochastic gradient descent with warm restarts. In: International Conference on Learning Representations (ICLR); 2017. https:\/\/doi.org\/10.48550\/arXiv.1608.03983"},{"key":"pone.0358777.ref029","unstructured":"Radford A, Kim JW, Xu T, Brockman G, McLeavey C, Sutskever I. Robust speech recognition via large-scale weak supervision. In: Proceedings of the International Conference on Machine Learning (ICML), 2023. https:\/\/doi.org\/10.48550\/arXiv.2212.04356"},{"issue":"6","key":"pone.0358777.ref030","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual Comparisons by Ranking Methods","volume":"1","author":"F Wilcoxon","year":"1945","journal-title":"Biometrics Bulletin"},{"issue":"2","key":"pone.0358777.ref031","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"S Holm","year":"1979","journal-title":"Scand J Stat"},{"issue":"2","key":"pone.0358777.ref032","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s00530-014-0446-1","article-title":"Mean opinion score (MOS) revisited: methods and applications, limitations and alternatives","volume":"22","author":"RC Streijl","year":"2014","journal-title":"Multimedia Systems"},{"key":"pone.0358777.ref033","unstructured":"Kim J, Kong J, Son J. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In: 2021. https:\/\/doi.org\/10.48550\/arXiv.2106.06103"},{"issue":"2","key":"pone.0358777.ref034","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0318297","article-title":"Speech emotion recognition using fine-tuned Wav2vec2.0 and neural controlled differential equations classifier","volume":"20","author":"N Wang","year":"2025","journal-title":"PLoS One"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0358777","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:30:55Z","timestamp":1790184655000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0358777"}},"subtitle":[],"editor":[{"given":"Anirban","family":"Bhowmick","sequence":"first","affiliation":[{"name":"VIT Bhopal University","place":["INDIA"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":34,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0358777","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]}}}