{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T03:04:26Z","timestamp":1760151866325,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,11,28]],"date-time":"2022-11-28T00:00:00Z","timestamp":1669593600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["11874219","12274221"],"award-info":[{"award-number":["11874219","12274221"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The multichannel variational autoencoder (MVAE) integrates the rule-based update of a separation matrix and the deep generative model and proves to be a competitive speech separation method. However, the output (global) permutation ambiguity still exists and turns out to be a fundamental problem in applications. In this paper, we address this problem by employing two dedicated encoders. One encodes the speaker identity for the guidance of the output sorting, and the other encodes the linguistic information for the reconstruction of the source signals. The instance normalization (IN) and the adaptive instance normalization (adaIN) are applied to the networks to disentangle the speaker representations from the content representations. The separated sources are arranged in designated order by a symmetric permutation alignment scheme. In the experiments, we test the proposed method in different gender combinations and various reverberant conditions and generalize it to unseen speakers. The results validate its reliable sorting accuracy and good separation performance. The proposed method outperforms the other baseline methods and maintains stable performance, achieving over 20 dB SIR improvement even in high reverberant environments.<\/jats:p>","DOI":"10.3390\/sym14122514","type":"journal-article","created":{"date-parts":[[2022,11,28]],"date-time":"2022-11-28T08:13:09Z","timestamp":1669623189000},"page":"2514","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multichannel Variational Autoencoder-Based Speech Separation in Designated Speaker Order"],"prefix":"10.3390","volume":"14","author":[{"given":"Lele","family":"Liao","sequence":"first","affiliation":[{"name":"Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guoliang","family":"Cheng","sequence":"additional","affiliation":[{"name":"Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haoxin","family":"Ruan","sequence":"additional","affiliation":[{"name":"Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Chen","sequence":"additional","affiliation":[{"name":"Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9683-3768","authenticated-orcid":false,"given":"Jing","family":"Lu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, Nanjing 210093, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Makino, S., Lee, T.-W., and Sawada, H. (2007). Blind Speech Separation, Springer.","DOI":"10.1007\/978-1-4020-6479-1"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1162\/089976600300015312","article-title":"Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces","volume":"12","author":"Hoyer","year":"2000","journal-title":"Neural Comput."},{"key":"ref_3","unstructured":"Lee, I., Hao, J., and Lee, T.-W. (April, January 31). Adaptive Independent Vector Analysis for the Separation of Convoluted Mixtures Using EM Algorithm. Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1162\/neco.2010.11-08-906","article-title":"Independent Vector Analysis for Source Separation Using a Mixture of Gaussians Prior","volume":"22","author":"Hao","year":"2010","journal-title":"Neural Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gu, Z., Lu, J., and Chen, K. (2019, January 15\u201319). Speech Separation Using Independent Vector Analysis with an Amplitude Variable Gaussian Mixture Model. Proceedings of the INTERSPEECH 2019, Graz, Austria.","DOI":"10.21437\/Interspeech.2019-2076"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1672","DOI":"10.1109\/TSP.2011.2181836","article-title":"Joint Blind Source Separation with Multivariate Gaussian Model: Algorithms and Performance Analysis","volume":"60","author":"Anderson","year":"2012","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_7","unstructured":"Naik, G.R., and Wang, W. (2014). Frequency Domain Blind Source Separation Based on Independent Vector Analysis with a Multivariate Generalized Gaussian Source Prior. Blind Source Separation: Advances in Theory, Algorithms and Applications, Springer. Signals and Communication Technology."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"132871","DOI":"10.1109\/ACCESS.2020.3010342","article-title":"Hybrid Source Prior Based Independent Vector Analysis for Blind Separation of Speech Signals","volume":"8","author":"Khan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1109\/TASLP.2016.2577880","article-title":"Determined Blind Source Separation Unifying Independent Vector Analysis and Nonnegative Matrix Factorization","volume":"24","author":"Kitamura","year":"2016","journal-title":"IEEE ACM Trans. Audio Speech Lang. Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e12","DOI":"10.1017\/ATSIP.2019.5","article-title":"A Review of Blind Source Separation Methods: Two Converging Routes to ILRMA Originating from ICA and NMF","volume":"8","author":"Sawada","year":"2019","journal-title":"APSIPA Trans. Signal Inf. Process."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ono, N. (2011, January 16\u201319). Stable and Fast Update Rules for Independent Vector Analysis Based on Auxiliary Function Technique. Proceedings of the 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.","DOI":"10.1109\/ASPAA.2011.6082320"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1038\/44565","article-title":"Learning the Parts of Objects by Non-Negative Matrix Factorization","volume":"401","author":"Lee","year":"1999","journal-title":"Nature"},{"key":"ref_13","unstructured":"Lee, D., and Seung, H.S. (2001). Algorithms for Non-Negative Matrix Factorization. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Xie, Y., Xie, K., Yang, J., and Xie, S. (2018). Underdetermined Blind Source Separation Combining Tensor Decomposition and Nonnegative Matrix Factorization. Symmetry, 10.","DOI":"10.3390\/sym10100521"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1891","DOI":"10.1162\/neco_a_01217","article-title":"Supervised Determined Source Separation with Multichannel Variational Autoencoder","volume":"31","author":"Kameoka","year":"2019","journal-title":"Neural Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mogami, S., Sumino, H., Kitamura, D., Takamune, N., Takamichi, S., Saruwatari, H., and Ono, N. (2018, January 3\u20137). Independent Deeply Learned Matrix Analysis for Multichannel Audio Source Separation. Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy.","DOI":"10.23919\/EUSIPCO.2018.8553246"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1601","DOI":"10.1109\/TASLP.2019.2925450","article-title":"Independent Deeply Learned Matrix Analysis for Determined Audio Source Separation","volume":"27","author":"Makishima","year":"2019","journal-title":"IEEE ACM Trans. Audio Speech Lang. Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1702","DOI":"10.1109\/TASLP.2018.2842159","article-title":"Supervised Speech Separation Based on Deep Learning: An Overview","volume":"26","author":"Wang","year":"2018","journal-title":"IEEE ACM Trans Audio Speech Lang Process."},{"key":"ref_19","unstructured":"Doersch, C. (2021). Tutorial on Variational Autoencoders. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hyv\u00e4rinen, A., Karhunen, J., and Oja, E. (2001). Independent Component Analysis, Wiley.","DOI":"10.1002\/0471221317"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"168104","DOI":"10.1109\/ACCESS.2019.2954120","article-title":"Underdetermined Source Separation Based on Generalized Multichannel Variational Autoencoder","volume":"7","author":"Seki","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, L., Kameoka, H., and Makino, S. (2019, January 12\u201317). Fast MVAE: Joint Separation and Classification of Mixed Sources Based on Multichannel Variational Autoencoder with Auxiliary Classifier. Proceedings of the ICASSP 2019\u20132019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682623"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kameoka, H., Kaneko, T., Tanaka, K., and Hojo, N. (2020). ACVAE-VC: Non-Parallel Many-to-Many Voice Conversion with Auxiliary Classifier Variational Autoencoder. arXiv.","DOI":"10.1109\/TASLP.2019.2917232"},{"key":"ref_24","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2017). Instance Normalization: The Missing Ingredient for Fast Stylization. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, Q., Zhang, Y., Yin, S., Wang, Y., and Wu, G. (2021). A Novel Underdetermined Blind Source Separation Method Based on OPTICS and Subspace Projection. Symmetry, 13.","DOI":"10.3390\/sym13091677"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chou, J., Yeh, C., and Lee, H. (2019). One-Shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization. arXiv.","DOI":"10.21437\/Interspeech.2019-2663"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19\u201324). Librispeech: An ASR Corpus Based on Public Domain Audio Books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.","DOI":"10.1109\/ICASSP.2015.7178964"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hadad, E., Heese, F., Vary, P., and Gannot, S. (2014, January 8\u201311). Multichannel Audio Database in Various Acoustic Environments. Proceedings of the 2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC), Juan-les-Pins, France.","DOI":"10.1109\/IWAENC.2014.6954309"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1462","DOI":"10.1109\/TSA.2005.858005","article-title":"Performance Measurement in Blind Audio Source Separation","volume":"14","author":"Vincent","year":"2006","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_30","unstructured":"Rix, A.W., Beerends, J.G., Hollier, M.P., and Hekstra, A.P. (2001, January 7\u201311). Perceptual Evaluation of Speech Quality (PESQ)-a New Method for Speech Quality Assessment of Telephone Networks and Codecs. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), Salt Lake City, UT, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Taal, C.H., Hendriks, R.C., Heusdens, R., and Jensen, J. (2010, January 14\u201319). A Short-Time Objective Intelligibility Measure for Time-Frequency Weighted Noisy Speech. Proceedings of the 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, USA.","DOI":"10.1109\/ICASSP.2010.5495701"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., and Khudanpur, S. (2018, January 15\u201320). X-Vectors: Robust Dnn Embeddings for Speaker Recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8461375"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Prince, S.J., and Elder, J.H. (2007, January 26). Probabilistic Linear Discriminant Analysis for Inferences about Identity. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil.","DOI":"10.1109\/ICCV.2007.4409052"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Anjos, A., El-Shafey, L., Wallace, R., G\u00fcnther, M., McCool, C., and Marcel, S. (2012, January 29). Bob: A Free Signal Processing and Machine Learning Toolbox for Researchers. Proceedings of the 20th ACM International Conference on Multimedia, Nara, Japan.","DOI":"10.1145\/2393347.2396517"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1049\/el.2011.3988","article-title":"Overcoming Block Permutation Problem in Frequency Domain Blind Source Separation When Using AuxIVA Algorithm","volume":"48","author":"Liang","year":"2012","journal-title":"Electron. Lett."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/12\/2514\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:28:31Z","timestamp":1760146111000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/12\/2514"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,28]]},"references-count":35,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["sym14122514"],"URL":"https:\/\/doi.org\/10.3390\/sym14122514","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2022,11,28]]}}}