{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T21:45:17Z","timestamp":1771883117856,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T00:00:00Z","timestamp":1641945600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T00:00:00Z","timestamp":1641945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the National Key R and D Program of China","award":["2018YFB1305200"],"award-info":[{"award-number":["2018YFB1305200"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61771333"],"award-info":[{"award-number":["61771333"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Tianjin Municipal Science and Technology Project","award":["18ZXZNGX00330"],"award-info":[{"award-number":["18ZXZNGX00330"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J AUDIO SPEECH MUSIC PROC."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Conventional automatic speech recognition (ASR) and emerging end-to-end (E2E) speech recognition have achieved promising results after being provided with sufficient resources. However, for low-resource language, the current ASR is still challenging. The Lhasa dialect is the most widespread Tibetan dialect and has a wealth of speakers and transcriptions. Hence, it is meaningful to apply the ASR technique to the Lhasa dialect for historical heritage protection and cultural exchange. Previous work on Tibetan speech recognition focused on selecting phone-level acoustic modeling units and incorporating tonal information but underestimated the influence of limited data. The purpose of this paper is to improve the speech recognition performance of the low-resource Lhasa dialect by adopting multilingual speech recognition technology on the E2E structure based on the transfer learning framework. Using transfer learning, we first establish a monolingual E2E ASR system for the Lhasa dialect with different source languages to initialize the ASR model to compare the positive effects of source languages on the Tibetan ASR model. We further propose a multilingual E2E ASR system by utilizing initialization strategies with different source languages and multilevel units, which is proposed for the first time. Our experiments show that the performance of the proposed method-based ASR system exceeds that of the E2E baseline ASR system. Our proposed method effectively models the low-resource Lhasa dialect and achieves a relative 14.2% performance improvement in character error rate (CER) compared to DNN-HMM systems. Moreover, from the best monolingual E2E model to the best multilingual E2E model of the Lhasa dialect, the system\u2019s performance increased by 8.4% in CER.<\/jats:p>","DOI":"10.1186\/s13636-021-00233-4","type":"journal-article","created":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T17:02:53Z","timestamp":1642006973000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Improving low-resource Tibetan end-to-end ASR by multilingual and multilevel unit modeling"],"prefix":"10.1186","volume":"2022","author":[{"given":"Siqing","family":"Qin","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Longbiao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheng","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianwu","family":"Dang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lixin","family":"Pan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,1,12]]},"reference":[{"issue":"3","key":"233_CR1","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1080\/00401706.1991.10484833","volume":"33","author":"B. H. Juang","year":"1991","unstructured":"B. H. Juang, L. R. Rabiner, Hidden Markov models for speech recognition. Technometrics. 33(3), 251\u2013272 (1991).","journal-title":"Technometrics"},{"key":"233_CR2","volume-title":"Automatic speech recognition\u2013a brief history of the technology development, vol. 1","author":"B. -H. Juang","year":"2005","unstructured":"B. -H. Juang, L. R. Rabiner, Automatic speech recognition\u2013a brief history of the technology development, vol. 1 (Georgia Institute of Technology. Atlanta Rutgers University and the University of California, Santa Barbara, 2005)."},{"key":"233_CR3","doi-asserted-by":"crossref","unstructured":"L. Deng, G. Hinton, B. Kingsbury, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. New types of deep neural network learning for speech recognition and related applications: an overview (IEEE, 2013), pp. 8599\u20138603.","DOI":"10.1109\/ICASSP.2013.6639344"},{"issue":"1","key":"233_CR4","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/TASL.2011.2134090","volume":"20","author":"G. E. Dahl","year":"2011","unstructured":"G. E. Dahl, D. Yu, L. Deng, A. Acero, Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(1), 30\u201342 (2011).","journal-title":"IEEE Trans. Audio Speech Lang. Process"},{"issue":"6","key":"233_CR5","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","volume":"29","author":"G. Hinton","year":"2012","unstructured":"G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. -r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Proc. Mag.29(6), 82\u201397 (2012).","journal-title":"IEEE Signal Proc. Mag."},{"issue":"10","key":"233_CR6","doi-asserted-by":"publisher","first-page":"1533","DOI":"10.1109\/TASLP.2014.2339736","volume":"22","author":"O. Abdel-Hamid","year":"2014","unstructured":"O. Abdel-Hamid, A. -r. Mohamed, H. Jiang, L. Deng, G. Penn, D. Yu, Convolutional neural networks for speech recognition. IEEE\/ACM Trans. Audio Speech Lang. Process. 22(10), 1533\u20131545 (2014).","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process"},{"key":"233_CR7","doi-asserted-by":"crossref","unstructured":"H. Sak, A. Senior, F. Beaufays, Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128 (2014).","DOI":"10.21437\/Interspeech.2014-80"},{"key":"233_CR8","doi-asserted-by":"crossref","unstructured":"A. Graves, N. Jaitly, A. -r. Mohamed, in 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. Hybrid speech recognition with deep bidirectional LSTM (IEEE, 2013), pp. 273\u2013278.","DOI":"10.1109\/ASRU.2013.6707742"},{"key":"233_CR9","doi-asserted-by":"crossref","unstructured":"H. Wang, K. Khyuru, J Li, G. Li, J. Dang, L. Huang, in Proc. APSIPA ASC. Investigation on acoustic modeling with different phoneme set for continuous Lhasa Tibetan recognition based on DNN method, (2016).","DOI":"10.1109\/APSIPA.2016.7820795"},{"key":"233_CR10","doi-asserted-by":"crossref","unstructured":"J. Li, H. Wang, L. Wang, J. Dang, K. Khuru, G. Lobsang, in Proc. ISCSLP. Exploring tonal information for lhasa dialect acoustic modeling, (2016).","DOI":"10.1109\/ISCSLP.2016.7918447"},{"key":"233_CR11","doi-asserted-by":"crossref","unstructured":"D. Povey, V. Peddinti, D. Galvez, P. Ghahrmani, V. Manohar, X. Na, Y. Wang, S. Khudanpur, in Proc. INTERSPEECH. Purely sequence-trained neural networks for ASR based on lattice-free MMI, (2016).","DOI":"10.21437\/Interspeech.2016-595"},{"key":"233_CR12","doi-asserted-by":"crossref","unstructured":"J. Yan, Z. Lv, S. Huang, H. Yu, in Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under-Resourced Languages. Low-resource Tibetan dialect acoustic modeling based on transfer learning, (2018).","DOI":"10.21437\/SLTU.2018-2"},{"key":"233_CR13","unstructured":"J. Chorowski, D. Bahdanau, K. Cho, Y. Bengio, End-to-end continuous speech recognition using attention-based recurrent nn: first results. arXiv preprint arXiv:1412.1602 (2014)."},{"key":"233_CR14","doi-asserted-by":"crossref","unstructured":"W. Chan, N. Jaitly, Q. Le, O. Vinyals, in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Listen, attend and spell: a neural network for large vocabulary conversational speech recognition (IEEE, 2016), pp. 4960\u20134964.","DOI":"10.1109\/ICASSP.2016.7472621"},{"issue":"8","key":"233_CR15","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1109\/JSTSP.2017.2763455","volume":"11","author":"S. Watanabe","year":"2017","unstructured":"S. Watanabe, T. Hori, S. Kim, J. R. Hershey, T. Hayashi, Hybrid ctc\/attention architecture for end-to-end speech recognition. IEEE J. Sel. Top. Sig. Process. 11(8), 1240\u20131253 (2017).","journal-title":"IEEE J. Sel. Top. Sig. Process"},{"key":"233_CR16","doi-asserted-by":"crossref","unstructured":"A. Graves, A. -r. Mohamed, G. Hinton, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Speech recognition with deep recurrent neural networks (IEEE, 2013), pp. 6645\u20136649.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"233_CR17","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, I. Polosukhin, in Advances in Neural Information Processing Systems. Attention is all you need, (2017), pp. 5998\u20136008."},{"key":"233_CR18","unstructured":"A. Graves, N. Jaitly, in Proc. ICML. Towards end-to-end speech recognition with recurrent neural networks, (2014)."},{"key":"233_CR19","doi-asserted-by":"crossref","unstructured":"Y. Miao, M. Gowayyed, F. Metze, in Proc. IEEE-ASRU. EESEN: end-to-end speech recognition using deep RNN models and WFST-based decoding, (2015), pp. 167\u2013174.","DOI":"10.1109\/ASRU.2015.7404790"},{"key":"233_CR20","unstructured":"J. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, Y. Bengio, in Proc. NIPS. Attention-based models for speech recognition, (2015)."},{"key":"233_CR21","doi-asserted-by":"crossref","unstructured":"W. Chan, N. Jaitly, Q. Le, O. Vinyals, in Proc. IEEE-ICASSP. Listen, attend and spell: a neural network for large vocabulary conversational speech recognition, (2016).","DOI":"10.1109\/ICASSP.2016.7472621"},{"key":"233_CR22","doi-asserted-by":"crossref","unstructured":"H. Hadian, H. Sameti, D. Povey, S. Khudanpur, in Proc. INTERSPEECH. End-to-end speech recognition using lattice-free MMI, (2018).","DOI":"10.21437\/Interspeech.2018-1423"},{"issue":"8","key":"233_CR23","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1109\/JSTSP.2017.2763455","volume":"11","author":"S. Watanabe","year":"2017","unstructured":"S. Watanabe, T. Hori, S. Kim, J. R. Hershey, T. Hayashi, Hybrid CTC\/attention architecture for end-to-end speech recognition. IEEE J. Sel. Top. Sig. Process. 11(8), 1240\u20131253 (2017).","journal-title":"IEEE J. Sel. Top. Sig. Process"},{"key":"233_CR24","doi-asserted-by":"crossref","unstructured":"S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N. Enrique, Y. Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, T. Ochiai, in Proc. INTERSPEECH. Espnet: end-to-end speech processing toolkit, (2018).","DOI":"10.21437\/Interspeech.2018-1456"},{"key":"233_CR25","doi-asserted-by":"crossref","unstructured":"S. Ueno, H. Inaguma, M. Mimura, T. Kawahara, in Proc. IEEE-ICASSP. Acoustic-to-word attention-based model complemented with character-level CTC-based model, (2018), pp. 5804\u20135808.","DOI":"10.1109\/ICASSP.2018.8462576"},{"key":"233_CR26","doi-asserted-by":"crossref","unstructured":"T. Hori, S. Watanabe, Y. Zhang, W. Chan, in Proc. INTERSPEECH. Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM, (2017).","DOI":"10.21437\/Interspeech.2017-1296"},{"key":"233_CR27","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, in arXiv Preprint Arxiv:1706.03762. Attention is all you need, (2017)."},{"key":"233_CR28","doi-asserted-by":"crossref","unstructured":"L. Dong, S. Xu, B. Xu, in Proc. IEEE-ICASSP. Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition, (2018).","DOI":"10.1109\/ICASSP.2018.8462506"},{"key":"233_CR29","unstructured":"S. Zhou, S. Xu, B. Xu, in arXiv Preprint Arxiv:1806.05059. Multilingual end-to-end speech recognition with a single transformer on low-resource languages, (2018)."},{"key":"233_CR30","doi-asserted-by":"crossref","unstructured":"S. Zhou, L. Dong, S. Xu, B. Xu, in arXiv Preprint Arxiv:1805.06239. A comparison of modeling units in sequence-to-sequence speech recognition with the transformer on Mandarin Chinese, (2018).","DOI":"10.21437\/Interspeech.2018-1107"},{"key":"233_CR31","doi-asserted-by":"crossref","unstructured":"S. Zhou, L. Dong, S. Xu, B. Xu, in Proc. INTERSPEECH. Syllable-based sequence-to-sequence speech recognition with the transformer in Mandarin Chinese, (2018).","DOI":"10.21437\/Interspeech.2018-1107"},{"key":"233_CR32","doi-asserted-by":"publisher","unstructured":"V. M. Shetty, M. Sagaya Mary N.J., in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Improving the performance of transformer based low resource speech recognition for Indian languages, (2020), pp. 8279\u20138283. https:\/\/doi.org\/10.1109\/ICASSP40776.2020.9053808.","DOI":"10.1109\/ICASSP40776.2020.9053808"},{"key":"233_CR33","unstructured":"S. Zhou, S. Xu, B. Xu, Multilingual end-to-end speech recognition with a single transformer on low-resource languages. arXiv preprint arXiv:1806.05059 (2018)."},{"key":"233_CR34","doi-asserted-by":"crossref","unstructured":"B. Zoph, D. Yuret, J. May, K. Knight, Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201 (2016).","DOI":"10.18653\/v1\/D16-1163"},{"key":"233_CR35","doi-asserted-by":"crossref","unstructured":"J. Cho, M. K. Baskar, R. Li, M. Wiesner, S. H. Mallidi, N. Yalta, M. Karafiat, S. Watanabe, T. Hori, in 2018 IEEE Spoken Language Technology Workshop (SLT). Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling (IEEE, 2018), pp. 521\u2013527.","DOI":"10.1109\/SLT.2018.8639655"},{"key":"233_CR36","unstructured":"J. Meyer, Multi-task and transfer learning in low-resource speech recognition (The University of Arizona, 2019)."},{"key":"233_CR37","doi-asserted-by":"crossref","unstructured":"S. Dalmia, R. Sanabria, F. Metze, A. W. Black, in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Sequence-based multi-lingual low resource speech recognition (IEEE, 2018), pp. 4909\u20134913.","DOI":"10.1109\/ICASSP.2018.8461802"},{"key":"233_CR38","doi-asserted-by":"crossref","unstructured":"L. Pan, S. Li, L. Wang, J. Dang, in Proc. APSIPA ASC. Effective training end-to-end ASR systems for low-resource Lhasa dialect of Tibetan language, (2019).","DOI":"10.1109\/APSIPAASC47483.2019.9023100"},{"key":"233_CR39","doi-asserted-by":"crossref","unstructured":"T. N. Sainath, R. Prabhavalka, S. Kumar, S. Lee, A. Kannan, D. Rybach, V. Schogol, P. Nguyen, B. Li, Y. Wu, Z. Chen, C. Chiu, in Proc. IEEE-ICASSP. No need for a lexicon? Evaluating the value of the pronunciation lexicon in end-to-end models, (2018), pp. 5859\u20135863.","DOI":"10.1109\/ICASSP.2018.8462380"},{"key":"233_CR40","doi-asserted-by":"crossref","unstructured":"A. Kannan, Y. Wu, P. Nguyen, T. N. Sainath, Z. Chen, R. Prabhavalkar, in Proc. IEEE-ICASSP. An analysis of incorporating an external language model into a sequence-to-sequence model, (2018), pp. 5824\u20135828.","DOI":"10.1109\/ICASSP.2018.8462682"},{"key":"233_CR41","doi-asserted-by":"crossref","unstructured":"H. Bu, J. Du, X. Na, B. Wu, H. Zheng, in Proc. Oriental COCOSDA. AIShell-1: an open-source Mandarin speech corpus and a speech recognition baseline, (2017).","DOI":"10.1109\/ICSDA.2017.8384449"}],"container-title":["EURASIP Journal on Audio, Speech, and Music Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13636-021-00233-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13636-021-00233-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13636-021-00233-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T17:14:03Z","timestamp":1642007643000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13636-021-00233-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,12]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["233"],"URL":"https:\/\/doi.org\/10.1186\/s13636-021-00233-4","relation":{},"ISSN":["1687-4722"],"issn-type":[{"value":"1687-4722","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,12]]},"assertion":[{"value":"26 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2"}}