{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T14:28:15Z","timestamp":1765376895270,"version":"3.46.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>The contextual representations generated by multilingual pre-trained language models involve semantic content that conveys the meaning of sentences, as well as language-dependent nuances that indicate language-specific information. However, for cross-lingual machine learning tasks, particularly those suffering from data scarcity, the language-specific information blended in the contextual representation not only increases complexity but also compromises performance. This challenge is especially critical for natural language understanding (NLU) tasks such as intent detection (ID) and slot filling (SF), which are fundamental for the functionality of multilingual dialogue systems. Central to the idea is eradicating language-specific information from the contextual representation generated by an encoder through an adversarial manner, while preserving semantic information through input reconstruction via a decoder. In this regard, we propose an encoder-decoder model that employs adversarial learning techniques to enhance knowledge transferability across diverse languages for cross-lingual NLU tasks. Experimental results on two publicly available datasets, Facebook-multilingual (XTOD) and Persian-ATIS, demonstrate that our model significantly outperforms its main counterparts and achieves competitive results compared to state-of-the-art models across diverse languages in zero-shot scenarios. Notably, our findings highlight that achieving language-independent representations through adversarial learning followed by multi-task learning improves the model's performance in terms of accuracy and F1-score for NLU tasks.<\/jats:p>","DOI":"10.1145\/3771926","type":"journal-article","created":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T10:14:42Z","timestamp":1760523282000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["An Invasive Embedding Model in Favor of Low-Resource Languages Understanding"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9373-9629","authenticated-orcid":false,"given":"Saedeh","family":"Tahery","sequence":"first","affiliation":[{"name":"K. N. Toosi University of Technology","place":["Tehran, Iran (the Islamic Republic of)"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2850-0616","authenticated-orcid":false,"given":"Saeed","family":"Farzi","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler","place":["Trento, Italy"]},{"name":"K. N. Toosi University of Technology","place":["Trento, Italy"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,12,10]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISAS60782.2023.10391814"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11431-020-1692-3"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.42"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2021\/622"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.metanlp-1.5"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-acl.319"},{"key":"e_1_3_3_8_2","first-page":"351","article-title":"Crossing the conversational chasm: A primer on multilingual task-oriented dialogue systems","author":"Razumovskaia E.","year":"2022","unstructured":"E. Razumovskaia, G. GLAVA\u0160, O. Majewska, A. Korhonen, and I. Vuli\u0107. 2022. Crossing the conversational chasm: A primer on multilingual task-oriented dialogue systems. Journal of Artificial Intelligence Research 74 (2022), 351\u20131402.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.5555\/3243335"},{"key":"e_1_3_3_10_2","first-page":"4158","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)","author":"Tahery S.","year":"2024","unstructured":"S. Tahery, S. Kianian, and S. Farzi. 2024. Cross-Lingual NLU: Mitigating language-specific impact in embeddings leveraging adversarial learning. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 4158\u20134163."},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K19-1035"},{"key":"e_1_3_3_12_2","first-page":"2672","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow I.","year":"2014","unstructured":"I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. BENGIO. 2014. Generative adversarial nets. Advances in Neural Information Processing Systems 27 (2014), 2672\u20132680.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-023-09132-5"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.201"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1380"},{"key":"e_1_3_3_16_2","article-title":"Learned in translation: Contextualized word vectors","author":"Mccann B.","year":"2017","unstructured":"B. Mccann, J. Bradbury, C. Xiong, and R. Socher. 2017. Learned in translation: Contextualized word vectors. Advances in Neural Information Processing Systems 15 (2017), 6294--6305.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-3023"},{"key":"e_1_3_3_18_2","first-page":"4483","volume-title":"Proceedings of the EMNLP 2020","author":"Lauscher A.","year":"2020","unstructured":"A. Lauscher, V. Ravishankar, I. vuli\u0107, and G. Glava\u0161. 2020. From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers. In Proceedings of the EMNLP 2020. 4483\u20134499."},{"key":"e_1_3_3_19_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics","author":"Devlin J.","year":"2019","unstructured":"J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. 4171\u20134186."},{"key":"e_1_3_3_20_2","first-page":"8440","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Conneau A.","year":"2019","unstructured":"A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzm\u00e1n, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 8440\u20138451."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1309"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3609483"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i17.29843"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00482"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-022-01761-x"},{"key":"e_1_3_3_26_2","first-page":"2809","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC COLING 2024)","author":"JI S.","year":"2024","unstructured":"S. JI, T. Mickus, V. Segonne, and J. Tiedemann. 2024. Can machine translation bridge multilingual pretraining and cross-lingual transfer learning? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC COLING 2024). 2809\u20132818."},{"key":"e_1_3_3_27_2","first-page":"1","article-title":"Lexicon-based fine-tuning of multilingual language models for low-resource language sentiment analysis","author":"Dhananjaya V.","year":"2024","unstructured":"V. Dhananjaya, S. Ranathunga, and S. Jayasena. 2024. Lexicon-based fine-tuning of multilingual language models for low-resource language sentiment analysis. CAAI Transactions on Intelligence Technology 9 (2024), 1\u201310.","journal-title":"CAAI Transactions on Intelligence Technology"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.11.025"},{"key":"e_1_3_3_29_2","first-page":"264","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics: EACL 2024","author":"Lu H.","year":"2024","unstructured":"H. Lu, H. Huang, D. Zhang, F. Wei, and W. LAM. 2024. Revamping multilingual agreement bidirectionally via switched back-translation for multilingual neural machine translation. In Proceedings of the Findings of the Association for Computational Linguistics: EACL 2024. 264\u2013275."},{"key":"e_1_3_3_30_2","first-page":"418","volume-title":"Proceedings of the 6th Conference on Machine Translation (WMT)","author":"Liao B.","year":"2021","unstructured":"B. Liao, S. Khadivi, and S. Hewavitharana. 2021. Back-translation for large-scale multilingual machine translation. In Proceedings of the 6th Conference on Machine Translation (WMT) Association for Computational Linguistics, 418\u2013424."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.42"},{"key":"e_1_3_3_32_2","volume-title":"Proceedings of the Machine Learning Research","author":"song K.","year":"2019","unstructured":"K. song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu. 2019. Mass: Masked sequence to sequence pre-training for language generation. In Proceedings of the Machine Learning Research."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-short.68"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-020-09866-x"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.841"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.163"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/SLT.2014.7078634"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2016-402"},{"key":"e_1_3_3_39_2","first-page":"5467","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019","author":"H. E","year":"2019","unstructured":"E H., NIU P., Z. Chen, and M. Song. 2019. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, 5467\u20135471."},{"key":"e_1_3_3_40_2","article-title":"Multitask learning with knowledge base for joint intent detection and slot filling","author":"He T.","year":"2021","unstructured":"T. He, X. Xu, Y. Wu, H. Wang, and J. CHEN. 2021. Multitask learning with knowledge base for joint intent detection and slot filling. Applied Sciences 11, 4887 (2021).","journal-title":"Applied Sciences"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9414110"},{"key":"e_1_3_3_42_2","first-page":"4743","article-title":"Bi-Directional Joint neural networks for intent classification and slot filling","author":"Han S. C.","year":"2021","unstructured":"S. C. Han, S. Long, H. Li, H. Weld, and J. Poon. 2021. Bi-Directional Joint neural networks for intent classification and slot filling. Proceedings of the. Interspeech. 4743\u20134747.","journal-title":"Proceedings of the. Interspeech"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-020-09718-4"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2954766"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2118"},{"key":"e_1_3_3_46_2","first-page":"2993","volume-title":"Proceedings of the IJCAI","author":"Zhang X.","year":"2016","unstructured":"X. Zhang and H. Wang. 2016. A joint model of intent determination and slot filling for spoken language understanding. In Proceedings of the IJCAI, New York, NY, USA, 2993\u20132999."},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2016-1352"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2024.127725"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.410"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2018.8461905"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6362"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/533"},{"key":"e_1_3_3_53_2","first-page":"4492","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Ma Z.","year":"2022","unstructured":"Z. Ma, J. Ye, X. Yang, and J. Liu. 2022. Hcld: A hierarchical framework for zero-shot cross-lingual dialogue system. In Proceedings of the 29th International Conference on Computational Linguistics. 4492\u20134498."},{"key":"e_1_3_3_54_2","first-page":"6388","volume-title":"Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI-24)","author":"Li Z.","year":"2024","unstructured":"Z. Li, C. Hu, J. Chen, Z. Chen, X. Guo, and R. Zhang. 2024. Improving Zero-Shot Cross-Lingual Transfer via Progressive Code-Switching. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI-24), 6388\u20136396."},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.191"},{"key":"e_1_3_3_56_2","first-page":"1","article-title":"Transforming conversations with AI\u2014a comprehensive study of ChatGPT","author":"Bansal G.","year":"2024","unstructured":"G. Bansal, V. Chamola, A. Hussain, M. Guizani, and D. Niyato. 2024. Transforming conversations with AI\u2014a comprehensive study of ChatGPT. Cognitive Computation 5 (2024), 1\u201324.","journal-title":"Cognitive Computation"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2025.3574115"},{"key":"e_1_3_3_58_2","first-page":"17877","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)","author":"Zhu Z.","year":"2024","unstructured":"Z. Zhu, X. Cheng, H. An, Z. Wang, D. Chen, and Z. Huang. 2024. Zero-Shot spoken language understanding via large language models: A preliminary study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 17877\u201317883."},{"key":"e_1_3_3_59_2","unstructured":"J.-G. Ortiz-Barajas H. Gomez-Adorno and T. Solorio. 2024. HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling. arXiv:2407.01411. Retrieved from https:\/\/arxiv.org\/abs\/2407.01411"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.eacl-srw.29"},{"key":"e_1_3_3_61_2","first-page":"7871","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis M.","year":"2019","unstructured":"M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 7871\u20137880."},{"key":"e_1_3_3_62_2","author":"Akbari M.","year":"2023","unstructured":"M. Akbari, A. H. Karimi, T. Saeedi, Z. Saeidi, K. Ghezelbash, F. Shamsezat, M. Akbari, and A. Mohades. 2023. A Persian Benchmark for Joint Intent Detection and Slot Filling. arXiv:2303.00408. Retrieved from https:\/\/arxiv.org\/abs\/2303.00408","journal-title":"A Persian Benchmark for Joint Intent Detection and Slot Filling"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.26615\/978-954-452-072-4_025"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3771926","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T14:25:13Z","timestamp":1765376713000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3771926"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,10]]},"references-count":62,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3771926"],"URL":"https:\/\/doi.org\/10.1145\/3771926","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2025,12,10]]},"assertion":[{"value":"2024-08-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}