{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:27:11Z","timestamp":1760059631606,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T00:00:00Z","timestamp":1750896000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Foundation and Cutting-Edge Technologies Research Program of Henan Province","award":["252102211067","252102210064","252102210124"],"award-info":[{"award-number":["252102211067","252102210064","252102210124"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Bilingual Word Embeddings (BWEs) are crucial for multilingual NLP tasks, enabling cross-lingual transfer. While traditional joint training methods require bilingual corpora, their applicability is limited for many language pairs, especially low-resource ones. Unsupervised methods, relying on the isomorphism assumption, suffer from performance degradation when dealing with non-isomorphic embedding spaces, which are common in distant language pairs. This structural asymmetry challenges conventional approaches. To address these limitations, we propose a novel unsupervised joint training method for BWEs. We leverage monolingual corpora and introduce a dynamic programming algorithm to extract bilingual text segments, facilitating concurrent BWE training without relying on explicit bilingual supervision. Our approach effectively mitigates the challenge posed by asymmetric, non-isomorphic spaces by jointly learning BWEs in a shared space. Extensive experiments demonstrate the superiority of our method compared to existing approaches, particularly for distant language pairs exhibiting significant structural asymmetry<\/jats:p>","DOI":"10.3390\/sym17071005","type":"journal-article","created":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T03:39:59Z","timestamp":1750909199000},"page":"1005","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Addressing Structural Asymmetry: Unsupervised Joint Training of Bilingual Embeddings for Non-Isomorphic Spaces"],"prefix":"10.3390","volume":"17","author":[{"given":"Lei","family":"Meng","sequence":"first","affiliation":[{"name":"College of Information Engineering, Xuchang University, Xuchang 461000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaona","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Software, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shangfeng","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Software, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaojun","family":"Zhao","sequence":"additional","affiliation":[{"name":"College of Software, Zhengzhou University of Light Industry, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,26]]},"reference":[{"key":"ref_1","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2020). mT5: A massively multilingual pre-trained text-to-text transformer. arXiv.","DOI":"10.18653\/v1\/2021.naacl-main.41"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wu, S., and Dredze, M. (2019, January 3\u20137). Beto, Bentz, Becas: The surprising cross-lingual effectiveness of Bert. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019. Association for Computational Linguistics, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1077"},{"key":"ref_4","unstructured":"Pires, T., Schlinger, E., and Garrette, D. (August, January 28). How Multilingual is Multilingual BERT?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_5","unstructured":"Zhu, S., Xu, S., Sun, H., Pan, L., Cui, M., Du, J., Jin, R., Branco, A., and Xiong, D. (2024). Multilingual Large Language Models: A Systematic Survey. arXiv."},{"key":"ref_6","unstructured":"Chiruzzo, L., Ritter, A., and Wang, L. (May, January 29). Large Language Models Are Cross-Lingual Knowledge-Free Reasoners. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, NM, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xu, Y., Hu, L., Zhao, J., Qiu, Z., Xu, K., Ye, Y., and Gu, H. (2025). A survey on multilingual large language models: Corpora, alignment, and bias. Front. Comput. Sci., 19.","DOI":"10.1007\/s11704-024-40579-4"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"101818","DOI":"10.1016\/j.inffus.2023.101818","article-title":"Bilingual word embedding fusion for robust unsupervised bilingual lexicon induction","volume":"97","author":"Cao","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/s10115-023-01925-3","article-title":"Mining parallel sentences from internet with multi-view knowledge distillation for low-resource language pairs","volume":"66","author":"Zhu","year":"2024","journal-title":"Knowl. Inf. Syst."},{"key":"ref_10","first-page":"1","article-title":"Unsupervised parallel sentences of machine translation for Asian language pairs","volume":"22","author":"Zhu","year":"2023","journal-title":"ACM Trans. Asian-Low-Resour. Lang. Inf. Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1007\/s10590-021-09274-0","article-title":"Improving bilingual word embeddings mapping with monolingual context information","volume":"35","author":"Zhu","year":"2021","journal-title":"Mach. Transl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, F., Dong, J., and Luo, H. (2024, January 20\u201327). Cl2cm: Improving cross-lingual cross-modal retrieval via cross-lingual knowledge transfer. Proceedings of the AAAI Conference on Artificial Intelligence 2024, Vancouver, BC, USA.","DOI":"10.1609\/aaai.v38i6.28376"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sun, Y., Zhu, S., Yifan, F., and Mi, C. (2021, January 6\u201311). Parallel sentences mining with transfer learning in an unsupervised setting. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, Online.","DOI":"10.18653\/v1\/2021.naacl-srw.17"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhu, S., Mi, C., and Shi, X. (2021, January 23\u201325). An explainable evaluation of unsupervised transfer learning for parallel sentences mining. Proceedings of the Web and Big Data: 5th International Joint Conference, APWeb-WAIM 2021, Guangzhou, China. Proceedings, Part I 5.","DOI":"10.1007\/978-3-030-85896-4_22"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"111304","DOI":"10.1016\/j.knosys.2023.111304","article-title":"Unsupervised multilingual machine translation with pretrained cross-lingual encoders","volume":"284","author":"Shen","year":"2024","journal-title":"Knowl.-Based Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"S\u00f8gaard, A., Ruder, S., and Vuli\u0107, I. (2018, January 15\u201320). On the Limitations of Unsupervised Bilingual Dictionary Induction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, VIC, Australia.","DOI":"10.18653\/v1\/P18-1072"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Glavas, G., Litschko, R., Ruder, S., and Vulic, I. (2019). How to (properly) evaluate cross-lingual word embeddings: On strong baselines, comparative analyses, and some misconceptions. arXiv.","DOI":"10.18653\/v1\/P19-1070"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1109\/TASLPRO.2025.3536174","article-title":"Reshaping Word Embedding Space with Monolingual Synonyms for Bilingual Lexicon Induction","volume":"33","author":"Ding","year":"2025","journal-title":"IEEE Trans. Audio, Speech Lang. Process."},{"key":"ref_19","unstructured":"Sannigrahi, S., and Read, J. (2022, January 26). Isomorphic Cross-lingual Embeddings for Low-Resource Languages. Proceedings of the 7th Workshop on Representation Learning for NLP, RepL4NLP@ACL 2022, Dublin, Ireland."},{"key":"ref_20","unstructured":"Gouws, S., Bengio, Y., and Corrado, G. (2015, January 6\u201311). Bilbowa: Fast bilingual distributed representations without word alignments. Proceedings of the International Conference on Machine Learning. PMLR, 2015, Lille, France."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3610289","article-title":"Wad-x: Improving zero-shot cross-lingual transfer via adapter-based word alignment","volume":"22","author":"Ahmat","year":"2023","journal-title":"ACM Trans. Asian -Low-Resour. Lang. Inf. Process."},{"key":"ref_22","unstructured":"Lample, G., Conneau, A., Ranzato, M., Denoyer, L., and J\u00e9gou, H. (May, January 30). Word translation without parallel data. Proceedings of the International Conference on Learning Representations 2018, Vancouver, BC, Canada."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2606","DOI":"10.1109\/TASLP.2023.3290425","article-title":"Dual Word Embedding for Robust Unsupervised Bilingual Lexicon Induction","volume":"31","author":"Cao","year":"2023","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_24","unstructured":"Feng, Z., Cao, H., Zhao, T., Wang, W., and Peng, W. (2022, January 12\u201317). Cross-lingual feature extraction from monolingual corpora for low-resource unsupervised bilingual lexicon induction. Proceedings of the 29th International Conference on Computational lLinguistics, Gyeongju, Republic of Korea."},{"key":"ref_25","unstructured":"Garnier, P., and Guinet, G. (2024). Semi-Supervised Learning for Bilingual Lexicon Induction. arXiv."},{"key":"ref_26","unstructured":"Bear, D. (2025). Leveraging Bilingual Dictionaries to Learn Word Embeddings for Low-Resource Languages, University of New Brunswick."},{"key":"ref_27","unstructured":"Artetxe, M., Labaka, G., and Agirre, E. (August, January 30). Learning bilingual word embeddings with (almost) no bilingual data. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xing, C., Wang, D., Liu, C., and Lin, Y. (2015). Normalized Word Embedding and Orthogonal Transform for Bilingual Word Translation In NAACL HLT 2015, The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, USA, 31 May 31\u20135 June 2015, The Association for Computational Linguistics.","DOI":"10.3115\/v1\/N15-1104"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Artetxe, M., Labaka, G., and Agirre, E. (2018). A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. arXiv.","DOI":"10.18653\/v1\/P18-1073"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Miaschi, A., and Dell\u2019Orletta, F. (2020, January 9). Contextual and non-contextual word embeddings: An in-depth linguistic investigation. Proceedings of the 5th Workshop on Representation Learning for NLP, Online.","DOI":"10.18653\/v1\/2020.repl4nlp-1.15"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Camacho-Collados, J., and Pilehvar, M.T. (2020, January 8\u201313). Embeddings in natural language processing. Proceedings of the 28th International Conference on Computational Linguistics: Tutorial Abstracts, Online.","DOI":"10.18653\/v1\/2020.coling-tutorials.2"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"10345","DOI":"10.1007\/s10462-023-10419-1","article-title":"Impact of word embedding models on text analytics in deep learning environment: A review","volume":"56","author":"Asudani","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1016\/j.inffus.2022.08.024","article-title":"Beyond word embeddings: A survey","volume":"89","author":"Incitti","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, I., Glava\u0161, G., Reichart, R., and Korhonen, A. (2019, January 3\u20137). Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1449"},{"key":"ref_36","unstructured":"Ding, Q., Cao, H., and Zhao, T. (20224, January 20\u201327). Enhancing bilingual lexicon induction via bi-directional translation pair retrieving. Proceedings of the AAAI Conference on Artificial Intelligence 2024, Vancouver, BC, Canada."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nakashole, N., and Flauger, R. (2018, January 15\u201320). Characterizing Departures from Linearity in Word Translation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, VIC, Australia.","DOI":"10.18653\/v1\/P18-2036"},{"key":"ref_38","unstructured":"Thrampoulidis, C. (2024). Implicit optimization bias of next-token prediction in linear models. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lample, G., Ott, M., Conneau, A., Denoyer, L., and Ranzato, M. (November, January 31). Phrase-Based & Neural Unsupervised Machine Translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1549"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1162\/tacl_a_00288","article-title":"Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond","volume":"7","author":"Artetxe","year":"2019","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Schwenk, H., Wenzek, G., Edunov, S., Grave, E., Joulin, A., and Fan, A. (2021, January 1\u20136). CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2021.acl-long.507"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"El-Kishky, A., Chaudhary, V., Guzm\u00e1n, F., and Koehn, P. (2020, January 16\u201320). CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.","DOI":"10.18653\/v1\/2020.emnlp-main.480"},{"key":"ref_43","first-page":"3111","article-title":"Distributed representations of words and phrases and their compositionality","volume":"26","author":"Mikolov","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_44","unstructured":"Hangya, V., and Fraser, A. (August, January 28). Unsupervised parallel sentence extraction with parallel segment detection helps machine translation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_45","unstructured":"Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (November, January 31). Unsupervised Multilingual Word Embeddings. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium."},{"key":"ref_46","unstructured":"Ammar, W., Mulcaire, G., Tsvetkov, Y., Lample, G., Dyer, C., and Smith, N.A. (2016). Massively Multilingual Word Embeddings. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1162\/coli_a_00446","article-title":"Survey of Low-Resource Machine Translation","volume":"48","author":"Haddow","year":"2022","journal-title":"Comput. Linguist."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/7\/1005\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:58:54Z","timestamp":1760032734000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/7\/1005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,26]]},"references-count":47,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["sym17071005"],"URL":"https:\/\/doi.org\/10.3390\/sym17071005","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,6,26]]}}}