{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T21:22:05Z","timestamp":1784064125136,"version":"3.55.0"},"reference-count":184,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2025,5,6]],"date-time":"2025-05-06T00:00:00Z","timestamp":1746489600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>Multilingual Language Models (MLLMs) such as mBERT, XLM, XLM-R, and the like, have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger MLLMs\u00a0covering a large number of languages, (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating MLLMs,\u00a0 (iii) analysing the performance of MLLMs\u00a0on monolingual, zero-shot cross-lingual and bilingual tasks, (iv) understanding the universal language patterns (if any) learnt by MLLMs,\u00a0 and (v) augmenting the (often) limited capacity of MLLMs\u00a0 to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to MLLMs. Based on our survey, we recommend some promising directions of future research.<\/jats:p>","DOI":"10.1145\/3727339","type":"journal-article","created":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T13:46:10Z","timestamp":1743515170000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["A Primer on Pretrained Multilingual Language Models"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4248-646X","authenticated-orcid":false,"given":"Sumanth","family":"Doddapaneni","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, IIT Madras, Chennai, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4976-0700","authenticated-orcid":false,"given":"Gowtham","family":"Ramesh","sequence":"additional","affiliation":[{"name":"Advanced Micro Devices Inc, Santa Clara, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3687-9922","authenticated-orcid":false,"given":"Mitesh","family":"Khapra","sequence":"additional","affiliation":[{"name":"Wadhwani School of Data Science &amp; AI, Indian Institute of Technology Madras, Chennai, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3143-9875","authenticated-orcid":false,"given":"Anoop","family":"Kunchukuttan","sequence":"additional","affiliation":[{"name":"Microsoft India R&amp;D Pvt Ltd, Hyderabad, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8732-0183","authenticated-orcid":false,"given":"Pratyush","family":"Kumar","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, IIT Madras, Chennai, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,6]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"4344","volume-title":"Proceedings of the 13th Language Resources and Evaluation Conference (LREC 2022) (Marseille, France, 20\u201325 June 2022)","author":"Abadji Julien","year":"2022","unstructured":"Julien Abadji, Pedro Javier Ortiz Su\u00e1rez, Laurent Romary, and Beno\u00eet Sagot. 2022. Towards a cleaner document-oriented multilingual crawled corpus. In Proceedings of the 13th Language Resources and Evaluation Conference (LREC 2022) (Marseille, France, 20\u201325 June 2022), Nicoletta Calzolari, Fr\u00e9d\u00e9ric B\u00e9chet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, H\u00e9l\u00e8ne Mazo, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, 4344\u20134355. https:\/\/aclanthology.org\/2022.lrec-1.463"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2401.01053"},{"key":"e_1_3_2_4_2","first-page":"4781","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020) (Marseille, France, May 11\u201316, 2020)","author":"Agerri Rodrigo","year":"2020","unstructured":"Rodrigo Agerri, I\u00f1aki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, and Eneko Agirre. 2020. Give your text representation models some love: The case for basque. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020) (Marseille, France, May 11\u201316, 2020), Nicoletta Calzolari, Fr\u00e9d\u00e9ric B\u00e9chet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, H\u00e9l\u00e8ne Mazo, Asunci\u00f3n Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, 4781\u20134788. https:\/\/www.aclweb.org\/anthology\/2020.lrec-1.588\/"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.755"},{"key":"e_1_3_2_6_2","unstructured":"Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin Alexandre Passos Siamak Shakeri Emanuel Taropa Paige Bailey Zhifeng Chen Eric Chu Jonathan H. Clark Laurent El Shafey Yanping Huang Kathy Meier-Hellstern Gaurav Mishra Erica Moreira Mark Omernick Kevin Robinson Sebastian Ruder Yi Tay Kefan Xiao Yuanzhong Xu Yujing Zhang Gustavo Hernandez Abrego Junwhan Ahn Jacob Austin Paul Barham Jan Botha James Bradbury Siddhartha Brahma Kevin Brooks Michele Catasta Yong Cheng Colin Cherry Christopher A. Choquette-Choo Aakanksha Chowdhery Cl\u00e9ment Crepy Shachi Dave Mostafa Dehghani Sunipa Dev Jacob Devlin Mark D\u00edaz Nan Du Ethan Dyer Vlad Feinberg Fangxiaoyu Feng Vlad Fienber Markus Freitag Xavier Garcia Sebastian Gehrmann Lucas Gonzalez Guy Gur-Ari Steven Hand Hadi Hashemi Le Hou Joshua Howland Andrea Hu Jeffrey Hui Jeremy Hurwitz Michael Isard Abe Ittycheriah Matthew Jagielski Wenhao Jia Kathleen Kenealy Maxim Krikun Sneha Kudugunta Chang Lan Katherine Lee Benjamin Lee Eric Li Music Li Wei Li YaGuang Li Jian Li Hyeontaek Lim Hanzhao Lin Zhongtao Liu Frederick Liu Marcello Maggioni Aroma Mahendru Joshua Maynez Vedant Misra Maysam Moussalem Zachary Nado John Nham Eric Ni Andrew Nystrom Alicia Parrish Marie Pellat Martin Polacek Alex Polozov Reiner Pope Siyuan Qiao Emily Reif Bryan Richter Parker Riley Alex Castro Ros Aurko Roy Brennan Saeta Rajkumar Samuel Renee Shelby Ambrose Slone Daniel Smilkov David R. So Daniel Sohn Simon Tokumine Dasha Valter Vijay Vasudevan Kiran Vodrahalli Xuezhi Wang Pidong Wang Zirui Wang Tao Wang John Wieting Yuhuai Wu Kelvin Xu Yunhan Xu Linting Xue Pengcheng Yin Jiahui Yu Qiao Zhang Steven Zheng Ce Zheng Weikang Zhou Denny Zhou Slav Petrov and Yonghui Wu. 2023. PaLM 2 Technical Report. arXiv:2305.10403 [cs.CL] https:\/\/arxiv.org\/abs\/2305.10403"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.FINDINGS-ACL.215"},{"key":"e_1_3_2_8_2","first-page":"4623","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020","author":"Artetxe Mikel","year":"2020","unstructured":"Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020) Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 4623\u20134637. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.421\/"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.421"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00288"},{"key":"e_1_3_2_11_2","unstructured":"Dzmitry Bahdanau Kyunghyun Cho and Yoshua Bengio. 2016. Neural Machine Translation by Jointly Learning to Align and Translate. arxiv:1409.0473 [cs.CL]"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2312.09993"},{"key":"e_1_3_2_13_2","article-title":"Language (technology) is power: A critical survey of \u201cbias\u201d in NLP","author":"Blodgett Su Lin","year":"2020","unstructured":"Su Lin Blodgett, Solon Barocas, Hal Daum\u00e9 III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of \u201cbias\u201d in NLP. arXiv preprint arXiv:2005.14050 (2020).","journal-title":"arXiv preprint arXiv:2005.14050"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K16-1002"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972474"},{"key":"e_1_3_2_16_2","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 NeurIPS 2020 December 6-12 2020 Virtual Hugo Larochelle Marc.Aurelio Ranzato Raia Hadsell Maria-Florina Balcan and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","unstructured":"Samuel Cahyawijaya Holy Lovenia Alham Fikri Aji Genta Indra Winata Bryan Wilie Fajri Koto Rahmad Mahendra Christian Wibisono Ade Romadhony Karissa Vincentio Jennifer Santoso David Moeljadi Cahya Wirawan Frederikus Hudi Muhammad Satrio Wicaksono Ivan Halim Parmonangan Ika Alfina Ilham Firdausi Putra Samsul Rahmadani Yulianti Oenang Ali Akbar Septiandri James Jaya Kaustubh D. Dhole Arie Ardiyanti Suryani Rifki Afina Putri Dan Su Keith Stevens Made Nindyatama Nityasya Muhammad Farid Adilazuarda Ryan Hadiwijaya Ryandito Diandaru Tiezheng Yu Vito Ghifari Wenliang Dai Yan Xu Dyah Damapuspita Haryo Akbarianto Wibowo Cuk Tho Ichwanul Muslim Karo Karo Tirana Fatyanosa Ziwei Ji Graham Neubig Timothy Baldwin Sebastian Ruder Pascale Fung Herry Sujaini Sakriani Sakti and Ayu Purwarianti. 2023. NusaCrowd: Open source initiative for indonesian NLP resources. In Findings of the Association for Computational Linguistics: ACL 2023 Toronto Canada July 9-14 2023 Anna Rogers Jordan L. Boyd-Graber and Naoaki Okazaki (Eds.). Association for Computational Linguistics 13745\u201313818. 10.18653\/V1\/2023.FINDINGS-ACL.868","DOI":"10.18653\/V1\/2023.FINDINGS-ACL.868"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","unstructured":"Samuel Cahyawijaya Genta Indra Winata Bryan Wilie Karissa Vincentio Xiaohong Li Adhiguna Kuncoro Sebastian Ruder Zhi Yuan Lim Syafri Bahar Masayu Khodra Ayu Purwarianti and Pascale Fung. 2021. IndoNLG: Benchmark and resources for evaluating indonesian natural language generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing Marie-Francine Moens Xuanjing Huang Lucia Specia and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics Online and Punta Cana Dominican Republic 8875\u20138898. 10.18653\/v1\/2021.emnlp-main.699","DOI":"10.18653\/v1\/2021.emnlp-main.699"},{"key":"e_1_3_2_19_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (Addis Ababa, Ethiopia, April 26\u201330, 2020)","author":"Cao Steven","year":"2020","unstructured":"Steven Cao, Nikita Kitaev, and Dan Klein. 2020. Multilingual alignment of contextual word representations. In Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (Addis Ababa, Ethiopia, April 26\u201330, 2020). OpenReview.net. https:\/\/openreview.net\/forum?id=r1xCMyBtPS"},{"key":"e_1_3_2_20_2","article-title":"Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation","author":"Cer Daniel","year":"2017","unstructured":"Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 (2017).","journal-title":"arXiv preprint arXiv:1708.00055"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.42"},{"key":"e_1_3_2_22_2","first-page":"5564","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020)","author":"Chi Ethan A.","year":"2020","unstructured":"Ethan A. Chi, John Hewitt, and Christopher D. Manning. 2020. Finding universal grammatical relations in multilingual BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5564\u20135577. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.493\/"},{"key":"e_1_3_2_23_2","first-page":"12","volume-title":"Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL\/IJCNLP 2020) (Suzhou, China, December 4\u20137, 2020","author":"Chi Zewen","year":"2020","unstructured":"Zewen Chi, Li Dong, Furu Wei, Xianling Mao, and Heyan Huang. 2020. Can monolingual pretrained models help cross-lingual classification?. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL\/IJCNLP 2020) (Suzhou, China, December 4\u20137, 2020), Kam-Fai Wong, Kevin Knight, and Hua Wu (Eds.). Association for Computational Linguistics, 12\u201317. https:\/\/www.aclweb.org\/anthology\/2020.aacl-main.2\/"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.280"},{"key":"e_1_3_2_25_2","first-page":"3418","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(ACL\/IJCNLP 2021) (Volume 1: Long Papers) (Virtual Event, August 1\u20136, 2021)","author":"Chi Zewen","year":"2021","unstructured":"Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, and Furu Wei. 2021. Improving pretrained cross-lingual language models via self-labeled word alignment. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(ACL\/IJCNLP 2021) (Volume 1: Long Papers) (Virtual Event, August 1\u20136, 2021), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 3418\u20133430. https:\/\/aclanthology.org\/2021.acl-long.265"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Zewen Chi Shaohan Huang Li Dong Shuming Ma Saksham Singhal Payal Bajaj Xia Song and Furu Wei. 2021. XLM-E: Cross-lingual Language Model Pre-training via ELECTRA. arxiv:2106.16138 [cs.CL]","DOI":"10.18653\/v1\/2022.acl-long.427"},{"key":"e_1_3_2_27_2","article-title":"What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties","author":"Choenni Rochelle","year":"2020","unstructured":"Rochelle Choenni and Ekaterina Shutova. 2020. What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties. arXiv preprint arXiv:2009.12862 (2020).","journal-title":"arXiv preprint arXiv:2009.12862"},{"key":"e_1_3_2_28_2","unstructured":"Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra Adam Roberts Paul Barham Hyung Won Chung Charles Sutton Sebastian Gehrmann Parker Schuh Kensen Shi Sasha Tsvyashchenko Joshua Maynez Abhishek Rao Parker Barnes Yi Tay Noam Shazeer Vinodkumar Prabhakaran Emily Reif Nan Du Ben Hutchinson Reiner Pope James Bradbury Jacob Austin Michael Isard Guy Gur-Ari Pengcheng Yin Toju Duke Anselm Levskaya Sanjay Ghemawat Sunipa Dev Henryk Michalewski Xavier Garcia Vedant Misra Kevin Robinson Liam Fedus Denny Zhou Daphne Ippolito David Luan Hyeontaek Lim Barret Zoph Alexander Spiridonov Ryan Sepassi David Dohan Shivani Agrawal Mark Omernick Andrew M. Dai Thanumalayan Sankaranarayana Pillai Marie Pellat Aitor Lewkowycz Erica Moreira Rewon Child Oleksandr Polozov Katherine Lee Zongwei Zhou Xuezhi Wang Brennan Saeta Mark Diaz Orhan Firat Michele Catasta Jason Wei Kathy Meier-Hellstern Douglas Eck Jeff Dean Slav Petrov and Noah Fiedel. 2023. PaLM: Scaling language modeling with pathways. J. Mach. Learn. Res. 24 (2023) 240:1\u2013240:113. http:\/\/jmlr.org\/papers\/v24\/22-1144.html"},{"key":"e_1_3_2_29_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR 2021) (Virtual Event, Austria, May 3\u20137, 2021","author":"Chung Hyung Won","year":"2021","unstructured":"Hyung Won Chung, Thibault F\u00e9vry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. Rethinking embedding coupling in pre-trained language models. In Proceedings of the 9th International Conference on Learning Representations (ICLR 2021) (Virtual Event, Austria, May 3\u20137, 2021). OpenReview.net. https:\/\/openreview.net\/forum?id=xpFFI_NtgpW"},{"key":"e_1_3_2_30_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR 2023) (Kigali, Rwanda, May 1\u20135, 2023)","author":"Chung Hyung Won","year":"2023","unstructured":"Hyung Won Chung, Xavier Garcia, Adam Roberts, Yi Tay, Orhan Firat, Sharan Narang, and Noah Constant. 2023. UniMax: Fairer and more effective language sampling for large-scale multilingual pretraining. In Proceedings of the 11th International Conference on Learning Representations (ICLR 2023) (Kigali, Rwanda, May 1\u20135, 2023). OpenReview.net. https:\/\/openreview.net\/pdf?id=kXwdL1cWOAi"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.367"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","DOI":"10.1162\/tacl_a_00317","article-title":"TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages","author":"Clark Jonathan H.","year":"2020","unstructured":"Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics (2020).","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"e_1_3_2_33_2","first-page":"8440","article-title":"Unsupervised cross-lingual representation learning at scale","author":"Conneau Alexis","year":"2020","unstructured":"Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm\u00e1n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 8440\u20138451. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.747\/","journal-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020"},{"key":"e_1_3_2_34_2","first-page":"7057","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019) (8\u201314 December 2019, Vancouver, BC, Canada","author":"Conneau Alexis","year":"2019","unstructured":"Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019) (8\u201314 December 2019, Vancouver, BC, Canada). Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d\u2019Alch\u00e9-Buc, Emily B. Fox, and Roman Garnett (Eds.). 7057\u20137067. http:\/\/papers.nips.cc\/paper\/8928-cross-lingual-language-model-pretraining"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1269"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.536"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","unstructured":"Marta R. Costa-juss\u00e0 James Cross Onur \u00c7elebi Maha Elbayad Kenneth Heafield Kevin Heffernan Elahe Kalbassi Janice Lam Daniel Licht Jean Maillard Anna Y. Sun Skyler Wang Guillaume Wenzek Al Youngblood Bapi Akula Lo\u00efc Barrault Gabriel Mejia Gonzalez Prangthip Hansanti John Hoffman Semarley Jarrett Kaushik Ram Sadagopan Dirk Rowe Shannon Spruit Chau Tran Pierre Andrews Necip Fazil Ayan Shruti Bhosale Sergey Edunov Angela Fan Cynthia Gao Vedanuj Goswami Francisco Guzm\u00e1n Philipp Koehn Alexandre Mourachko Christophe Ropers Safiyyah Saleem Holger Schwenk and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation. CoRR abs\/2207.04672 (2022). 10.48550\/ARXIV.2207.04672arXiv:2207.04672","DOI":"10.48550\/ARXIV.2207.04672"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2304.08177"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3406095"},{"key":"e_1_3_2_40_2","article-title":"Mono vs multilingual transformer-based models: A comparison across several language tasks","volume":"2007","author":"Feij\u00f3 Diego de Vargas","year":"2020","unstructured":"Diego de Vargas Feij\u00f3 and Viviane Pereira Moreira. 2020. Mono vs multilingual transformer-based models: A comparison across several language tasks. CoRR abs\/2007.09757 (2020). arxiv:2007.09757https:\/\/arxiv.org\/abs\/2007.09757","journal-title":"CoRR"},{"key":"e_1_3_2_41_2","unstructured":"Wietse de Vries Andreas van Cranenburgh Arianna Bisazza Tommaso Caselli Gertjan van Noord and Malvina Nissim. 2019. BERTje: A Dutch BERT Model. arxiv:1912.09582 [cs.CL]"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.389"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.ACL-LONG.693"},{"key":"e_1_3_2_45_2","volume-title":"Proceedings of the 3rd International Workshop on Paraphrasing (IWP\u201905)","author":"Dolan William B.","year":"2005","unstructured":"William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the 3rd International Workshop on Paraphrasing (IWP\u201905)."},{"key":"e_1_3_2_46_2","first-page":"13042","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019) (December 8\u201314, 2019, Vancouver, BC, Canada","author":"Dong Li","year":"2019","unstructured":"Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019) (December 8\u201314, 2019, Vancouver, BC, Canada), Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d\u2019Alch\u00e9-Buc, Emily B. Fox, and Roman Garnett (Eds.). 13042\u201313054. https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/c20bb2d9a50d5ac1f713f8b34d9aac5a-Abstract.html"},{"key":"e_1_3_2_47_2","first-page":"2112","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Dou Zi-Yi","year":"2021","unstructured":"Zi-Yi Dou and Graham Neubig. 2021. Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Association for Computational Linguistics, Online, 2112\u20132128. https:\/\/www.aclweb.org\/anthology\/2021.eacl-main.181"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.358"},{"key":"e_1_3_2_49_2","first-page":"644","volume-title":"Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Dyer Chris","year":"2013","unstructured":"Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Atlanta, Georgia, 644\u2013648. https:\/\/www.aclweb.org\/anthology\/N13-1073"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1572"},{"key":"e_1_3_2_51_2","article-title":"Language-agnostic BERT sentence embedding","volume":"2007","author":"Feng Fangxiaoyu","year":"2020","unstructured":"Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic BERT sentence embedding. CoRR abs\/2007.01852 (2020). arxiv:2007.01852https:\/\/arxiv.org\/abs\/2007.01852","journal-title":"CoRR"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","unstructured":"Jack FitzGerald Christopher Hench Charith Peris Scott Mackie Kay Rottmann Ana Sanchez Aaron Nash Liam Urbach Vishesh Kakarala Richa Singh Swetha Ranganath Laurie Crist Misha Britan Wouter Leeuwis G\u00f6khan T\u00fcr and Prem Natarajan. 2023. MASSIVE: A 1M-example multilingual natural language understanding dataset with 51 typologically-diverse languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) ACL 2023 Toronto Canada July 9-14 2023 Anna Rogers Jordan L. Boyd-Graber and Naoaki Okazaki (Eds.). Association for Computational Linguistics 4277\u20134302. 10.18653\/V1\/2023.ACL-LONG.235","DOI":"10.18653\/V1\/2023.ACL-LONG.235"},{"key":"e_1_3_2_53_2","article-title":"Interpretable multi-dataset evaluation for named entity recognition","author":"Fu Jinlan","year":"2020","unstructured":"Jinlan Fu, Pengfei Liu, and Graham Neubig. 2020. Interpretable multi-dataset evaluation for named entity recognition. arXiv preprint arXiv:2011.06854 (2020).","journal-title":"arXiv preprint arXiv:2011.06854"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","unstructured":"Jay P. Gala Pranjal A. Chitale Raghavan AK Sumanth Doddapaneni Varun Gumma Aswanth Kumar Janki Nawale Anupama Sujatha Ratish Puduppully Vivek Raghavan Pratyush Kumar Mitesh M. Khapra Raj Dabre and Anoop Kunchukuttan. 2023. IndicTrans2: Towards high-quality and accessible machine translation models for all 22 scheduled Indian languages. CoRR abs\/2305.16307 (2023). 10.48550\/ARXIV.2305.16307arxiv:2305.16307","DOI":"10.48550\/ARXIV.2305.16307"},{"key":"e_1_3_2_55_2","first-page":"202","volume-title":"Proceedings of the 5th Conference on Machine Translation","author":"Goyal Vikrant","year":"2020","unstructured":"Vikrant Goyal, Anoop Kunchukuttan, Rahul Kejriwal, Siddharth Jain, and Amit Bhagwat. 2020. Contact relatedness can help improve multilingual NMT: Microsoft STCI-MT @ WMT20. In Proceedings of the 5th Conference on Machine Translation. Association for Computational Linguistics, Online, 202\u2013206. https:\/\/www.aclweb.org\/anthology\/2020.wmt-1.19"},{"key":"e_1_3_2_56_2","volume-title":"Language Universals","author":"Greenberg Joseph H.","year":"1966","unstructured":"Joseph H. Greenberg. 1966. Language Universals. Mouton The Hague."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2021.FINDINGS-ACL.413"},{"key":"e_1_3_2_58_2","unstructured":"Kaiming He Haoqi Fan Yuxin Wu Saining Xie and Ross Girshick. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. arxiv:1911.05722 [cs.CV]"},{"key":"e_1_3_2_59_2","series-title":"Proceedings of Machine Learning Research","first-page":"2790","volume-title":"Proceedings of the 36th International Conference on Machine Learning","volume":"97","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 2790\u20132799. http:\/\/proceedings.mlr.press\/v97\/houlsby19a.html"},{"key":"e_1_3_2_60_2","volume-title":"Proceedings of the 10th International Conference on Learning Representations (ICLR 2022) (Virtual Event, April 25\u201329, 2022","author":"Hu Edward J.","year":"2022","unstructured":"Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations (ICLR 2022) (Virtual Event, April 25\u201329, 2022). OpenReview.net. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"e_1_3_2_61_2","unstructured":"Junjie Hu Melvin Johnson Orhan Firat Aditya Siddhant and Graham Neubig. 2021. Explicit Alignment Objectives for Multilingual Bidirectional Encoders. arxiv:2010.07972 [cs.CL]"},{"key":"e_1_3_2_62_2","series-title":"Proceedings of Machine Learning Research","first-page":"4411","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13\u201318 July 2020, Virtual Event","volume":"119","author":"Hu Junjie","year":"2020","unstructured":"Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13\u201318 July 2020, Virtual Event(Proceedings of Machine Learning Research, Vol. 119). PMLR, 4411\u20134421. http:\/\/proceedings.mlr.press\/v119\/hu20b.html"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1252"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5603"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.ACL-LONG.61"},{"key":"e_1_3_2_66_2","volume-title":"First Quora Dataset Release: Question Pairs","author":"Iyer Shankar","year":"2017","unstructured":"Shankar Iyer, Nikhil Dandekar, and Kornel Csernai. 2017. First Quora Dataset Release: Question Pairs. https:\/\/data.quora.com\/First-Quora-Dataset-Release-Question-Pairs"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.147"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1162\/TACL_A_00300"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.560"},{"key":"e_1_3_2_70_2","volume-title":"8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020","author":"K Karthikeyan","year":"2020","unstructured":"Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilingual BERT: An empirical study. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=HJeT3yrtDr"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.445"},{"key":"e_1_3_2_72_2","article-title":"AMMUS: A survey of transformer-based pretrained models in natural language processing","volume":"2108","author":"Kalyan Katikapalli Subramanyam","year":"2021","unstructured":"Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and S. Sangeetha. 2021. AMMUS: A survey of transformer-based pretrained models in natural language processing. ArXiv abs\/2108.05542 (2021).","journal-title":"ArXiv"},{"key":"e_1_3_2_73_2","first-page":"3146","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017 (December 4\u20139, 2017, Long Beach, CA)","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017 (December 4\u20139, 2017, Long Beach, CA)Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 3146\u20133154. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/6449f44a102fde848669bdd9eb6b76fa-Abstract.html"},{"key":"e_1_3_2_74_2","article-title":"On the evaluation of contextual embeddings for zero-shot cross-lingual transfer learning","volume":"2004","author":"Keung Phillip","year":"2020","unstructured":"Phillip Keung, Yichao Lu, Julian Salazar, and Vikas Bhardwaj. 2020. On the evaluation of contextual embeddings for zero-shot cross-lingual transfer learning. CoRR abs\/2004.15001 (2020). arxiv:2004.15001https:\/\/arxiv.org\/abs\/2004.15001","journal-title":"CoRR"},{"key":"e_1_3_2_75_2","unstructured":"Mohammed Safi Ur Rahman Khan Priyam Mehta Ananth Sankar Umashankar Kumaravelan Sumanth Doddapaneni Suriyaprasaad G Varun Balan G Sparsh Jain Anoop Kunchukuttan Pratyush Kumar Raj Dabre and Mitesh M. Khapra. 2024. IndicLLMSuite: A blueprint for creating Pre-training and fine-tuning datasets for Indian Languages. arxiv:2403.06350 [cs.CL]"},{"key":"e_1_3_2_76_2","unstructured":"Simran Khanuja Diksha Bansal Sarvesh Mehtani Savya Khosla Atreyee Dey Balaji Gopalan Dilip Kumar Margam Pooja Aggarwal Rajiv Teja Nagipogu Shachi Dave Shruti Gupta Subhash Chandra Bose Gali Vish Subramanian and Partha Talukdar. 2021. MuRIL: Multilingual representations for Indian languages. arxiv:2103.10730 [cs.CL]"},{"key":"e_1_3_2_77_2","article-title":"MergeDistill: Merging pre-trained language models using distillation","volume":"2106","author":"Khanuja Simran","year":"2021","unstructured":"Simran Khanuja, Melvin Johnson, and P. Talukdar. 2021. MergeDistill: Merging pre-trained language models using distillation. ArXiv abs\/2106.02834 (2021).","journal-title":"ArXiv"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/d18-2012"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.360"},{"key":"e_1_3_2_80_2","unstructured":"Hugo Lauren\u00e7on Lucile Saulnier Thomas Wang Christopher Akiki Albert Villanova del Moral Teven Le Scao Leandro von Werra Chenghao Mou Eduardo Gonz\u00e1lez Ponferrada Huu Nguyen J\u0301\u2019org Frohberg Mario Sasko Quentin Lhoest Angelina McMillan-Major G\u00e9rard Dupont Stella Biderman Anna Rogers Loubna Ben Allal Francesco De Toni Giada Pistilli Olivier Nguyen Somaieh Nikpoor Maraim Masoud Pierre Colombo Javier de la Rosa Paulo Villegas Tristan Thrush Shayne Longpre Sebastian Nagel Leon Weber Manuel Mu\u00f1oz Jian Zhu Daniel van Strien Zaid Alyafeai Khalid Almubarak Minh Chien Vu Itziar Gonzalez-Dios Aitor Soroa Kyle Lo Manan Dey Pedro Ortiz Suarez Aaron Gokaslan Shamik Bose David Ifeoluwa Adelani Long Phan Hieu Tran Ian Yu Suhas Pai Jenny Chim Violette Lepercq Suzana Ilic Margaret Mitchell Alexandra Sasha Luccioni and Yacine Jernite. 2022. The BigScience ROOTS Corpus: A 1.6TB composite multilingual dataset. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022 NeurIPS 2022 New Orleans LA USA November 28 - December 9 2022 Sanmi Koyejo S. Mohamed A. Agarwal Danielle Belgrave K. Cho and A. Oh (Eds.). http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/ce9e92e3de2372a4b93353eb7f3dc0bd-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_81_2","article-title":"From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers","volume":"2005","author":"Lauscher Anne","year":"2020","unstructured":"Anne Lauscher, Vinit Ravishankar, Ivan Vulic, and Goran Glavas. 2020. From zero to hero: On the limitations of zero-shot cross-lingual transfer with multilingual transformers. CoRR abs\/2005.00633 (2020). arxiv:2005.00633https:\/\/arxiv.org\/abs\/2005.00633","journal-title":"CoRR"},{"key":"e_1_3_2_82_2","first-page":"2479","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Le Hang","year":"2020","unstructured":"Hang Le, Lo\u00efc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Beno\u00eet Crabb\u00e9, Laurent Besacier, and Didier Schwab. 2020. FlauBERT: Unsupervised language model pre-training for French. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, Marseille, France, 2479\u20132490. https:\/\/www.aclweb.org\/anthology\/2020.lrec-1.302"},{"key":"e_1_3_2_83_2","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (NeurIPS 2020) (December 6\u201312, 2020, virtual","author":"Lewis Mike","year":"2020","unstructured":"Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida Wang, and Luke Zettlemoyer. 2020. Pre-training via paraphrasing. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (NeurIPS 2020) (December 6\u201312, 2020, virtual). Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/d6f1dd034aabde7657e6680444ceff62-Abstract.html"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2020.ACL-MAIN.703"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.653"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","unstructured":"Yaobo Liang Nan Duan Yeyun Gong Ning Wu Fenfei Guo Weizhen Qi Ming Gong Linjun Shou Daxin Jiang Guihong Cao Xiaodong Fan Ruofei Zhang Rahul Agrawal Edward Cui Sining Wei Taroon Bharti Ying Qiao Jiun-Hung Chen Winnie Wu Shuguang Liu Fan Yang Daniel Campos Rangan Majumder and Ming Zhou. 2020. XGLUE: A new benchmark datasetfor cross-lingual pre-training understanding and generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing EMNLP 2020 Online November 16-20 2020 Bonnie Webber Trevor Cohn Yulan He and Yang Liu (Eds.). Association for Computational Linguistics 6008\u20136018. 10.18653\/V1\/2020.EMNLP-MAIN.484","DOI":"10.18653\/V1\/2020.EMNLP-MAIN.484"},{"key":"e_1_3_2_87_2","article-title":"How language-neutral is multilingual BERT?","author":"Libovick\u1ef3 Jind\u0159ich","year":"2019","unstructured":"Jind\u0159ich Libovick\u1ef3, Rudolf Rosa, and Alexander Fraser. 2019. How language-neutral is multilingual BERT? arXiv preprint arXiv:1911.03310 (2019).","journal-title":"arXiv preprint arXiv:1911.03310"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.150"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.245"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2401.13303"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","unstructured":"Xi Victoria Lin Todor Mihaylov Mikel Artetxe Tianlu Wang Shuohui Chen Daniel Simig Myle Ott Naman Goyal Shruti Bhosale Jingfei Du Ramakanth Pasunuru Sam Shleifer Punit Singh Koura Vishrav Chaudhary Brian O\u2019Horo Jeff Wang Luke Zettlemoyer Zornitsa Kozareva Mona T. Diab Veselin Stoyanov and Xian Li. 2022. Few-shot learning with multilingual generative language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing EMNLP 2022 Abu Dhabi United Arab Emirates December 7-11 2022 Yoav Goldberg Zornitsa Kozareva and Yue Zhang (Eds.). Association for Computational Linguistics 9019\u20139052. 10.18653\/V1\/2022.EMNLP-MAIN.616","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.616"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","unstructured":"Yu-Hsiang Lin Chian-Yu Chen Jean Lee Zirui Li Yuyan Zhang Mengzhou Xia Shruti Rijhwani Junxian He Zhisong Zhang Xuezhe Ma Antonios Anastasopoulos Patrick Littell and Graham Neubig. 2019. Choosing transfer languages for cross-lingual learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics Florence Italy 3125\u20133135. 10.18653\/v1\/P19-1301","DOI":"10.18653\/v1\/P19-1301"},{"key":"e_1_3_2_93_2","article-title":"A study of cross-lingual ability and language-specific information in multilingual BERT","volume":"2004","author":"Liu Chi-Liang","year":"2020","unstructured":"Chi-Liang Liu, Tsung-Yuan Hsu, Yung-Sung Chuang, and Hung-yi Lee. 2020. A study of cross-lingual ability and language-specific information in multilingual BERT. CoRR abs\/2004.09205 (2020). arxiv:2004.09205https:\/\/arxiv.org\/abs\/2004.09205","journal-title":"CoRR"},{"key":"e_1_3_2_94_2","article-title":"Explainaboard: An explainable leaderboard for NLP","author":"Liu Pengfei","year":"2021","unstructured":"Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, and Graham Neubig. 2021. Explainaboard: An explainable leaderboard for NLP. arXiv preprint arXiv:2104.06387 (2021).","journal-title":"arXiv preprint arXiv:2104.06387"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K19-1004"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00343"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1162\/TACL_A_00343"},{"key":"e_1_3_2_98_2","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","volume":"1907","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. ArXiv abs\/1907.11692 (2019).","journal-title":"ArXiv"},{"key":"e_1_3_2_99_2","article-title":"Exploring fine-tuning techniques for pre-trained cross-lingual models via continual learning","volume":"2004","author":"Liu Zihan","year":"2020","unstructured":"Zihan Liu, Genta Indra Winata, Andrea Madotto, and Pascale Fung. 2020. Exploring fine-tuning techniques for pre-trained cross-lingual models via continual learning. CoRR abs\/2004.14218 (2020). arxiv:2004.14218https:\/\/arxiv.org\/abs\/2004.14218","journal-title":"CoRR"},{"key":"e_1_3_2_100_2","unstructured":"Fuli Luo Wei Wang Jiahao Liu Yijia Liu Bin Bi Songfang Huang Fei Huang and Luo Si. 2021. {VECO}: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation. https:\/\/openreview.net\/forum?id=YjNv-hzM8BE"},{"key":"e_1_3_2_101_2","unstructured":"Shuming Ma Jian Yang Haoyang Huang Zewen Chi Li Dong Dongdong Zhang Hany Hassan Awadalla Alexandre Muzio Akiko Eriguchi Saksham Singhal Xia Song Arul Menezes and Furu Wei. 2020. XLM-T: Scaling up multilingual machine translation with pretrained cross-lingual transformer encoders. CoRR abs\/2012.15547 (2020). arXiv:2012.15547 https:\/\/arxiv.org\/abs\/2012.15547"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.645"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.ACL-LONG.582"},{"key":"e_1_3_2_104_2","article-title":"Exploiting similarities among languages for machine translation","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013).","journal-title":"arXiv preprint arXiv:1309.4168"},{"key":"e_1_3_2_105_2","article-title":"Towards lingua Franca named entity recognition with BERT","volume":"1912","author":"Moon Taesun","year":"2019","unstructured":"Taesun Moon, Parul Awasthy, Jian Ni, and Radu Florian. 2019. Towards lingua Franca named entity recognition with BERT. CoRR abs\/1912.01389 (2019). arxiv:1912.01389http:\/\/arxiv.org\/abs\/1912.01389","journal-title":"CoRR"},{"key":"e_1_3_2_106_2","article-title":"When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models","volume":"2010","author":"M\u00fcller Benjamin","year":"2020","unstructured":"Benjamin M\u00fcller, Antonis Anastasopoulos, Beno\u00eet Sagot, and Djam\u00e9 Seddah. 2020. When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models. CoRR abs\/2010.12858 (2020). arxiv:2010.12858https:\/\/arxiv.org\/abs\/2010.12858","journal-title":"CoRR"},{"key":"e_1_3_2_107_2","unstructured":"Benjamin Muller Benoit Sagot and Djam\u00e9 Seddah. 2020. Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi. arxiv:2005.00318 [cs.CL]"},{"key":"e_1_3_2_108_2","article-title":"Trankit: A light-weight transformer-based Toolkit for multilingual natural language processing","volume":"2101","author":"Nguyen Minh","year":"2021","unstructured":"Minh Nguyen, Viet Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. Trankit: A light-weight transformer-based Toolkit for multilingual natural language processing. CoRR abs\/2101.03289 (2021). arxiv:2101.03289https:\/\/arxiv.org\/abs\/2101.03289","journal-title":"CoRR"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2312.11011"},{"key":"e_1_3_2_110_2","first-page":"296","volume-title":"Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Nguyen Toan Q.","year":"2017","unstructured":"Toan Q. Nguyen and David Chiang. 2017. Transfer learning across low-resource, related languages for neural machine translation. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Asian Federation of Natural Language Processing, Taipei, Taiwan, 296\u2013301. https:\/\/www.aclweb.org\/anthology\/I17-2050"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","unstructured":"Xuan-Phi Nguyen Wenxuan Zhang Xin Li Mahani Aljunied Qingyu Tan Liying Cheng Guanzheng Chen Yue Deng Sen Yang Chaoqun Liu Hang Zhang and Lidong Bing. 2023. SeaLLMs - Large language models for southeast asia. CoRR abs\/2312.00738 (2023). 10.48550\/ARXIV.2312.00738arXiv:2312.00738","DOI":"10.48550\/ARXIV.2312.00738"},{"key":"e_1_3_2_112_2","unstructured":"Joakim Nivre Mitchell Abrams \u017deljko Agi\u0107 Lars Ahrenberg Lene Antonsen Katya Aplonova Maria Jesus Aranzabe Gashaw Arutie Masayuki Asahara Luma Ateyah Mohammed Attia and et. al. 2018. Universal Dependencies 2.3. http:\/\/hdl.handle.net\/11234\/1-2895 LINDAT\/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (\u00daFAL) Faculty of Mathematics and Physics Charles University."},{"key":"e_1_3_2_113_2","unstructured":"Joakim Nivre Rogier Blokland Niko Partanen and Michael Rie\u00dfler. 2018. Universal Dependencies 2.2."},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1162\/089120103321337421"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","unstructured":"Odunayo Ogundepo Tajuddeen R. Gwadabe Clara E. Rivera Jonathan H. Clark Sebastian Ruder David Ifeoluwa Adelani Bonaventure F. P. Dossou Abdou Aziz Diop Claytone Sikasote Gilles Hacheme Happy Buzaaba Ignatius Ezeani Rooweither Mabuya Salomey Osei Chris Emezue Albert Njoroge Kahira Shamsuddeen Hassan Muhammad Akintunde Oladipo Abraham Toluwase Owodunni Atnafu Lambebo Tonja Iyanuoluwa Shode Akari Asai Tunde Oluwaseyi Ajayi Clemencia Siro Steven Arthur Mofetoluwa Adeyemi Orevaoghene Ahia Aremu Anuoluwapo Oyinkansola Awosan Chiamaka Chukwuneke Bernard Opoku Awokoya Ayodele Verrah Otiende Christine Mwase Boyd Sinkala Andre Niyongabo Rubungo Daniel A. Ajisafe Emeka Felix Onwuegbuzia Habib Mbow Emile Niyomutabazi Eunice Mukonde Falalu Ibrahim Lawan Ibrahim Said Ahmad Jesujoba O. Alabi Martin Namukombo Chinedu Emmanuel Mbonu Mofya Phiri Neo Putini Ndumiso Mngoma Priscilla A. Amuok Ruqayya Nasir Iro and Sonia Adhiambo. 2023. AfriQA: Cross-lingual open-retrieval question answering for african languages. CoRR abs\/2305.06897 (2023). 10.48550\/ARXIV.2305.06897arXiv:2305.06897","DOI":"10.48550\/ARXIV.2305.06897"},{"issue":"1","key":"e_1_3_2_116_2","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1515\/pralin-2016-0013","article-title":"Efficient word alignment with Markov chain Monte Carlo","volume":"106","author":"\u00d6stling Robert","year":"2016","unstructured":"Robert \u00d6stling and J\u00f6rg Tiedemann. 2016. Efficient word alignment with Markov chain Monte Carlo. The Prague Bulletin of Mathematical Linguistics 106, 1 (2016), 125\u2013146.","journal-title":"The Prague Bulletin of Mathematical Linguistics"},{"key":"e_1_3_2_117_2","doi-asserted-by":"crossref","unstructured":"Xuan Ouyang Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu and Haifeng Wang. 2021. ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora. arxiv:2012.15674 [cs.CL]","DOI":"10.18653\/v1\/2021.emnlp-main.3"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000073"},{"key":"e_1_3_2_119_2","article-title":"AdapterHub: A framework for adapting transformers","volume":"2007","author":"Pfeiffer Jonas","year":"2020","unstructured":"Jonas Pfeiffer, Andreas R\u00fcckl\u00e9, Clifton Poth, Aishwarya Kamath, Ivan Vulic, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. AdapterHub: A framework for adapting transformers. CoRR abs\/2007.07779 (2020). arxiv:2007.07779https:\/\/arxiv.org\/abs\/2007.07779","journal-title":"CoRR"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.617"},{"key":"e_1_3_2_121_2","article-title":"UNKs everywhere: Adapting multilingual language models to new scripts","volume":"2012","author":"Pfeiffer Jonas","year":"2020","unstructured":"Jonas Pfeiffer, Ivan Vulic, Iryna Gurevych, and Sebastian Ruder. 2020. UNKs everywhere: Adapting multilingual language models to new scripts. CoRR abs\/2012.15562 (2020). arxiv:2012.15562https:\/\/arxiv.org\/abs\/2012.15562","journal-title":"CoRR"},{"key":"e_1_3_2_122_2","first-page":"557","volume-title":"Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL\/IJCNLP 2020) (Suzhou, China, December 4\u20137, 2020)","author":"Phang Jason","year":"2020","unstructured":"Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020. English intermediate-task training improves zero-shot cross-lingual transfer too. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL\/IJCNLP 2020) (Suzhou, China, December 4\u20137, 2020), Kam-Fai Wong, Kevin Knight, and Hua Wu (Eds.). Association for Computational Linguistics, 557\u2013575. https:\/\/www.aclweb.org\/anthology\/2020.aacl-main.56\/"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p19-1493"},{"key":"e_1_3_2_124_2","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201920)","author":"Ponti Edoardo M.","year":"2020","unstructured":"Edoardo M. Ponti, Goran Glava\u0161, Olga Majewska, Qianchu Liu, Ivan Vuli\u0107, and Anna Korhonen. 2020. XCOPA: A multilingual dataset for causal commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201920). https:\/\/ducdauge.github.io\/files\/xcopa.pdf"},{"key":"e_1_3_2_125_2","article-title":"Pre-trained models for natural language processing: A survey","volume":"2003","author":"Qiu Xipeng","year":"2020","unstructured":"Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. ArXiv abs\/2003.08271 (2020).","journal-title":"ArXiv"},{"key":"e_1_3_2_126_2","first-page":"140:1\u2013140:67","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21 (2020), 140:1\u2013140:67. http:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_127_2","first-page":"2475","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Ralethe Sello","year":"2020","unstructured":"Sello Ralethe. 2020. Adaptation of deep bidirectional transformers for Afrikaans language. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, Marseille, France, 2475\u20132478. https:\/\/www.aclweb.org\/anthology\/2020.lrec-1.301"},{"key":"e_1_3_2_128_2","volume-title":"Advances in Neural Information Processing Systems","author":"Rebuffi Sylvestre-Alvise","year":"2017","unstructured":"Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning multiple visual domains with residual adapters. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc.https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/e7b24b112a44fdd9ee93bdf998c6ca0e-Paper.pdf"},{"key":"e_1_3_2_129_2","unstructured":"Youngbin Ro Yukyung Lee and Pilsung Kang. 2020. Multi2OIE: Multilingual Open Information Extraction based on Multi-Head Attention with BERT. arxiv:2009.08128 [cs.CL]"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00349"},{"key":"e_1_3_2_131_2","first-page":"29","volume-title":"Proceedings of the 1st NLPL Workshop on Deep Learning for Natural Language Processing","author":"R\u00f6nnqvist Samuel","year":"2019","unstructured":"Samuel R\u00f6nnqvist, Jenna Kanerva, Tapio Salakoski, and Filip Ginter. 2019. Is multilingual BERT fluent in language generation?. In Proceedings of the 1st NLPL Workshop on Deep Learning for Natural Language Processing. Link\u00f6ping University Electronic Press, Turku, Finland, 29\u201336. https:\/\/www.aclweb.org\/anthology\/W19-6204"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.477"},{"key":"e_1_3_2_133_2","doi-asserted-by":"crossref","unstructured":"Sebastian Ruder Jonathan H. Clark Alexander Gutkin Mihir Kale Min Ma Massimo Nicosia Shruti Rijhwani Parker Riley Jean Michel A. Sarr Xinyi Wang John Wieting Nitish Gupta Anna Katanova Christo Kirov Dana L. Dickinson Brian Roark Bidisha Samanta Connie Tao David Ifeoluwa Adelani Vera Axelrod Isaac Caswell Colin Cherry Dan Garrette R. Reeve Ingle Melvin Johnson Dmitry Panteleev and Partha Talukdar. 2023. XTREME-UP: A user-centric scarce-data benchmark for under-represented languages. In Findings of the Association for Computational Linguistics: EMNLP 2023 Singapore December 6-10 2023 Houda Bouamor Juan Pino and Kalika Bali (Eds.). Association for Computational Linguistics 1856\u20131884. https:\/\/aclanthology.org\/2023.findings-emnlp.125","DOI":"10.18653\/v1\/2023.findings-emnlp.125"},{"key":"e_1_3_2_134_2","article-title":"XTREME-R: Towards more challenging and nuanced multilingual evaluation","volume":"2104","author":"Ruder Sebastian","year":"2021","unstructured":"Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Graham Neubig, and Melvin Johnson. 2021. XTREME-R: Towards more challenging and nuanced multilingual evaluation. CoRR abs\/2104.07412 (2021). arxiv:2104.07412https:\/\/arxiv.org\/abs\/2104.07412","journal-title":"CoRR"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11640"},{"key":"e_1_3_2_136_2","first-page":"3118","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, (ACL\/IJCNLP 2021), (Volume 1: Long Papers)(Virtual Event, August 1\u20136, 2021","author":"Rust Phillip","year":"2021","unstructured":"Phillip Rust, Jonas Pfeiffer, Ivan Vulic, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? On the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, (ACL\/IJCNLP 2021), (Volume 1: Long Papers)(Virtual Event, August 1\u20136, 2021), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 3118\u20133135. https:\/\/aclanthology.org\/2021.acl-long.243"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","unstructured":"Scao Teven Le Angela Fan Christopher Akiki Ellie Pavlick Suzana Ili\u2019c Daniel Hesslow Roman Castagn\u2019e Alexandra Sasha Luccioni Fran\u00e7ois Yvon Matthias Gall\u00e9 Jonathan Tow Alexander M. Rush Stella Biderman Albert Webson Pawan Sasanka Ammanamanchi Thomas Wang Beno\u00eet Sagot Niklas Muennighoff Albert Villanova del Moral Olatunji Ruwase Rachel Bawden Stas Bekman Angelina McMillan-Major Iz Beltagy Huu Nguyen Lucile Saulnier Samson Tan Pedro Ortiz Suarez Victor Sanh Hugo Laurenccon Yacine Jernite Julien Launay Margaret Mitchell Colin Raffel Aaron Gokaslan Adi Simhi Aitor Soroa Etxabe Alham Fikri Aji Amit Alfassy Anna Rogers Ariel Kreisberg Nitzav Canwen Xu Chenghao Mou Chris C. Emezue Christopher Klamm Colin Leong Daniel Alexander van Strien David Ifeoluwa Adelani Dragomir R. Radev Eduardo Gonz\u2019alez Ponferrada Efrat Levkovizh Ethan Kim Eyal Natan Francesco De Toni G\u00e9rard Dupont Germ\u00e1n Kruszewski Giada Pistilli Hady ElSahar Hamza Benyamina Hieu Trung Tran Ian Yu Idris Abdulmumin Isaac Johnson Itziar Gonzalez-Dios Javier de la Rosa Jenny Chim Jesse Dodge Jian Zhu Jonathan Chang Jorg Frohberg Josephine Tobing Joydeep Bhattacharjee Khalid Almubarak Kimbo Chen Kyle Lo Leandro von Werra Leon Weber Long Phan Loubna Ben Allal Ludovic Tanguy Manan Dey Manuel Romero Mu\u00f1oz Maraim Masoud Mar\u00eda Grandury Mario vSavsko Max Huang Maximin Coavoux Mayank Singh Mike Tian-Jian Jiang Minh Chien Vu Moham-mad A. Jauhar Mustafa Ghaleb Nishant Subramani Nora Kassner Nurulaqilla Khamis Olivier Nguyen Omar Espejel Ona de Gibert Paulo Villegas Peter Henderson Pierre Colombo Priscilla Amuok Quentin Lhoest Rheza Harliman Rishi Bommasani Roberto L\u2019opez Rui Ribeiro Salomey Osei Sampo Pyysalo Sebastian Nagel Shamik Bose Shamsuddeen Hassan Muhammad Shanya Sharma Sharma S. Longpre Somaieh Nikpoor S. Silberberg Suhas Pai Sydney Zink Tiago Timponi Torrent Timo Schick Tristan Thrush Valentin Danchev Vassilina Nikoulina Veronika Laippala Violette Lepercq Vrinda Prabhu Zaid Alyafeai Zeerak Talat Arun Raja Benjamin Heinzerling Chenglei Si Elizabeth Salesky Sabrina J. Mielke Wilson Y. Lee Abheesht Sharma Andrea Santilli Antoine Chaffin Arnaud Stiegler Debajyoti Datta Eliza Szczechla Gunjan Chhablani Han Wang Harshit Pandey Hendrik Strobelt Jason Alan Fries Jos Rozen Leo Gao Lintang Sutawika M. Saiful Bari Maged S. Al-Shaibani Matteo Manica Nihal V. Nayak Ryan Teehan Samuel Albanie Sheng Shen Srulik Ben-David Stephen H. Bach Taewoon Kim Tali Bers Thibault F\u00e9vry Trishala Neeraj Urmish Thakker Vikas Raunak Xiang Tang Zheng-Xin Yong Zhiqing Sun Shaked Brody Y. Uri Hadar Tojarieh Adam Roberts Hyung Won Chung Jaesung Tae Jason Phang Ofir Press Conglong Li Deepak Narayanan Hatim Bourfoune Jared Casper Jeff Rasley Max Ryabinin Mayank Mishra Minjia Zhang Mohammad Shoeybi Myriam Peyrounette Nicolas Patry Nouamane Tazi Omar Sanseviero Patrick von Platen Pierre Cornette Pierre Franccois Lavall\u2019ee R\u00e9mi Lacroix Samyam Rajbhandari Sanchit Gandhi Shaden Smith St\u00e9phane Requena Suraj Patil Tim Dettmers Ahmed Baruwa Amanpreet Singh Anastasia Cheveleva Anne-Laure Ligozat Arjun Subramonian Aur\u2019elie N\u2019ev\u2019eol Charles Lovering Daniel H. Garrette Deepak R. Tunuguntla Ehud Reiter Ekaterina Taktasheva Ekaterina Voloshina Eli Bogdanov Genta Indra Winata Hailey Schoelkopf Jan-Christoph Kalo Jekaterina Novikova Jessica Zosa Forde Xiangru Tang Jungo Kasai Ken Kawamura Liam Hazan Marine Carpuat Miruna Clinciu Najoung Kim Newton Cheng Oleg Serikov Omer Antverg Oskar van der Wal Rui Zhang Ruochen Zhang Sebastian Gehrmann Shachar Mirkin S. Osher Pais Tatiana Shavrina Thomas Scialom Tian Yun Tomasz Limisiewicz Verena Rieser Vitaly Protasov Vladislav Mikhailov Yada Pruksachatkun Yonatan Belinkov Zachary Bamberger Zden\u011bk Kasner Zden\u011bk Kasner Amanda Pestana Amir Feizpour Ammar Khan Amy Faranak Ananda Santa Rosa Santos Anthony Hevia Antigona Unldreaj Arash Aghagol Arezoo Abdollahi Aycha Tammour Azadeh HajiHosseini Bahareh Behroozi Benjamin Ayoade Ajibade Bharat Kumar Saxena Carlos Mu\u00f1oz Ferrandis Danish Contractor David M. Lansky Davis David Douwe Kiela Duong Anh Nguyen Edward Tan Emi Baylor Ezinwanne Ozoani Fatim Tahirah Mirza Frankline Ononiwu Habib Rezanejad H. A. Jones Indrani Bhattacharya Irene Solaiman Irina Sedenko Isar Nejadgholi Jan Passmore Joshua Seltzer Julio Bonis Sanz Karen Fort L\u00edvia Dutra Mairon Samagaio Maraim Elbadri Margot Mieskes Marissa Gerchick Martha Akinlolu Michael McKenna Mike Qiu Muhammed Ghauri Mykola Burynok Nafis Abrar Nazneen Rajani Nour Elkott Nourhan Fahmy Olanrewaju Samuel Ran An R. P. Kromann Ryan Hao Samira Alizadeh Sarmad Shubber Silas L. Wang Sourav Roy Sylvain Viguier Thanh-Cong Le Tobi Oyebade Trieu Nguyen Hai Le Yoyo Yang Zach Nguyen Abhinav Ramesh Kashyap Alfredo Palasciano Alison Callahan Anima Shukla Antonio Miranda-Escalada Ayush Kumar Singh Benjamin Beilharz Bo Wang Caio Matheus Fonseca de Brito Chenxi Zhou Chirag Jain Chuxin Xu Cl\u00e9mentine Fourrier Daniel Le\u2019on Perin\u2019an Daniel Molano Dian Yu Enrique Manjavacas Fabio Barth Florian Fuhrimann Gabriel Altay Giyaseddin Bayrak Gully Burns Helena U. Vrabec Iman I. B. Bello Isha Dash Ji Soo Kang John Giorgi Jonas Golde Jos\u00e9 D. Posada Karthi Sivaraman Lokesh Bulchandani Lu Liu Luisa Shinzato Madeleine Hahn de Bykhovetz Maiko Takeuchi Marc P\u00e0mies Mar\u00eda Andrea Castillo Marianna Nezhurina Mario Sanger Matthias Samwald Michael Cullan Michael Weinberg M. Wolf Mina Mihaljcic Minna Liu Moritz Freidank Myungsun Kang Natasha Seelam Nathan Dahlberg Nicholas Michio Broad Nikolaus Muellner Pascale Fung Patricia Haller Patrick Haller Renata Eisenberg Robert Martin Rodrigo Canalli Rosaline Su Ruisi Su Samuel Cahyawijaya Samuele Garda Shlok S. Deshmukh Shubhanshu Mishra Sid Kiblawi Simon Ott Sinee Sang-aroonsiri Srishti Kumar Stefan Schweter Sushil Pratap Bharati Tanmay Laud Th\u00e9o Gigant Tomoya Kainuma Wojciech Kusa Yanis Labrak Yashasvi Bajaj Y. Venkatraman Yifan Xu Ying Xu Yu Xu Zhee Xao Tan Zhongli Xie Zifan Ye Mathilde Bras Younes Belkada and Thomas Wolf. 2022. BLOOM: A 176B-Parameter Open-Access multilingual language model. CoRR abs\/2211.05100 (2022). 10.48550\/ARXIV.2211.05100arxiv:2211.05100","DOI":"10.48550\/ARXIV.2211.05100"},{"key":"e_1_3_2_138_2","doi-asserted-by":"crossref","first-page":"5149","DOI":"10.1109\/ICASSP.2012.6289079","article-title":"Japanese and Korean voice search","author":"Schuster M.","year":"2012","unstructured":"M. Schuster and Kaisuke Nakajima. 2012. Japanese and Korean voice search. Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201912) (2012), 5149\u20135152.","journal-title":"Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201912)"},{"key":"e_1_3_2_139_2","volume-title":"Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018)","author":"Schwenk Holger","year":"2018","unstructured":"Holger Schwenk and Xian Li. 2018. A corpus for multilingual document classification in eight languages. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018) (Miyazaki, Japan, 7\u201312), Nicoletta Calzolari (Conference chair), Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Koiti Hasida, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, H\u00e9l\u00e8ne Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis, and Takenobu Tokunaga (Eds.). European Language Resources Association (ELRA), Paris, France."},{"key":"e_1_3_2_140_2","doi-asserted-by":"crossref","unstructured":"Rico Sennrich Barry Haddow and Alexandra Birch. 2016. Improving Neural Machine Translation Models with Monolingual Data. arxiv:1511.06709 [cs.CL]","DOI":"10.18653\/v1\/P16-1009"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2204.07580"},{"key":"e_1_3_2_143_2","article-title":"SEA-LION (Southeast Asian Languages In One Network): A Family of Large Language Models for Southeast Asia","author":"Singapore AI","year":"2023","unstructured":"AI Singapore. 2023. SEA-LION (Southeast Asian Languages In One Network): A Family of Large Language Models for Southeast Asia. https:\/\/github.com\/aisingapore\/sealion.","journal-title":"https:\/\/github.com\/aisingapore\/sealion"},{"key":"e_1_3_2_144_2","doi-asserted-by":"crossref","unstructured":"Harman Singh Nitish Gupta Shikhar Bharadwaj Dinesh Tewari and Partha Talukdar. 2024. IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages. arxiv:2404.16816 [cs.CL]","DOI":"10.18653\/v1\/2024.acl-long.595"},{"key":"e_1_3_2_145_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.18653\/v1\/D19-6106","volume-title":"Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo\u201919)","author":"Singh Jasdeep","year":"2019","unstructured":"Jasdeep Singh, Bryan McCann, Richard Socher, and Caiming Xiong. 2019. BERT is not an interlingua and the bias of tokenization. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo\u201919). 47\u201355."},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D13-1170"},{"key":"e_1_3_2_147_2","series-title":"Proceedings of Machine Learning Research","first-page":"5926","volume-title":"Proceedings of the 36th International Conference on Machine Learning (ICML 2019) (9\u201315 June 2019, Long Beach, CA)","volume":"97","author":"Song Kaitao","year":"2019","unstructured":"Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. MASS: Masked sequence to sequence pre-training for language generation. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019) (9\u201315 June 2019, Long Beach, CA)(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 5926\u20135936. http:\/\/proceedings.mlr.press\/v97\/song19d.html"},{"key":"e_1_3_2_148_2","article-title":"Efficient transformers: A survey","author":"Tay Yi","year":"2020","unstructured":"Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020. Efficient transformers: A survey. arXiv preprint arXiv:2009.06732 (2020).","journal-title":"arXiv preprint arXiv:2009.06732"},{"key":"e_1_3_2_149_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR 2023) (Kigali, Rwanda, May 1\u20135, 2023","author":"Tay Yi","year":"2023","unstructured":"Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. 2023. UL2: Unifying language learning paradigms. In Proceedings of the 11th International Conference on Learning Representations (ICLR 2023) (Kigali, Rwanda, May 1\u20135, 2023). OpenReview.net. https:\/\/openreview.net\/pdf?id=6ruVLB727MC"},{"key":"e_1_3_2_150_2","unstructured":"Gemma Team Thomas Mesnard Cassidy Hardin Robert Dadashi Surya Bhupatiraju Shreya Pathak Laurent Sifre Morgane Rivi\u00e8re Mihir Sanjay Kale Juliette Love Pouya Tafti L\u00e9onard Hussenot Aakanksha Chowdhery Adam Roberts Aditya Barua Alex Botev Alex Castro-Ros Ambrose Slone Am\u00e9lie H\u00e9liou Andrea Tacchetti Anna Bulanova Antonia Paterson Beth Tsai Bobak Shahriari Charline Le Lan Christopher A. Choquette-Choo Cl\u00e9ment Crepy Daniel Cer Daphne Ippolito David Reid Elena Buchatskaya Eric Ni Eric Noland Geng Yan George Tucker George-Christian Muraru Grigory Rozhdestvenskiy Henryk Michalewski Ian Tenney Ivan Grishchenko Jacob Austin James Keeling Jane Labanowski Jean-Baptiste Lespiau Jeff Stanway Jenny Brennan Jeremy Chen Johan Ferret Justin Chiu Justin Mao-Jones Katherine Lee Kathy Yu Katie Millican Lars Lowe Sjoesund Lisa Lee Lucas Dixon Machel Reid Maciej Miku\u0142a Mateo Wirth Michael Sharman Nikolai Chinaev Nithum Thain Olivier Bachem Oscar Chang Oscar Wahltinez Paige Bailey Paul Michel Petko Yotov Pier Giuseppe Sessa Rahma Chaabouni Ramona Comanescu Reena Jana Rohan Anil Ross McIlroy Ruibo Liu Ryan Mullins Samuel L. Smith Sebastian Borgeaud Sertan Girgin Sholto Douglas Shree Pandya Siamak Shakeri Soham De Ted Klimenko Tom Hennigan Vlad Feinberg Wojciech Stokowiec Yu hui Chen Zafarali Ahmed Zhitao Gong Tris Warkentin Ludovic Peran Minh Giang Cl\u00e9ment Farabet Oriol Vinyals Jeff Dean Koray Kavukcuoglu Demis Hassabis Zoubin Ghahramani Douglas Eck Joelle Barral Fernando Pereira Eli Collins Armand Joulin Noah Fiedel Evan Senter Alek Andreev and Kathleen Kenealy. 2024. Gemma: Open models based on gemini research and technology. arXiv:2403.08295 [cs.CL]"},{"key":"e_1_3_2_151_2","volume-title":"COLING-02: Proceedings of the 6th Conference on Natural Language Learning 2002 (CoNLL\u201902)","author":"Sang Erik F. Tjong Kim","year":"2002","unstructured":"Erik F. Tjong Kim Sang. 2002. Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition. In COLING-02: Proceedings of the 6th Conference on Natural Language Learning 2002 (CoNLL\u201902). https:\/\/www.aclweb.org\/anthology\/W02-2024"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.3115\/1119176.1119195"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar Aur\u00e9lien Rodriguez Armand Joulin Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models. CoRR abs\/2302.13971 (2023). 10.48550\/ARXIV.2302.13971arxiv:2302.13971","DOI":"10.48550\/ARXIV.2302.13971"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton-Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aur\u00e9lien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. CoRR abs\/2307.09288 (2023). 10.48550\/ARXIV.2307.09288arxiv:2307.09288","DOI":"10.48550\/ARXIV.2307.09288"},{"key":"e_1_3_2_155_2","article-title":"Revisiting the primacy of English in zero-shot cross-lingual transfer","volume":"2106","author":"Turc Iulia","year":"2021","unstructured":"Iulia Turc, Kenton Lee, Jacob Eisenstein, Ming-Wei Chang, and Kristina Toutanova. 2021. Revisiting the primacy of English in zero-shot cross-lingual transfer. CoRR abs\/2106.16171 (2021). arxiv:2106.16171https:\/\/arxiv.org\/abs\/2106.16171","journal-title":"CoRR"},{"key":"e_1_3_2_156_2","article-title":"UDapter: Language adaptation for truly universal dependency parsing","volume":"2004","author":"\u00dcst\u00fcn Ahmet","year":"2020","unstructured":"Ahmet \u00dcst\u00fcn, Arianna Bisazza, Gosse Bouma, and Gertjan van Noord. 2020. UDapter: Language adaptation for truly universal dependency parsing. CoRR abs\/2004.14327 (2020). arxiv:2004.14327https:\/\/arxiv.org\/abs\/2004.14327","journal-title":"CoRR"},{"key":"e_1_3_2_157_2","unstructured":"Aaron van den Oord Yazhe Li and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding. arxiv:1807.03748 [cs.LG]"},{"key":"e_1_3_2_158_2","article-title":"Attention is all you need","volume":"1706","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. CoRR abs\/1706.03762 (2017). arxiv:1706.03762http:\/\/arxiv.org\/abs\/1706.03762","journal-title":"CoRR"},{"key":"e_1_3_2_159_2","article-title":"Orthogonal language and task adapters in zero-shot cross-lingual transfer","volume":"2012","author":"Vidoni Marko","year":"2020","unstructured":"Marko Vidoni, Ivan Vulic, and Goran Glavas. 2020. Orthogonal language and task adapters in zero-shot cross-lingual transfer. CoRR abs\/2012.06460 (2020). arxiv:2012.06460https:\/\/arxiv.org\/abs\/2012.06460","journal-title":"CoRR"},{"key":"e_1_3_2_160_2","article-title":"Multilingual is not enough: BERT for finnish","volume":"1912","author":"Virtanen Antti","year":"2019","unstructured":"Antti Virtanen, Jenna Kanerva, Rami Ilo, Jouni Luoma, Juhani Luotolahti, Tapio Salakoski, Filip Ginter, and Sampo Pyysalo. 2019. Multilingual is not enough: BERT for finnish. CoRR abs\/1912.07076 (2019). arxiv:1912.07076http:\/\/arxiv.org\/abs\/1912.07076","journal-title":"CoRR"},{"key":"e_1_3_2_161_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1580"},{"key":"e_1_3_2_162_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5446"},{"key":"e_1_3_2_163_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w18-5446"},{"key":"e_1_3_2_164_2","doi-asserted-by":"crossref","first-page":"1448","DOI":"10.18653\/v1\/2020.semeval-1.189","volume-title":"Proceedings of the 14th Workshop on Semantic Evaluation","author":"Wang Shuohuan","year":"2020","unstructured":"Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, and Yu Sun. 2020. Galileo at SemEval-2020 task 12: Multi-lingual learning for offensive language identification using pre-trained language models. In Proceedings of the 14th Workshop on Semantic Evaluation. International Committee for Computational Linguistics, Barcelona (online), 1448\u20131455. https:\/\/www.aclweb.org\/anthology\/2020.semeval-1.189"},{"key":"e_1_3_2_165_2","first-page":"3317","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020","author":"Wang Xinyu","year":"2020","unstructured":"Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Fei Huang, and Kewei Tu. 2020. Structure-level knowledge distillation for multilingual sequence labeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Online, July 5\u201310, 2020), Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 3317\u20133330. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.304\/"},{"key":"e_1_3_2_166_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1575"},{"key":"e_1_3_2_167_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.240"},{"key":"e_1_3_2_168_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (Addis Ababa, Ethiopia, April 26\u201330, 2020","author":"Wang Zirui","year":"2020","unstructured":"Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, and Jaime G. Carbonell. 2020. Cross-lingual alignment vs joint training: A comparative study and a simple unified framework. In Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) (Addis Ababa, Ethiopia, April 26\u201330, 2020) OpenReview.net. https:\/\/openreview.net\/forum?id=S1l-C0NtwS"},{"key":"e_1_3_2_169_2","doi-asserted-by":"publisher","unstructured":"Xiangpeng Wei Haoran Wei Huan Lin Tianhao Li Pei Zhang Xingzhang Ren Mei Li Yu Wan Zhiwei Cao Binbin Xie Tianxiang Hu Shangjie Li Binyuan Hui Bowen Yu Dayiheng Liu Baosong Yang Fei Huang and Jun Xie. 2023. PolyLM: An open source polyglot large language model. CoRR abs\/2307.06018 (2023). 10.48550\/ARXIV.2307.06018arxiv:2307.06018","DOI":"10.48550\/ARXIV.2307.06018"},{"key":"e_1_3_2_170_2","volume-title":"9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\u20137, 2021","author":"Wei Xiangpeng","year":"2021","unstructured":"Xiangpeng Wei, Rongxiang Weng, Yue Hu, Luxi Xing, Heng Yu, and Weihua Luo. 2021. On learning universal representations across languages. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\u20137, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=Uu1Nw-eeTxJ"},{"key":"e_1_3_2_171_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1101"},{"key":"e_1_3_2_172_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1077"},{"key":"e_1_3_2_173_2","first-page":"120","volume-title":"Proceedings of the 5th Workshop on Representation Learning for NLP, RepL4NLP@ACL 2020, Online, July 9, 2020","author":"Wu Shijie","year":"2020","unstructured":"Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT?. In Proceedings of the 5th Workshop on Representation Learning for NLP, RepL4NLP@ACL 2020, Online, July 9, 2020, Spandana Gella, Johannes Welbl, Marek Rei, Fabio Petroni, Patrick S. H. Lewis, Emma Strubell, Min Joon Seo, and Hannaneh Hajishirzi (Eds.). Association for Computational Linguistics, 120\u2013130. https:\/\/www.aclweb.org\/anthology\/2020.repl4nlp-1.16\/"},{"key":"e_1_3_2_174_2","unstructured":"Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey Jeff Klingner Apurva Shah Melvin Johnson Xiaobing Liu Lukasz Kaiser Stephan Gouws Yoshikiyo Kato Taku Kudo Hideto Kazawa Keith Stevens George Kurian Nishant Patil Wei Wang Cliff Young Jason Smith Jason Riesa Alex Rudnick Oriol Vinyals Greg Corrado Macduff Hughes and Jeffrey Dean. 2016. Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. CoRR abs\/1609.08144 (2016). arXiv:1609.08144 http:\/\/arxiv.org\/abs\/1609.08144"},{"key":"e_1_3_2_175_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2404.00929"},{"key":"e_1_3_2_176_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2021.NAACL-MAIN.41"},{"key":"e_1_3_2_177_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6480"},{"key":"e_1_3_2_178_2","volume-title":"Proceedings of EMNLP","author":"Yang Yinfei","year":"2019","unstructured":"Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. In Proceedings of EMNLP."},{"key":"e_1_3_2_179_2","first-page":"1425","volume-title":"Proceedings of the 14th Workshop on Semantic Evaluation, (SemEval@COLING 2020) (Barcelona (Online), December 12\u201313, 2020","author":"Zampieri Marcos","year":"2020","unstructured":"Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and \u00c7agri \u00c7\u00f6ltekin. 2020. SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020). In Proceedings of the 14th Workshop on Semantic Evaluation, (SemEval@COLING 2020) (Barcelona (Online), December 12\u201313, 2020), Aur\u00e9lie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, and Ekaterina Shutova (Eds.). International Committee for Computational Linguistics, 1425\u20131447. https:\/\/www.aclweb.org\/anthology\/2020.semeval-1.188\/"},{"key":"e_1_3_2_180_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.146"},{"key":"e_1_3_2_181_2","series-title":"Proceedings of Machine Learning Research","first-page":"11328","volume-title":"Proceedings of the 37th International Conference on Machine Learning (ICML 2020) (13\u201318 July 2020, Virtual Event)","volume":"119","author":"Zhang Jingqing","year":"2020","unstructured":"Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020) (13\u201318 July 2020, Virtual Event)(Proceedings of Machine Learning Research, Vol. 119). PMLR, 11328\u201311339. http:\/\/proceedings.mlr.press\/v119\/zhang20ae.html"},{"key":"e_1_3_2_182_2","article-title":"Inducing language-agnostic multilingual representations","volume":"2008","author":"Zhao Wei","year":"2020","unstructured":"Wei Zhao, Steffen Eger, Johannes Bjerva, and Isabelle Augenstein. 2020. Inducing language-agnostic multilingual representations. CoRR abs\/2008.09112 (2020). arxiv:2008.09112https:\/\/arxiv.org\/abs\/2008.09112","journal-title":"CoRR"},{"key":"e_1_3_2_183_2","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201919)","author":"Zheng Wenzhao","year":"2019","unstructured":"Wenzhao Zheng, Zhaodong Chen, Jiwen Lu, and Jie Zhou. 2019. Hardness-aware deep metric learning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201919)."},{"key":"e_1_3_2_184_2","article-title":"Incorporating BERT into neural machine translation","volume":"2002","author":"Zhu Jinhua","year":"2020","unstructured":"Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. 2020. Incorporating BERT into neural machine translation. CoRR abs\/2002.06823 (2020). arxiv:2002.06823https:\/\/arxiv.org\/abs\/2002.06823","journal-title":"CoRR"},{"key":"e_1_3_2_185_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-2512"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3727339","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3727339","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:56:56Z","timestamp":1750298216000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3727339"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,6]]},"references-count":184,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3727339"],"URL":"https:\/\/doi.org\/10.1145\/3727339","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,6]]},"assertion":[{"value":"2021-12-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-06","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}