{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T13:42:27Z","timestamp":1782913347231,"version":"3.54.5"},"reference-count":36,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T00:00:00Z","timestamp":1714089600000},"content-version":"vor","delay-in-days":116,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,4,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Large-scale pretrained language models (LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translation, without being explicitly trained on parallel corpora. It is intriguing how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7.5B, to perform multilingual translation following given instructions. Firstly, we show that multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language, the translation performance depends on its similarity to English and the amount of data used in the pretraining phase. Secondly, we find that LLMs\u2019 ability to carry out translation instructions relies on the understanding of translation instructions and the alignment among different languages. With multilingual finetuning with translation instructions, LLMs could learn to perform the translation task well even for those language pairs unseen during the instruction tuning phase.<\/jats:p>","DOI":"10.1162\/tacl_a_00655","type":"journal-article","created":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T18:14:19Z","timestamp":1714155259000},"page":"576-592","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":37,"title":["Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions"],"prefix":"10.1162","volume":"12","author":[{"given":"Jiahuan","family":"Li","sequence":"first","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, China lijh@smail.nju.edu.cn"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Zhou","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, China zhouh@smail.nju.edu.cn"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shujian","family":"Huang","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, China huangsj@nju.edu.cn"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shanbo","family":"Cheng","sequence":"additional","affiliation":[{"name":"Bytedance, China chengshanbo@bytedance.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiajun","family":"Chen","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, China chenjj@nju.edu.cn"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,4,30]]},"reference":[{"key":"2024042618140080400_bib1","doi-asserted-by":"publisher","first-page":"8857","DOI":"10.18653\/v1\/2023.findings-acl.564","article-title":"In-context examples selection for machine translation","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Agrawal","year":"2023"},{"key":"2024042618140080400_bib2","doi-asserted-by":"publisher","first-page":"1184","DOI":"10.18653\/v1\/N19-1121","article-title":"Consistency by agreement in zero-shot neural machine translation","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Al-Shedivat","year":"2019"},{"key":"2024042618140080400_bib3","article-title":"The missing ingredient in zero-shot neural machine translation","author":"Arivazhagan","year":"2019","journal-title":"CoRR"},{"key":"2024042618140080400_bib4","article-title":"Language models are few-shot learners","author":"Brown","year":"2020","journal-title":"CoRR"},{"key":"2024042618140080400_bib5","article-title":"Scaling instruction-finetuned language models","author":"Chung","year":"2022","journal-title":"CoRR"},{"key":"2024042618140080400_bib6","article-title":"No language left behind: Scaling human-centered machine translation","author":"Costa-juss\u00e0","year":"2022","journal-title":"CoRR"},{"key":"2024042618140080400_bib7","doi-asserted-by":"publisher","first-page":"5960","DOI":"10.18653\/v1\/2020.emnlp-main.480","article-title":"CCAligned: A massive collection of cross-lingual web-document pairs","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)","author":"El-Kishky","year":"2020"},{"key":"2024042618140080400_bib8","article-title":"Beyond English-centric multilingual machine translation","author":"Fan","year":"2020"},{"key":"2024042618140080400_bib9","article-title":"The unreasonable effectiveness of few-shot learning for machine translation","author":"Garcia","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib10","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1162\/tacl_a_00474","article-title":"The Flores-101 evaluation benchmark for low-resource and multilingual machine translation","volume":"10","author":"Goyal","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618140080400_bib11","doi-asserted-by":"publisher","first-page":"1258","DOI":"10.18653\/v1\/P19-1121","article-title":"Improved zero-shot neural machine translation via ignoring spurious correlations","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jiatao","year":"2019"},{"key":"2024042618140080400_bib12","article-title":"How good are GPT models at machine translation? A comprehensive evaluation","author":"Hendy","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib13","article-title":"LoRA: Low-rank adaptation of large language models","volume-title":"International Conference on Learning Representations","author":"Edward","year":"2022"},{"key":"2024042618140080400_bib14","article-title":"ParroT: Translating during chat using large language models","author":"Jiao","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib15","article-title":"Is ChatGPT a good translator? Yes with GPT-4 as the engine","author":"Jiao","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib16","article-title":"Scaling laws for neural language models","author":"Kaplan","year":"2020","journal-title":"CoRR"},{"key":"2024042618140080400_bib17","article-title":"Adam: A method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"CoRR"},{"key":"2024042618140080400_bib18","doi-asserted-by":"publisher","first-page":"9019","DOI":"10.18653\/v1\/2022.emnlp-main.616","article-title":"Few-shot learning with multilingual generative language models","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Xi","year":"2022"},{"key":"2024042618140080400_bib19","doi-asserted-by":"publisher","first-page":"8","DOI":"10.18653\/v1\/E17-2002","article-title":"URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors","volume-title":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers","author":"Littell","year":"2017"},{"key":"2024042618140080400_bib20","doi-asserted-by":"publisher","first-page":"1051","DOI":"10.18653\/v1\/2021.naacl-main.83","article-title":"Improving pretrained models for zero-shot multi-label text classification through reinforced label hierarchy reasoning","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Liu","year":"2021"},{"key":"2024042618140080400_bib21","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.891","article-title":"Crosslingual generalization through multitask finetuning","author":"Muennighoff","year":"2023"},{"key":"2024042618140080400_bib22","unstructured":"OpenAI. 2023. Chatgpt (mar 23 version) [large language model]."},{"key":"2024042618140080400_bib23","doi-asserted-by":"publisher","first-page":"210","DOI":"10.18653\/v1\/2021.naacl-main.20","article-title":"Multilingual BERT post-pretraining alignment","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Pan","year":"2021"},{"key":"2024042618140080400_bib24","doi-asserted-by":"publisher","first-page":"186","DOI":"10.18653\/v1\/W18-6319","article-title":"A call for clarity in reporting BLEU scores","volume-title":"Proceedings of the Third Conference on Machine Translation: Research Papers","author":"Post","year":"2018"},{"key":"2024042618140080400_bib25","first-page":"578","article-title":"COMET-22: Unbabel-IST 2022 submission for the metrics shared task","volume-title":"Proceedings of the Seventh Conference on Machine Translation (WMT)","author":"Rei","year":"2022"},{"key":"2024042618140080400_bib26","article-title":"BLOOM: A 176B-parameter open-access multilingual language model","author":"Scao","year":"2022","journal-title":"arXiv preprint arXiv:2211.05100v4"},{"key":"2024042618140080400_bib27","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.18653\/v1\/2021.eacl-main.115","article-title":"WikiMatrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Schwenk","year":"2021"},{"key":"2024042618140080400_bib28","article-title":"Stanford Alpaca: An instruction-following LLaMA model","author":"Taori","year":"2023"},{"key":"2024042618140080400_bib29","article-title":"LLaMA: Open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib30","doi-asserted-by":"publisher","first-page":"15406","DOI":"10.18653\/v1\/2023.acl-long.859","article-title":"Prompting PaLM for translation: Assessing strategies and performance","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Vilar","year":"2023"},{"key":"2024042618140080400_bib31","article-title":"Finetuned language models are zero-shot learners","volume-title":"International Conference on Learning Representations","author":"Wei","year":"2022"},{"key":"2024042618140080400_bib32","doi-asserted-by":"publisher","first-page":"233","DOI":"10.18653\/v1\/2021.acl-short.31","article-title":"Multilingual agreement for multilingual neural machine translation","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Yang","year":"2021"},{"key":"2024042618140080400_bib33","article-title":"Prompting large language model for machine translation: A case study","author":"Zhang","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib34","doi-asserted-by":"publisher","first-page":"1628","DOI":"10.18653\/v1\/2020.acl-main.148","article-title":"Improving massively multilingual neural machine translation and zero-shot translation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Zhang","year":"2020"},{"key":"2024042618140080400_bib35","article-title":"LIMA: Less is more for alignment","author":"Zhou","year":"2023","journal-title":"CoRR"},{"key":"2024042618140080400_bib36","article-title":"Multilingual machine translation with large language models: Empirical results and analysis","author":"Zhu","year":"2023","journal-title":"CoRR"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00655\/2367429\/tacl_a_00655.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00655\/2367429\/tacl_a_00655.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T18:14:28Z","timestamp":1714155268000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00655\/120833\/Eliciting-the-Translation-Ability-of-Large"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":36,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00655","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}