{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T18:41:10Z","timestamp":1772044870692,"version":"3.50.1"},"reference-count":53,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T00:00:00Z","timestamp":1703116800000},"content-version":"vor","delay-in-days":354,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,12,21]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The availability of large, high-quality datasets has been a major driver of recent progress in question answering (QA). Such annotated datasets, however, are difficult and costly to collect, and rarely exist in languages other than English, rendering QA technology inaccessible to underrepresented languages. An alternative to building large monolingual training datasets is to leverage pre-trained language models (PLMs) under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are fine-tuned, thus avoiding costly annotation. Prompt tuning the PLM with only five examples per language delivers accuracy superior to translation-based baselines; it bridges nearly 60% of the gap between an English-only baseline and a fully-supervised upper bound fine-tuned on almost 50,000 hand-labeled examples; and consistently leads to improvements compared to directly fine-tuning a QA model on labeled examples in low resource settings. Experiments on the TyDiqa-GoldP and MLQA benchmarks show that few-shot prompt tuning for data synthesis scales across languages and is a viable alternative to large-scale annotation.1<\/jats:p>","DOI":"10.1162\/tacl_a_00625","type":"journal-article","created":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T20:20:09Z","timestamp":1703190009000},"page":"1754-1771","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":4,"title":["<scp>QAmeleon<\/scp>: Multilingual QA with Only 5 Examples"],"prefix":"10.1162","volume":"11","author":[{"given":"Priyanka","family":"Agrawal","sequence":"first","affiliation":[{"name":"Google DeepMind, UK. priyankagr@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chris","family":"Alberti","sequence":"additional","affiliation":[{"name":"Google DeepMind, USA. chrisalberti@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fantine","family":"Huot","sequence":"additional","affiliation":[{"name":"Google DeepMind, The Netherlands. fantinehuot@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joshua","family":"Maynez","sequence":"additional","affiliation":[{"name":"Google DeepMind, UK. joshuahm@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ji","family":"Ma","sequence":"additional","affiliation":[{"name":"Google Research, UK. maji@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebastian","family":"Ruder","sequence":"additional","affiliation":[{"name":"Google DeepMind, Germany. ruder@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kuzman","family":"Ganchev","sequence":"additional","affiliation":[{"name":"Google DeepMind, USA. kuzman@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dipanjan","family":"Das","sequence":"additional","affiliation":[{"name":"Google DeepMind, USA. dipanjand@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mirella","family":"Lapata","sequence":"additional","affiliation":[{"name":"Google DeepMind, UK. lapata@google.com"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,12,21]]},"reference":[{"key":"2023122120195547100_bib1","doi-asserted-by":"publisher","first-page":"6168","DOI":"10.18653\/v1\/P19-1620","article-title":"Synthetic QA corpora generation with roundtrip consistency","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Alberti","year":"2019"},{"key":"2023122120195547100_bib2","doi-asserted-by":"publisher","first-page":"7674","DOI":"10.18653\/v1\/2020.emnlp-main.618","article-title":"Translation artifacts in cross-lingual transfer learning","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Artetxe","year":"2020"},{"key":"2023122120195547100_bib3","doi-asserted-by":"publisher","first-page":"4623","DOI":"10.18653\/v1\/2020.acl-main.421","article-title":"On the cross-lingual transferability of monolingual representations","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Artetxe","year":"2020"},{"key":"2023122120195547100_bib4","unstructured":"Giusepppe\n              Attardi\n            \n          . 2015. Wikiextractor. https:\/\/github.com\/attardi\/wikiextractor"},{"key":"2023122120195547100_bib5","article-title":"Ms marco: A human generated machine reading comprehension dataset","author":"Bajaj","year":"2016","journal-title":"arXiv preprint arXiv:1611.09268"},{"key":"2023122120195547100_bib6","doi-asserted-by":"publisher","first-page":"8830","DOI":"10.18653\/v1\/2021.emnlp-main.696","article-title":"Improving question answering model robustness with synthetic adversarial data generation","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Bartolo","year":"2021"},{"key":"2023122120195547100_bib7","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2023122120195547100_bib8","article-title":"Evaluating large language models trained on code","author":"Chen","year":"2021","journal-title":"arXiv preprint arXiv:2107.03374"},{"key":"2023122120195547100_bib9","doi-asserted-by":"publisher","first-page":"7570","DOI":"10.1609\/aaai.v34i05.6256","article-title":"Cross-lingual natural language generation via pre-training","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Chi","year":"2020"},{"key":"2023122120195547100_bib10","doi-asserted-by":"publisher","first-page":"6170","DOI":"10.18653\/v1\/2022.acl-long.427","article-title":"Xlm-e: Cross-lingual language model pre-training via electra","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chi","year":"2022"},{"key":"2023122120195547100_bib11","article-title":"Palm: Scaling language modeling with pathways","author":"Chowdhery","year":"2022","journal-title":"arXiv preprint arXiv:2204.02311"},{"key":"2023122120195547100_bib12","article-title":"Scaling instruction-finetuned language models","author":"Chung","year":"2022","journal-title":"arXiv preprint arXiv:2210.11416"},{"key":"2023122120195547100_bib13","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1162\/tacl_a_00317","article-title":"TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages","volume":"8","author":"Clark","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122120195547100_bib14","article-title":"Promptagator: Few-shot dense retrieval from 8 examples","author":"Dai","year":"2022","journal-title":"arXiv preprint arXiv:2209 .11755"},{"key":"2023122120195547100_bib15","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023122120195547100_bib16","first-page":"138","article-title":"Learning a part-of-speech tagger from two hours of annotation","volume-title":"Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Garrette","year":"2013"},{"key":"2023122120195547100_bib17","doi-asserted-by":"publisher","first-page":"1896","DOI":"10.18653\/v1\/2020.findings-emnlp.171","article-title":"UNIFIEDQA: Crossing format boundaries with a single QA system","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Khashabi","year":"2020"},{"key":"2023122120195547100_bib18","first-page":"1318","article-title":"Translationese and its dialects","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Koppel","year":"2011"},{"key":"2023122120195547100_bib19","first-page":"151","article-title":"Synthetic data generation for multilingual domain-adaptable question answering systems","volume-title":"Proceedings of the 23rd Annual Conference of the European Association for Machine Translation","author":"Kramchaninova","year":"2022"},{"key":"2023122120195547100_bib20","doi-asserted-by":"publisher","first-page":"4863","DOI":"10.18653\/v1\/P19-1481","article-title":"Cross-lingual training for automatic question generation","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Kumar","year":"2019"},{"key":"2023122120195547100_bib21","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1162\/tacl_a_00276","article-title":"Natural questions: A benchmark for question answering research","volume":"7","author":"Kwiatkowski","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122120195547100_bib22","article-title":"Semi-supervised training data generation for multilingual question answering","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Lee","year":"2018"},{"key":"2023122120195547100_bib23","doi-asserted-by":"publisher","first-page":"3045","DOI":"10.18653\/v1\/2021.emnlp-main.243","article-title":"The power of scale for parameter-efficient prompt tuning","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Lester","year":"2021"},{"key":"2023122120195547100_bib24","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2023122120195547100_bib25","doi-asserted-by":"publisher","first-page":"7315","DOI":"10.18653\/v1\/2020.acl-main.653","article-title":"MLQA: Evaluating cross-lingual extractive question answering","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2023122120195547100_bib26","doi-asserted-by":"publisher","first-page":"1098","DOI":"10.1162\/tacl_a_00415","article-title":"PAQ: 65 million probably-asked questions and what you can do with them","volume":"9","author":"Lewis","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122120195547100_bib27","article-title":"Few-shot learning with multilingual language models","volume-title":"Proceedings of EMNLP 2022","author":"Xi","year":"2022"},{"key":"2023122120195547100_bib28","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/P19-1301","article-title":"Choosing transfer languages for cross-lingual learning","volume-title":"Proceedings of ACL 2019","author":"Lin","year":"2019"},{"key":"2023122120195547100_bib29","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","article-title":"BLEU: A method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni","year":"2002"},{"issue":"140","key":"2023122120195547100_bib30","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2023122120195547100_bib31","doi-asserted-by":"publisher","first-page":"2383","DOI":"10.18653\/v1\/D16-1264","article-title":"SQuAD: 100,000+ questions for machine comprehension of text","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Rajpurkar","year":"2016"},{"key":"2023122120195547100_bib32","doi-asserted-by":"publisher","first-page":"7016","DOI":"10.18653\/v1\/2021.emnlp-main.562","article-title":"Synthetic data augmentation for zero-shot cross-lingual question answering","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Riabi","year":"2021"},{"key":"2023122120195547100_bib33","doi-asserted-by":"publisher","first-page":"10215","DOI":"10.18653\/v1\/2021.emnlp-main.802","article-title":"XTREME-R: Towards more challenging and nuanced multilingual evaluation","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Ruder","year":"2021"},{"key":"2023122120195547100_bib34","doi-asserted-by":"publisher","first-page":"2339","DOI":"10.18653\/v1\/2021.naacl-main.185","article-title":"It\u2019s not just size that matters: Small language models are also few-shot learners","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Schick","year":"2021"},{"key":"2023122120195547100_bib35","doi-asserted-by":"publisher","first-page":"7881","DOI":"10.18653\/v1\/2020.acl-main.704","article-title":"Bleurt: Learning robust metrics for text generation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sellam","year":"2020"},{"key":"2023122120195547100_bib36","doi-asserted-by":"crossref","first-page":"35","DOI":"10.18653\/v1\/2021.inlg-1.4","article-title":"Towards zero-shot multilingual synthetic question and answer generation for cross-lingual reading comprehension","volume-title":"Proceedings of the 14th International Conference on Natural Language Generation","author":"Shakeri","year":"2021"},{"key":"2023122120195547100_bib37","doi-asserted-by":"publisher","first-page":"5445","DOI":"10.18653\/v1\/2020.emnlp-main.439","article-title":"End-to-end synthetic data generation for domain adaptation of question answering systems","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Shakeri","year":"2020"},{"key":"2023122120195547100_bib38","first-page":"4596","article-title":"Adafactor: Adaptive learning rates with sublinear memory cost","volume-title":"International Conference on Machine Learning","author":"Shazeer","year":"2018"},{"key":"2023122120195547100_bib39","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00533","article-title":"Meta-learning a cross-lingual manifold for semantic parsing","author":"Sherborne","year":"2022"},{"key":"2023122120195547100_bib40","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2210.03057","article-title":"Language models are multilingual chain-of-thought reasoners","author":"Shi","year":"2022"},{"key":"2023122120195547100_bib41","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2201.08670","article-title":"Context-tuning: Learning contextualized prompts for natural language generation","author":"Tang","year":"2022"},{"key":"2023122120195547100_bib42","doi-asserted-by":"publisher","first-page":"191","DOI":"10.18653\/v1\/W17-2623","article-title":"NewsQA: A machine comprehension dataset","volume-title":"Proceedings of the 2nd Workshop on Representation Learning for NLP","author":"Trischler","year":"2017"},{"key":"2023122120195547100_bib43","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.630","article-title":"Overcoming catastrophic forgetting in zero-shot cross-lingual generation","author":"Tu","year":"2022"},{"key":"2023122120195547100_bib44","article-title":"Chain of thought prompting elicits reasoning in large language models","author":"Wei","year":"2022","journal-title":"arXiv preprint arXiv:2201.11903"},{"key":"2023122120195547100_bib45","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2021.mrl-1.1","article-title":"Language models are few-shot multilingual learners","author":"Winata","year":"2021","journal-title":"Proceedings of the 1st Workshop on Multilingual Representation Learning"},{"key":"2023122120195547100_bib46","article-title":"Google\u2019s neural machine translation system: Bridging the gap between human and machine translation","author":"Yonghui","year":"2016","journal-title":"arXiv preprint arXiv:1609.08144"},{"key":"2023122120195547100_bib47","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1162\/tacl_a_00461","article-title":"Byt5: Towards a token-free future with pre-trained byte-to-byte models","volume":"10","author":"Xue","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122120195547100_bib48","doi-asserted-by":"publisher","first-page":"483","DOI":"10.18653\/v1\/2021.naacl-main.41","article-title":"mt5: A massively multilingual pre-trained text-to-text transformer","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Xue","year":"2021"},{"key":"2023122120195547100_bib49","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.296","article-title":"Re3: Generating longer stories with recursive reprompting and revision","author":"Yang","year":"2022"},{"key":"2023122120195547100_bib50","doi-asserted-by":"publisher","first-page":"2369","DOI":"10.18653\/v1\/D18-1259","article-title":"HotpotQA: A dataset for diverse, explainable multi-hop question answering","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Yang","year":"2018"},{"key":"2023122120195547100_bib51","doi-asserted-by":"publisher","first-page":"731","DOI":"10.18653\/v1\/2022.acl-long.54","article-title":"It is AI\u2019s turn to ask humans a question: Question-answer pair generation for children\u2019s story books","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Yao","year":"2022"},{"key":"2023122120195547100_bib52","doi-asserted-by":"publisher","first-page":"8547","DOI":"10.18653\/v1\/2021.emnlp-main.672","article-title":"Discrete and soft prompting for multilingual models","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Zhao","year":"2021"},{"key":"2023122120195547100_bib53","doi-asserted-by":"publisher","first-page":"5751","DOI":"10.18653\/v1\/2021.acl-long.447","article-title":"A closer look at few-shot crosslingual transfer: The choice of shotsmatters","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Zhao","year":"2021"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00625\/2200667\/tacl_a_00625.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00625\/2200667\/tacl_a_00625.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T20:20:21Z","timestamp":1703190021000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00625\/118856\/QAmeleon-Multilingual-QA-with-Only-5-Examples"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":53,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00625","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}