{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T01:35:55Z","timestamp":1769045755460,"version":"3.49.0"},"reference-count":60,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2021,4,6]],"date-time":"2021-04-06T00:00:00Z","timestamp":1617667200000},"content-version":"vor","delay-in-days":95,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,3,31]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We introduce an Edit-Based TransfOrmer with Repositioning (EDITOR), which makes sequence generation flexible by seamlessly allowing users to specify preferences in output lexical choice. Building on recent models for non-autoregressive sequence generation (Gu et al., 2019), EDITOR generates new sequences by iteratively editing hypotheses. It relies on a novel reposition operation designed to disentangle lexical choice from word positioning decisions, while enabling efficient oracles for imitation learning and parallel edits at decoding time. Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer (Gu et al., 2019) while speeding up decoding dramatically compared to constrained beam search (Post and Vilar, 2018). EDITOR also achieves comparable or better translation quality with faster decoding speed than the Levenshtein Transformer on standard Romanian-English, English-German, and English-Japanese machine translation tasks.<\/jats:p>","DOI":"10.1162\/tacl_a_00368","type":"journal-article","created":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:42:10Z","timestamp":1618447330000},"page":"311-328","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":20,"title":["EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints"],"prefix":"10.1162","volume":"9","author":[{"given":"Weijia","family":"Xu","sequence":"first","affiliation":[{"name":"University of Maryland, United States. weijia@cs.umd.edu"}]},{"given":"Marine","family":"Carpuat","sequence":"additional","affiliation":[{"name":"University of Maryland, United States. marine@cs.umd.edu"}]}],"member":"281","published-online":{"date-parts":[[2021,3,31]]},"reference":[{"key":"2021060823394161800_bib1","first-page":"187","article-title":"Generation of formal and informal sentences","volume-title":"Proceedings of the 13th European Workshop on Natural Language Generation","author":"Sheikha","year":"2011"},{"key":"2021060823394161800_bib2","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.18653\/v1\/D19-1166","article-title":"Controlling text complexity in neural machine translation","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Agrawal","year":"2019"},{"key":"2021060823394161800_bib3","first-page":"936","article-title":"Guided open vocabulary image captioning with constrained beam search","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Anderson","year":"2017"},{"key":"2021060823394161800_bib4","doi-asserted-by":"crossref","first-page":"1557","DOI":"10.18653\/v1\/D16-1162","article-title":"Incorporating discrete translation lexicons into neural machine translation","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Arthur","year":"2016"},{"key":"2021060823394161800_bib5","article-title":"Neural machine translation by jointly learning to align and translate","volume-title":"Proceedings of the 3th International Conference on Learning Representations","author":"Bahdanau","year":"2015"},{"key":"2021060823394161800_bib6","first-page":"152","article-title":"Statistical machine translation through global lexical selection and sentence reconstruction","volume-title":"Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics","author":"Bangalore","year":"2007"},{"issue":"1","key":"2021060823394161800_bib7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1162\/coli.2008.07-055-R2-06-29","article-title":"Statistical approaches to computer- assisted translation","volume":"35","author":"Barrachina","year":"2009","journal-title":"Computational Linguistics"},{"key":"2021060823394161800_bib8","doi-asserted-by":"crossref","first-page":"12","DOI":"10.3115\/v1\/W14-3302","article-title":"Findings of the 2014 workshop on statistical machine translation","volume-title":"Proceedings of the Ninth Workshop on Statistical Machine Translation","author":"Bojar","year":"2014"},{"key":"2021060823394161800_bib9","doi-asserted-by":"crossref","unstructured":"Ond\u0159ej\n              Bojar\n            , RajenChatterjee, ChristianFedermann, YvetteGraham, BarryHaddow, ShujianHuang, MatthiasHuck, PhilippKoehn, QunLiu, VarvaraLogacheva, ChristofMonz, MatteoNegri, MattPost, RaphaelRubino, LuciaSpecia, and MarcoTurchi. 2017. Findings of the 2017 conference on machine translation (WMT17). In Proceedings of the Second Conference on Machine Translation, pages 169\u2013214, Copenhagen, Denmark. Association for Computational Linguistics. DOI:\u00a0https:\/\/doi.org\/10.18653\/v1\/W17-4717","DOI":"10.18653\/v1\/W17-4717"},{"key":"2021060823394161800_bib10","doi-asserted-by":"crossref","first-page":"131","DOI":"10.18653\/v1\/W16-2301","article-title":"Findings of the 2016 conference on machine translation","volume-title":"Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers","author":"Bojar","year":"2016"},{"issue":"2","key":"2021060823394161800_bib11","first-page":"79","article-title":"A statistical approach to machine translation","volume":"16","author":"Brown","year":"1990","journal-title":"Computational Linguistics"},{"key":"2021060823394161800_bib12","first-page":"845","article-title":"Fast policy learning through imitation and reinforcement","volume-title":"Proceedings of the 2018 Conference on Uncertainty in Artificial Intelligence (UAI)","author":"Cheng","year":"2018"},{"key":"2021060823394161800_bib13","doi-asserted-by":"crossref","first-page":"103","DOI":"10.3115\/v1\/W14-4012","article-title":"On the properties of neural machine translation: Encoder\u2014decoder approaches","volume-title":"Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation","author":"Cho","year":"2014"},{"key":"2021060823394161800_bib14","first-page":"577","article-title":"Attention-based models for speech recognition","volume-title":"Advances in Neural Information Processing Systems","author":"Chorowski","year":"2015"},{"key":"2021060823394161800_bib15","first-page":"176","article-title":"Better hypothesis testing for statistical machine translation: Controlling for optimizer instability","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Clark","year":"2011"},{"issue":"3","key":"2021060823394161800_bib16","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s10994-009-5106-x","article-title":"Search-based structured prediction","volume":"75","author":"Hal Daum\u00e9","year":"2009","journal-title":"Machine Learning"},{"key":"2021060823394161800_bib17","doi-asserted-by":"crossref","first-page":"3063","DOI":"10.18653\/v1\/P19-1294","article-title":"Training neural machine translation to apply terminology constraints","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Dinu","year":"2019"},{"issue":"2","key":"2021060823394161800_bib18","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1162\/COLI_a_00218","article-title":"The operation sequence model\u2014Combining n-gram-based and phrase-based statistical machine translation","volume":"41","author":"Durrani","year":"2015","journal-title":"Computational Linguistics"},{"key":"2021060823394161800_bib19","doi-asserted-by":"crossref","first-page":"94","DOI":"10.18653\/v1\/W17-4912","article-title":"Controlling linguistic style aspects in neural language generation","volume-title":"Proceedings of the Workshop on Stylistic Variation","author":"Ficler","year":"2017"},{"key":"2021060823394161800_bib20","first-page":"148","article-title":"User-friendly text prediction for translators","volume-title":"Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)","author":"Foster","year":"2002"},{"key":"2021060823394161800_bib21","doi-asserted-by":"crossref","first-page":"6112","DOI":"10.18653\/v1\/D19-1633","article-title":"Mask-predict: Parallel decoding of conditional masked language models","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Ghazvininejad","year":"2019"},{"key":"2021060823394161800_bib22","article-title":"Non-autoregressive neural machine translation","volume-title":"International Conference on Learning Representations","author":"Jiatao","year":"2018"},{"key":"2021060823394161800_bib23","first-page":"11181","article-title":"Levenshtein transformer","volume-title":"Advances in Neural Information Processing Systems 32","author":"Jiatao","year":"2019"},{"key":"2021060823394161800_bib24","article-title":"Sockeye: A toolkit for neural machine translation","author":"Hieber","year":"2017","journal-title":"CoRR"},{"key":"2021060823394161800_bib25","doi-asserted-by":"crossref","first-page":"1535","DOI":"10.18653\/v1\/P17-1141","article-title":"Lexically constrained decoding for sequence generation using grid beam search","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Hokamp","year":"2017"},{"key":"2021060823394161800_bib26","first-page":"944","article-title":"Automatic evaluation of translation quality for distant language pairs","volume-title":"Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing","author":"Isozaki","year":"2010"},{"key":"2021060823394161800_bib27","article-title":"Deep encoder, shallow decoder: Reevaluating the speed-quality tradeoff in machine translation","author":"Kasai","year":"2020","journal-title":"arXiv preprint arXiv:2006.10369"},{"key":"2021060823394161800_bib28","article-title":"Adam: A method for stochastic optimization","volume-title":"Proceedings of the 3th International Conference on Learning Representations","author":"Kingma","year":"2015"},{"key":"2021060823394161800_bib29","first-page":"177","article-title":"Moses: Open source toolkit for statistical machine translation","volume-title":"Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions","author":"Koehn","year":"2007"},{"key":"2021060823394161800_bib30","doi-asserted-by":"crossref","first-page":"66","DOI":"10.18653\/v1\/D18-2012","article-title":"SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Kudo","year":"2018"},{"key":"2021060823394161800_bib31","article-title":"Unsupervised machine translation using monolingual corpora only","volume-title":"Proceedings of the 6th International Conference on Learning Representations","author":"Lample","year":"2018"},{"key":"2021060823394161800_bib32","article-title":"SEARNN: Training RNNs with global-local losses","volume-title":"International Conference on Learning Representations","author":"Leblond","year":"2018"},{"key":"2021060823394161800_bib33","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.18653\/v1\/D18-1149","article-title":"Deterministic non-autoregressive neural sequence modeling by iterative refinement","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Lee","year":"2018"},{"key":"2021060823394161800_bib34","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions, and reversals","volume-title":"Soviet Physics Doklady","author":"Levenshtein","year":"1966"},{"key":"2021060823394161800_bib35","doi-asserted-by":"crossref","first-page":"4282","DOI":"10.18653\/v1\/D19-1437","article-title":"FlowSeq: Non-autoregressive conditional sequence generation with generative flow","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Ma","year":"2019"},{"key":"2021060823394161800_bib36","first-page":"720","article-title":"What to talk about and how? selective generation using LSTMs with coarse-to-fine alignment","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Mei","year":"2016"},{"key":"2021060823394161800_bib37","first-page":"1","article-title":"Overview of the 4th workshop on Asian translation","volume-title":"Proceedings of the 4th Workshop on Asian Translation (WAT2017)","author":"Nakazawa","year":"2017"},{"key":"2021060823394161800_bib38","first-page":"35","article-title":"compare-mt: A tool for holistic comparison of language generation systems","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)","author":"Neubig","year":"2019"},{"key":"2021060823394161800_bib39","first-page":"334","article-title":"Improving lexical choice in neural machine translation","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Nguyen","year":"2018"},{"key":"2021060823394161800_bib40","first-page":"3918","article-title":"Parallel WaveNet: Fast high-fidelity speech synthesis","volume-title":"Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research","author":"van den Oord","year":"2018"},{"key":"2021060823394161800_bib41","first-page":"48","article-title":"Fairseq: A fast, extensible toolkit for sequence modeling","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)","author":"Ott","year":"2019"},{"key":"2021060823394161800_bib42","first-page":"311","article-title":"BLEU: A method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni","year":"2002"},{"key":"2021060823394161800_bib43","first-page":"1314","article-title":"Fast lexically constrained decoding with dynamic beam allocation for neural machine translation","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Post","year":"2018"},{"key":"2021060823394161800_bib44","first-page":"157","article-title":"Using the output embedding to improve language models","volume-title":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Computational","author":"Press","year":"2017"},{"key":"2021060823394161800_bib45","article-title":"Reinforcement and imitation learning via interactive no-regret learning","author":"Ross","year":"2014","journal-title":"CoRR"},{"key":"2021060823394161800_bib46","first-page":"627","article-title":"A reduction of imitation learning and structured prediction to no-regret online learning","volume-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics","author":"Ross","year":"2011"},{"key":"2021060823394161800_bib47","first-page":"35","article-title":"Controlling politeness in neural machine translation via side constraints","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Sennrich","year":"2016"},{"key":"2021060823394161800_bib48","first-page":"1715","article-title":"Neural machine translation of rare words with subword units","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics","author":"Sennrich","year":"2016"},{"key":"2021060823394161800_bib49","first-page":"449","article-title":"Code-switching for enhancing NMT with pre-specified translation","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Song","year":"2019"},{"key":"2021060823394161800_bib50","doi-asserted-by":"crossref","first-page":"175","DOI":"10.18653\/v1\/W18-5420","article-title":"An operation sequence model for explainable neural machine translation","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Stahlberg","year":"2018"},{"key":"2021060823394161800_bib51","first-page":"5976","article-title":"Insertion transformer: Flexible sequence generation via insertion operations","volume-title":"Proceedings of the 36th International Conference on Machine Learning","author":"Stern","year":"2019"},{"key":"2021060823394161800_bib52","first-page":"10086","article-title":"Blockwise parallel decoding for deep autoregressive models","volume-title":"Advances in Neural Information Processing Systems","author":"Stern","year":"2018"},{"key":"2021060823394161800_bib53","doi-asserted-by":"crossref","first-page":"3536","DOI":"10.18653\/v1\/2020.acl-main.325","article-title":"Lexically constrained neural machine translation with Levenshtein transformer","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Susanto","year":"2020"},{"key":"2021060823394161800_bib54","article-title":"Neural machine translation with external phrase memory","author":"Tang","year":"2016","journal-title":"arXiv preprint arXiv:1606.01792"},{"key":"2021060823394161800_bib55","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2021060823394161800_bib56","article-title":"A neural conversational model","volume-title":"ICML Deep Learning Workshop","author":"Vinyals","year":"2015"},{"key":"2021060823394161800_bib57","doi-asserted-by":"crossref","first-page":"479","DOI":"10.18653\/v1\/D18-1044","article-title":"Semi-autoregressive neural machine translation","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Wang","year":"2018"},{"issue":"01","key":"2021060823394161800_bib58","doi-asserted-by":"crossref","first-page":"5377","DOI":"10.1609\/aaai.v33i01.33015377","article-title":"Non-autoregressive machine translation with auxiliary regularization","volume":"33","author":"Wang","year":"2019","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2021060823394161800_bib59","first-page":"6716","article-title":"Non-monotonic sequential text generation","volume-title":"International Conference on Machine Learning","author":"Welleck","year":"2019"},{"key":"2021060823394161800_bib60","unstructured":"Fran\u00e7ois\n              Yvon\n             and Sadaf AbdulRauf. 2020. Utilisation de ressources lexicales et terminologiques en traduction neuronale. Research Report 2020-001, LIMSI-CNRS."}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00368\/1923848\/tacl_a_00368.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00368\/1923848\/tacl_a_00368.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T09:11:30Z","timestamp":1623229890000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00368\/98622\/EDITOR-An-Edit-Based-Transformer-with"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":60,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00368","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}