{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T04:58:54Z","timestamp":1779339534716,"version":"3.51.4"},"reference-count":50,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:p> Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal. In this work, we propose a novel decoding algorithm\u2014 InDIGO\u2014which supports flexible sequence generation in arbitrary orders through insertion operations. We extend Transformer, a state-of-the-art sequence generation model, to efficiently implement the proposed approach, enabling it to be trained with either a pre-defined generation order or adaptive orders obtained from beam-search. Experiments on four real-world tasks, including word order recovery, machine translation, image caption, and code generation, demonstrate that our algorithm can generate sequences following arbitrary orders, while achieving competitive or even better performance compared with the conventional left-to-right generation. The generated sequences show that InDIGO adopts adaptive generation orders based on input information. <\/jats:p>","DOI":"10.1162\/tacl_a_00292","type":"journal-article","created":{"date-parts":[[2019,11,13]],"date-time":"2019-11-13T19:34:37Z","timestamp":1573673677000},"page":"661-676","source":"Crossref","is-referenced-by-count":19,"title":["Insertion-based Decoding with Automatically Inferred Generation Order"],"prefix":"10.1162","volume":"7","author":[{"given":"Jiatao","family":"Gu","sequence":"first","affiliation":[{"name":"Facebook AI Research."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qi","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Oxford."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyunghyun","family":"Cho","sequence":"additional","affiliation":[{"name":"New York University, CIFAR Azrieli Global Scholar."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","doi-asserted-by":"crossref","first-page":"132","DOI":"10.18653\/v1\/P17-2021","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","volume":"2","author":"Aharoni Roee","year":"2017"},{"key":"bib2","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference Track Proceedings","author":"Bahdanau Dzmitry","year":"2015"},{"key":"bib3","first-page":"65","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005"},{"key":"bib4","volume-title":"MT Summit IX. Intl. Assoc. for Machine Translation","author":"Charniak Eugene","year":"2003"},{"key":"bib5","first-page":"263","volume-title":"Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics","author":"Chiang David","year":"2005"},{"key":"bib6","author":"Cho Kyunghyun","year":"2016","journal-title":"arXiv preprint arXiv:1605.03835"},{"key":"bib7","first-page":"577","volume-title":"NIPS","author":"Chorowski Jan K.","year":"2015"},{"key":"bib8","first-page":"199","volume-title":"NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12\u201317, 2016","author":"Dyer Chris","year":"2016"},{"issue":"1","key":"bib9","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s10994-005-0916-y","volume":"60","author":"Emami Ahmad","year":"2005","journal-title":"Machine Learning"},{"key":"bib10","first-page":"72","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 \u2013 August 4, Volume 2: Short Papers","author":"Eriguchi Akiko","year":"2017"},{"key":"bib11","first-page":"2942","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Ford Nicolas","year":"2018"},{"key":"bib12","volume-title":"6th International Conference on Learning Representations, ICLR 2018, Vancouver, Canada, April 30\u2013May 3, 2018, Conference Track Proceedings","author":"Gu Jiatao","year":"2018"},{"key":"bib13","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers","author":"Gu Jiatao","year":"2016"},{"key":"bib14","first-page":"770","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He Kaiming","year":"2016"},{"key":"bib16","first-page":"944","volume-title":"Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing","author":"Isozaki Hideki","year":"2010"},{"key":"bib17","first-page":"3128","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Karpathy Andrej","year":"2015"},{"key":"bib18","first-page":"1173","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Lee Jason","year":"2018"},{"key":"bib19","first-page":"740","volume-title":"European Conference on Computer Vision","author":"Lin Tsung-Yi","year":"2014"},{"key":"bib20","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers","author":"Ling Wang","year":"2016"},{"key":"bib21","first-page":"5523","volume-title":"Advances in Neural Information Processing Systems 31","author":"Mehri Shikib","year":"2018"},{"key":"bib22","volume-title":"6th International Conference on Learning Representations, ICLR 2018, Vancouver, Canada, April 30\u2013May 3, 2018, Conference Track Proceedings","author":"Mena Gonzalo","year":"2018"},{"key":"bib25","series-title":"ASE \u201915","first-page":"574","volume-title":"Proceedings of the 2015 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Oda Yusuke","year":"2015"},{"key":"bib26","series-title":"volume 80 of Proceedings of Machine Learning Research","first-page":"3918","volume-title":"Proceedings of the 35th International Conference on Machine Learning","author":"van den Oord Aaron","year":"2018"},{"key":"bib27","first-page":"V","volume-title":"Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on","volume":"5","author":"Pal Chris","year":"2006"},{"key":"bib28","first-page":"311","volume-title":"Proceedings of the 40th annual meeting on association for computational linguistics","author":"Papineni Kishore","year":"2002"},{"key":"bib29","doi-asserted-by":"crossref","first-page":"379","DOI":"10.18653\/v1\/D15-1044","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Rush Alexander M.","year":"2015"},{"key":"bib30","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.18653\/v1\/P16-1162","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Sennrich Rico","year":"2016"},{"key":"bib31","first-page":"464","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)","author":"Shaw Peter","year":"2018"},{"key":"bib32","first-page":"223","volume-title":"Proceedings of Association for Machine Translation in the Americas","author":"Snover Matthew","year":"2006"},{"key":"bib33","author":"Stern Mitchell","year":"2019","journal-title":"arXiv preprint arXiv:1902.03249"},{"key":"bib34","first-page":"10107","volume-title":"Advances in Neural Information Processing Systems","author":"Stern Mitchell","year":"2018"},{"key":"bib35","first-page":"6961","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Sun Qing","year":"2017"},{"key":"bib36","first-page":"1017","volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML-11)","author":"Sutskever Ilya","year":"2011"},{"key":"bib37","first-page":"3104","volume-title":"Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8\u201313 2014, Montreal, Quebec, Canada","author":"Sutskever Ilya","year":"2014"},{"key":"bib38","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS)","author":"Vaswani Ashish","year":"2017"},{"key":"bib39","first-page":"4566","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Vedantam Ramakrishna","year":"2015"},{"key":"bib40","author":"Vijayakumar Ashwin K","year":"2016","journal-title":"arXiv preprint arXiv:1610.02424"},{"key":"bib41","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2\u20134, 2016, Conference Track Proceedings","author":"Vinyals Oriol","year":"2016"},{"key":"bib42","first-page":"2692","volume-title":"Advances in Neural Information Processing Systems","author":"Vinyals Oriol","year":"2015"},{"key":"bib43","author":"Vinyals Oriol","year":"2015","journal-title":"arXiv preprint arXiv:1506.05869"},{"key":"bib44","doi-asserted-by":"crossref","first-page":"479","DOI":"10.18653\/v1\/D18-1044","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Wang Chunqi","year":"2018"},{"key":"bib45","doi-asserted-by":"crossref","first-page":"4772","DOI":"10.18653\/v1\/D18-1509","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Wang Xinyi","year":"2018"},{"key":"bib46","author":"Welleck Sean","year":"2019","journal-title":"arXiv preprint arXiv:1902.02192"},{"key":"bib47","first-page":"3602","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Wu Lijun","year":"2018"},{"key":"bib48","volume-title":"Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics","author":"Yamada Kenji","year":"2001"},{"key":"bib49","first-page":"440","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Yin Pengcheng","year":"2017"},{"key":"bib50","first-page":"310","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Zhang Xingxing","year":"2016"},{"key":"bib51","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00256"},{"key":"bib52","author":"Zhou Long","year":"2019","journal-title":"IJCAI"},{"key":"bib53","author":"Zhu Wanrong","year":"2019","journal-title":"arXiv, arXiv:1901.00158"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00292","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:30Z","timestamp":1615585170000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43529"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11]]},"references-count":50,"alternative-id":["10.1162\/tacl_a_00292"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00292","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11]]}}}