{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T14:43:51Z","timestamp":1773326631196,"version":"3.50.1"},"reference-count":61,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T00:00:00Z","timestamp":1691539200000},"content-version":"vor","delay-in-days":220,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Non-AutoRegressive (NAR) text generation models have drawn much attention because of their significantly faster decoding speed and good generation quality in machine translation. However, in a wider range of text generation tasks, existing NAR models lack proper pre-training, making them still far behind the pre-trained autoregressive models. In this paper, we propose Pre-trained Directed Acyclic Transformer (PreDAT) and a novel pre-training task to promote prediction consistency in NAR generation. Experiments on five text generation tasks show that our PreDAT remarkably outperforms existing pre-trained NAR models (+4.2 score on average) and even achieves better results than pre-trained autoregressive baselines in n-gram-based metrics, along with 17 times speedup in throughput. Further analysis shows that PreDAT benefits from the unbiased prediction order that alleviates the error accumulation problem in autoregressive generation, which provides new insights into the advantages of NAR generation.1<\/jats:p>","DOI":"10.1162\/tacl_a_00582","type":"journal-article","created":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T21:03:47Z","timestamp":1691615027000},"page":"941-959","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":6,"title":["Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation"],"prefix":"10.1162","volume":"11","author":[{"given":"Fei","family":"Huang","sequence":"first","affiliation":[{"name":"The CoAI group, Tsinghua University, Beijing, China Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, Beijing National Research Center for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China. f-huang18@mails.tsinghua.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pei","family":"Ke","sequence":"additional","affiliation":[{"name":"The CoAI group, Tsinghua University, Beijing, China Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, Beijing National Research Center for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China. kepei1106@outlook.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minlie","family":"Huang","sequence":"additional","affiliation":[{"name":"The CoAI group, Tsinghua University, Beijing, China Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, Beijing National Research Center for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China. aihuang@tsinghua.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,8,7]]},"reference":[{"key":"2023080921034339300_bib1","doi-asserted-by":"publisher","first-page":"700","DOI":"10.18653\/v1\/2022.findings-acl.58","article-title":"Why exposure bias matters: An imitation learning perspective of error accumulation in language generation","volume-title":"Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22\u201327, 2022","author":"Arora","year":"2022"},{"key":"2023080921034339300_bib2","doi-asserted-by":"publisher","first-page":"8398","DOI":"10.18653\/v1\/2022.acl-long.575","article-title":"latent-GLAT: Glancing at latent variables for parallel text generation","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22\u201327, 2022","author":"Bao","year":"2022"},{"key":"2023080921034339300_bib3","first-page":"1171","article-title":"Scheduled sampling for sequence prediction with recurrent neural networks","volume-title":"Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7\u201312, 2015, Montreal, Quebec, Canada","author":"Bengio","year":"2015"},{"key":"2023080921034339300_bib4","article-title":"Language models are few-shot learners","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual","author":"Brown","year":"2020"},{"key":"2023080921034339300_bib5","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2\u20137, 2019, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023080921034339300_bib6","doi-asserted-by":"publisher","first-page":"4884","DOI":"10.18653\/v1\/P19-1483","article-title":"Handling divergent reference texts when evaluating table-to-text generation","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28 \u2013 August 2, 2019, Volume 1: Long Papers","author":"Dhingra","year":"2019"},{"key":"2023080921034339300_bib7","doi-asserted-by":"publisher","first-page":"3431","DOI":"10.18653\/v1\/2021.acl-long.266","article-title":"Rejuvenating low-frequency words: Making the most of parallel data in non-autoregressive translation","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1\u20136, 2021","author":"Ding","year":"2021"},{"key":"2023080921034339300_bib8","article-title":"Understanding and improving lexical choice in non-autoregressive translation","volume-title":"9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\u20137, 2021","author":"Ding","year":"2021"},{"key":"2023080921034339300_bib9","doi-asserted-by":"publisher","first-page":"2417","DOI":"10.18653\/v1\/2022.acl-long.172","article-title":"Redistributing low-frequency words: Making the most of monolingual data in non-autoregressive translation","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22\u201327, 2022","author":"Ding","year":"2022"},{"key":"2023080921034339300_bib10","doi-asserted-by":"publisher","first-page":"7250","DOI":"10.18653\/v1\/2022.acl-long.501","article-title":"Is GPT-3 text indistinguishable from human text? Scarecrow: A framework for scrutinizing machine text","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22\u201327, 2022","author":"Dou","year":"2022"},{"key":"2023080921034339300_bib11","first-page":"2849","article-title":"Order-agnostic cross entropy for non-autoregressive machine translation","volume-title":"Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18\u201324 July 2021, Virtual Event","author":"Cunxiao","year":"2021"},{"key":"2023080921034339300_bib12","doi-asserted-by":"publisher","DOI":"10.1201\/9780429246593","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1994"},{"key":"2023080921034339300_bib13","first-page":"3515","article-title":"Aligned cross entropy for non-autoregressive machine translation","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13\u201318 July 2020, Virtual Event","author":"Ghazvininejad","year":"2020"},{"key":"2023080921034339300_bib14","doi-asserted-by":"publisher","first-page":"6111","DOI":"10.18653\/v1\/D19-1633","article-title":"Mask-predict: Parallel decoding of conditional masked language models","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3\u20137, 2019","author":"Ghazvininejad","year":"2019"},{"key":"2023080921034339300_bib15","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1145\/1143844.1143891","article-title":"Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks","volume-title":"Machine Learning, Proceedings of the Twenty-Third International Conference (ICML 2006), Pittsburgh, Pennsylvania, USA, June 25\u201329, 2006","author":"Graves","year":"2006"},{"key":"2023080921034339300_bib16","article-title":"Non-autoregressive neural machine translation","volume-title":"6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 \u2013 May 3, 2018, Conference Track Proceedings","author":"Jiatao","year":"2018"},{"key":"2023080921034339300_bib17","doi-asserted-by":"publisher","first-page":"120","DOI":"10.18653\/v1\/2021.findings-acl.11","article-title":"Fully non-autoregressive neural machine translation: Tricks of the trade","volume-title":"Findings of the Association for Computational Linguistics: ACL\/ IJCNLP 2021, Online Event, August 1\u20136, 2021","author":"Jiatao","year":"2021"},{"key":"2023080921034339300_bib18","first-page":"11179","article-title":"Levenshtein transformer","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8\u201314, 2019, Vancouver, BC, Canada","author":"Jiatao","year":"2019"},{"key":"2023080921034339300_bib19","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1162\/tacl_a_00302","article-title":"A knowledge-enhanced pretraining model for commonsense story generation","volume":"8","author":"Guan","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023080921034339300_bib20","doi-asserted-by":"publisher","first-page":"376","DOI":"10.18653\/v1\/2020.acl-main.36","article-title":"Jointly masked sequence-to-sequence model for non-autoregressive neural machine translation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\u201310, 2020","author":"Guo","year":"2020"},{"key":"2023080921034339300_bib21","article-title":"Incorporating BERT into parallel sequence decoding with adapters","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\u201312, 2020, virtual","author":"Guo","year":"2020"},{"key":"2023080921034339300_bib22","article-title":"The curious case of neural text degeneration","volume-title":"8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020","author":"Holtzman","year":"2020"},{"key":"2023080921034339300_bib23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.07515","article-title":"Non-autoregressive translation with layer-wise prediction and deep supervision","author":"Huang","year":"2022","journal-title":"The Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022"},{"key":"2023080921034339300_bib24","doi-asserted-by":"publisher","first-page":"1577","DOI":"10.18653\/v1\/2021.findings-acl.138","article-title":"NAST: A non-autoregressive generator with word alignment for unsupervised text style transfer","volume-title":"Findings of the Association for Computational Linguistics: ACL\/IJCNLP 2021, Online Event, August 1\u20136, 2021","author":"Huang","year":"2021"},{"key":"2023080921034339300_bib25","first-page":"9356","article-title":"On the learning of non-autoregressive transformers","volume-title":"International Conference on Machine Learning, ICML 2022, 17\u201323 July 2022, Baltimore, Maryland, USA","author":"Huang","year":"2022"},{"key":"2023080921034339300_bib26","first-page":"9410","article-title":"Directed acyclic transformer for non-autoregressive machine translation","volume-title":"International Conference on Machine Learning, ICML 2022, 17\u201323 July 2022, Baltimore, Maryland, USA","author":"Huang","year":"2022"},{"key":"2023080921034339300_bib27","article-title":"Improving non-autoregressive translation models without distillation","volume-title":"The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\u201329, 2022","author":"Huang","year":"2022"},{"key":"2023080921034339300_bib28","doi-asserted-by":"publisher","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","author":"Ji","year":"2022","journal-title":"CoRR"},{"key":"2023080921034339300_bib29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.11115","article-title":"Improving non-autoregressive generation with mixup training","author":"Jiang","year":"2021","journal-title":"arXiv preprint"},{"key":"2023080921034339300_bib30","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.18653\/v1\/D16-1139","article-title":"Sequence-level knowledge distillation","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1\u20134, 2016","author":"Kim","year":"2016"},{"key":"2023080921034339300_bib31","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6980","article-title":"Adam: A method for stochastic optimization","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference Track Proceedings","author":"Kingma","year":"2015"},{"key":"2023080921034339300_bib32","doi-asserted-by":"publisher","first-page":"1173","DOI":"10.18653\/v1\/D18-1149","article-title":"Deterministic non-autoregressive neural sequence modeling by iterative refinement","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Lee","year":"2018"},{"key":"2023080921034339300_bib33","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\u201310, 2020","author":"Lewis","year":"2020"},{"key":"2023080921034339300_bib34","first-page":"1044","article-title":"ELMER: A non-autoregressive pre-trained language model for efficient and effective text generation","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7\u201311, 2022","author":"Li","year":"2022"},{"key":"2023080921034339300_bib35","doi-asserted-by":"publisher","first-page":"5707","DOI":"10.18653\/v1\/D19-1573","article-title":"Hint-based training for non-autoregressive machine translation","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3\u20137, 2019","author":"Li","year":"2019"},{"key":"2023080921034339300_bib36","doi-asserted-by":"publisher","first-page":"3016","DOI":"10.18653\/v1\/D18-1336","article-title":"End-to-end non-autoregressive neural machine translation with connectionist temporal classification","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Libovick\u00fd","year":"2018"},{"key":"2023080921034339300_bib37","doi-asserted-by":"publisher","first-page":"408","DOI":"10.18653\/v1\/2021.findings-acl.36","article-title":"GLGE: A new general language generation evaluation benchmark","volume-title":"Findings of the Association for Computational Linguistics: ACL\/IJCNLP 2021, Online Event, August 1\u20136, 2021","author":"Liu","year":"2021"},{"key":"2023080921034339300_bib38","doi-asserted-by":"publisher","first-page":"1906","DOI":"10.18653\/v1\/2020.acl-main.173","article-title":"On faithfulness and factuality in abstractive summarization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\u201310, 2020","author":"Maynez","year":"2020"},{"key":"2023080921034339300_bib39","doi-asserted-by":"publisher","first-page":"839","DOI":"10.18653\/v1\/N16-1098","article-title":"A corpus and cloze evaluation for deeper understanding of commonsense stories","volume-title":"NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12\u201317, 2016","author":"Mostafazadeh","year":"2016"},{"key":"2023080921034339300_bib40","doi-asserted-by":"publisher","first-page":"1797","DOI":"10.18653\/v1\/D18-1206","article-title":"Don\u2019t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Narayan","year":"2018"},{"key":"2023080921034339300_bib41","doi-asserted-by":"publisher","first-page":"48","DOI":"10.18653\/v1\/N19-4009","article-title":"fairseq: A fast, extensible toolkit for sequence modeling","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2\u20137, 2019, Demonstrations","author":"Ott","year":"2019"},{"key":"2023080921034339300_bib42","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","article-title":"BLEU: A method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6\u201312, 2002, Philadelphia, PA, USA","author":"Papineni","year":"2002"},{"key":"2023080921034339300_bib43","article-title":"A deep reinforced model for abstractive summarization","volume-title":"6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 \u2013 May 3, 2018, Conference Track Proceedings","author":"Paulus","year":"2018"},{"key":"2023080921034339300_bib44","first-page":"8630","article-title":"BANG: Bridging autoregressive and non-autoregressive generation with large scale pretraining","volume-title":"Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18\u201324 July 2021, Virtual Event","author":"Qi","year":"2021"},{"key":"2023080921034339300_bib45","doi-asserted-by":"publisher","first-page":"2401","DOI":"10.18653\/v1\/2020.findings-emnlp.217","article-title":"ProphetNet: Predicting future n-gram for sequence-to-sequence pre-training","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16\u201320 November 2020","author":"Qi","year":"2020"},{"key":"2023080921034339300_bib46","doi-asserted-by":"publisher","first-page":"1993","DOI":"10.18653\/v1\/2021.acl-long.155","article-title":"Glancing transformer for non-autoregressive neural machine translation","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1\u20136, 2021","author":"Qian","year":"2021"},{"issue":"8","key":"2023080921034339300_bib47","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"2023080921034339300_bib48","first-page":"140:1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2023080921034339300_bib49","doi-asserted-by":"publisher","first-page":"2383","DOI":"10.18653\/v1\/D16-1264","article-title":"SQuAD: 100, 000+ questions for machine comprehension of text","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1\u20134, 2016","author":"Rajpurkar","year":"2016"},{"key":"2023080921034339300_bib50","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.06732","article-title":"Sequence level training with recurrent neural networks","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2\u20134, 2016, Conference Track Proceedings","author":"Ranzato","year":"2016"},{"key":"2023080921034339300_bib51","doi-asserted-by":"publisher","first-page":"3779","DOI":"10.18653\/v1\/2022.naacl-main.277","article-title":"One reference is not enough: Diverse distillation with reference selection for non-autoregressive translation","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10\u201315, 2022","author":"Shao","year":"2022"},{"key":"2023080921034339300_bib52","doi-asserted-by":"publisher","first-page":"3784","DOI":"10.18653\/v1\/2021.findings-emnlp.320","article-title":"Retrieval augmentation reduces hallucination in conversation","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event \/ Punta Cana, Dominican Republic, 16\u201320 November, 2021","author":"Shuster","year":"2021"},{"key":"2023080921034339300_bib53","first-page":"5926","article-title":"MASS: Masked sequence to sequence pre-training for language generation","volume-title":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\u201315 June 2019, Long Beach, California, USA","author":"Song","year":"2019"},{"key":"2023080921034339300_bib54","doi-asserted-by":"publisher","first-page":"234","DOI":"10.18653\/v1\/2021.eacl-main.18","article-title":"Non-autoregressive text generation with pre-trained language models","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 \u2013 23, 2021","author":"Yixuan","year":"2021"},{"key":"2023080921034339300_bib55","first-page":"3011","article-title":"Fast structured decoding for sequence models","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8\u201314, 2019, Vancouver, BC, Canada","author":"Sun","year":"2019"},{"key":"2023080921034339300_bib56","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4\u20139, 2017, Long Beach, CA, USA","author":"Vaswani","year":"2017"},{"key":"2023080921034339300_bib57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/SC41404.2022.00043","article-title":"Lightseq2: Accelerated training for transformer-based models on GPUs","volume-title":"SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13\u201318, 2022","author":"Wang","year":"2022"},{"key":"2023080921034339300_bib58","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.18653\/v1\/2020.acl-main.101","article-title":"Towards faithful neural table-to-text generation with content-matching constraints","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\u201310, 2020","author":"Wang","year":"2020"},{"key":"2023080921034339300_bib59","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"HuggingFace\u2019s transformers: State-of-the-art natural language processing","author":"Wolf","year":"2019","journal-title":"arXiv preprint"},{"key":"2023080921034339300_bib60","doi-asserted-by":"publisher","first-page":"2204","DOI":"10.18653\/v1\/P18-1205","article-title":"Personalizing dialogue agents: I have a dog, do you have pets too?","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15\u201320, 2018, Volume 1: Long Papers","author":"Zhang","year":"2018"},{"key":"2023080921034339300_bib61","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1109\/ICCV.2015.11","article-title":"Aligning books and movies: Towards story-like visual explanations by watching movies and reading books","volume-title":"2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7\u201313, 2015","author":"Zhu","year":"2015"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00582\/2153225\/tacl_a_00582.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00582\/2153225\/tacl_a_00582.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T21:04:08Z","timestamp":1691615048000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00582\/117169\/Directed-Acyclic-Transformer-Pre-training-for-High"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":61,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00582","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}