{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T08:55:08Z","timestamp":1775638508937,"version":"3.50.1"},"reference-count":66,"publisher":"MIT Press - Journals","license":[{"start":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T00:00:00Z","timestamp":1622160000000},"content-version":"vor","delay-in-days":147,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,5,26]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Recent approaches to data-to-text generation have adopted the very successful encoder-decoder architecture or variants thereof. These models generate text that is fluent (but often imprecise) and perform quite poorly at selecting appropriate content and ordering it coherently. To overcome some of these issues, we propose a neural model with a macro planning stage followed by a generation stage reminiscent of traditional methods which embrace separate modules for planning and surface realization. Macro plans represent high level organization of important content such as entities, events, and their interactions; they are learned from data and given as input to the generator. Extensive experiments on two data-to-text benchmarks (RotoWire and MLB) show that our approach outperforms competitive baselines in terms of automatic and human evaluation.<\/jats:p>","DOI":"10.1162\/tacl_a_00381","type":"journal-article","created":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T16:21:21Z","timestamp":1622218881000},"page":"510-527","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":33,"title":["Data-to-text Generation with Macro Planning"],"prefix":"10.1162","volume":"9","author":[{"given":"Ratish","family":"Puduppully","sequence":"first","affiliation":[{"name":"Institute for Language, Cognition and Computation, School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB, United Kingdom. r.puduppully@sms.ed.ac.uk"}]},{"given":"Mirella","family":"Lapata","sequence":"additional","affiliation":[{"name":"Institute for Language, Cognition and Computation, School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB, United Kingdom. mlap@inf.ed.ac.uk"}]}],"member":"281","published-online":{"date-parts":[[2021,5,27]]},"reference":[{"key":"2021060823403752900_bib1","article-title":"Neural machine translation by jointly learning to align and translate","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference Track Proceedings","author":"Bahdanau","year":"2015"},{"key":"2021060823403752900_bib2","first-page":"331","article-title":"Collective content selection for concept-to-text generation","volume-title":"Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing","author":"Barzilay","year":"2005"},{"key":"2021060823403752900_bib3","volume-title":"Natural Language Processing with Python","author":"Bird","year":"2009"},{"key":"2021060823403752900_bib4","doi-asserted-by":"crossref","first-page":"552","DOI":"10.18653\/v1\/D19-1052","article-title":"Neural data-to-text generation: A comparison between pipeline and end-to-end architectures","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Ferreira","year":"2019"},{"key":"2021060823403752900_bib5","first-page":"159","article-title":"The flow of thought and the flow of language","volume-title":"Syntax and Semantics","author":"Chafe","year":"1979"},{"key":"2021060823403752900_bib6","doi-asserted-by":"crossref","first-page":"675","DOI":"10.18653\/v1\/P18-1063","article-title":"Fast abstractive summarization with reinforce-selected sentence rewriting","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chen","year":"2018"},{"key":"2021060823403752900_bib7","article-title":"Generating referring expressions in a domain of objects and processes","author":"Dale","year":"1989"},{"key":"2021060823403752900_bib8","first-page":"89","article-title":"Content planner construction via evolutionary algorithms and a corpus-based fitness function","volume-title":"Proceedings of the International Natural Language Generation Conference","author":"Duboue","year":"2002"},{"key":"2021060823403752900_bib9","first-page":"172","article-title":"Empirically estimating order constraints for content planning in generation","volume-title":"Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics","author":"Duboue","year":"2001"},{"key":"2021060823403752900_bib10","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"2021060823403752900_bib11","doi-asserted-by":"crossref","first-page":"45","DOI":"10.18653\/v1\/W18-2706","article-title":"Controllable abstractive summarization","volume-title":"Proceedings of the 2nd Workshop on Neural Machine Translation and Generation","author":"Fan","year":"2018"},{"key":"2021060823403752900_bib12","doi-asserted-by":"crossref","first-page":"179","DOI":"10.18653\/v1\/P17-1017","article-title":"Creating training corpora for NLG micro-planners","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Gardent","year":"2017"},{"key":"2021060823403752900_bib13","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1613\/jair.5477","article-title":"Survey of the state of the art in natural language generation: Core tasks, applications and evaluation","volume":"61","author":"Gatt","year":"2018","journal-title":"J. Artif. Intell. Res."},{"key":"2021060823403752900_bib14","doi-asserted-by":"crossref","first-page":"3143","DOI":"10.18653\/v1\/D19-1310","article-title":"Table-to-text generation with effective hierarchical encoder on three dimensions (row, column and time)","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Gong","year":"2019"},{"key":"2021060823403752900_bib15","first-page":"1631","article-title":"Incorporating copying mechanism in sequence-to-sequence learning","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Jiatao","year":"2016"},{"key":"2021060823403752900_bib16","doi-asserted-by":"crossref","first-page":"140","DOI":"10.18653\/v1\/P16-1014","article-title":"Pointing the unknown words","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Gulcehre","year":"2016"},{"key":"2021060823403752900_bib17","volume-title":"Cohesion in English","author":"Halliday","year":"1976"},{"key":"2021060823403752900_bib18","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"issue":"1\u20132","key":"2021060823403752900_bib19","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/0004-3702(93)90021-3","article-title":"Automated discourse generation using discourse structure relations","volume":"63","author":"Hovy","year":"1993","journal-title":"Artificial Intelligence"},{"key":"2021060823403752900_bib20","doi-asserted-by":"crossref","first-page":"2102","DOI":"10.18653\/v1\/P19-1202","article-title":"Learning to select, track, and generate for data-to-text","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Iso","year":"2019"},{"key":"2021060823403752900_bib21","first-page":"1","article-title":"Corpus-trained text generation for summarization","volume-title":"Proceedings of the InternationalNatural Language Generation Conference","author":"Kan","year":"2002"},{"key":"2021060823403752900_bib22","doi-asserted-by":"crossref","first-page":"130","DOI":"10.3115\/1610195.1610218","article-title":"Investigating content selection for language generation using machine learning","volume-title":"Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009)","author":"Kelly","year":"2009"},{"key":"2021060823403752900_bib23","doi-asserted-by":"crossref","first-page":"1328","DOI":"10.18653\/v1\/D16-1140","article-title":"Controlling output length in neural encoder-decoders","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Kikuchi","year":"2016"},{"key":"2021060823403752900_bib24","doi-asserted-by":"crossref","first-page":"67","DOI":"10.18653\/v1\/P17-4012","article-title":"OpenNMT: Open-source toolkit for neural machine translation","volume-title":"Proceedings of ACL 2017, System Demonstrations","author":"Klein","year":"2017"},{"key":"2021060823403752900_bib25","first-page":"1503","article-title":"Inducing document plans for concept-to-text generation","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Konstas","year":"2013"},{"key":"2021060823403752900_bib26","doi-asserted-by":"crossref","DOI":"10.3115\/981311.981340","article-title":"Design of a knowledge-based report generator","volume-title":"21st Annual Meeting of the Association for Computational Linguistics","author":"Kukich","year":"1983"},{"issue":"4","key":"2021060823403752900_bib27","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1162\/coli_a_00363","article-title":"Scalable micro-planned generation of discourse from structured data","volume":"45","author":"Laha","year":"2020","journal-title":"Computational Linguistics"},{"key":"2021060823403752900_bib28","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.18653\/v1\/D16-1128","article-title":"Neural text generation from structured data with application to the biography domain","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Lebret","year":"2016"},{"key":"2021060823403752900_bib29","first-page":"115","article-title":"The paragraph as a grammatical unit","volume-title":"Syntax and Semantics","author":"Longacre","year":"1979"},{"key":"2021060823403752900_bib30","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107337855","volume-title":"Best-Worst Scaling: Theory, Methods and Applications","author":"Louviere","year":"2015"},{"key":"2021060823403752900_bib31","article-title":"Best-worst scaling: A model for the largest difference judgments","author":"Louviere","year":"1991","journal-title":"University of Alberta: Working Paper"},{"key":"2021060823403752900_bib32","doi-asserted-by":"crossref","first-page":"1412","DOI":"10.18653\/v1\/D15-1166","article-title":"Effective approaches to attention-based neural machine translation","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Luong","year":"2015"},{"key":"2021060823403752900_bib33","volume-title":"Text Generation","author":"McKeown","year":"1992"},{"key":"2021060823403752900_bib34","doi-asserted-by":"crossref","first-page":"277","DOI":"10.3115\/974557.974598","article-title":"Language generation for multimedia healthcare briefings","volume-title":"Fifth Conference on Applied Natural Language Processing","author":"McKeown","year":"1997"},{"key":"2021060823403752900_bib35","first-page":"720","article-title":"What to talk about and how? Selective generation using LSTMs with coarse-to-fine alignment","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Mei","year":"2016"},{"key":"2021060823403752900_bib36","first-page":"2267","article-title":"Step-by-step: Separating planning from realization in neural data-to-text generation","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Moryossef","year":"2019"},{"key":"2021060823403752900_bib37","doi-asserted-by":"crossref","first-page":"3879","DOI":"10.18653\/v1\/D18-1422","article-title":"Operation-guided neural networks for high fidelity data-to-text generation","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Nie","year":"2018"},{"key":"2021060823403752900_bib38","first-page":"201","article-title":"The E2E dataset: New challenges for end-to-end generation","volume-title":"Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue","author":"Novikova","year":"2017"},{"key":"2021060823403752900_bib39","article-title":"Maxdiff analysis: Simple counting, individual-level logit, and HB","author":"Orme","year":"2009","journal-title":"Sawtooth Software"},{"key":"2021060823403752900_bib40","first-page":"311","article-title":"BLEU: a method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni","year":"2002"},{"key":"2021060823403752900_bib41","article-title":"A deep reinforced model for abstractive summarization","volume-title":"International Conference on Learning Representations","author":"Paulus","year":"2018"},{"key":"2021060823403752900_bib42","first-page":"1516","article-title":"Bootstrapping generators from noisy data","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Perez-Beltrachini","year":"2018"},{"key":"2021060823403752900_bib43","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v33i01.33016908","article-title":"Data-to-text generation with content selection and planning","volume-title":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence","author":"Puduppully","year":"2019"},{"key":"2021060823403752900_bib44","doi-asserted-by":"crossref","first-page":"2023","DOI":"10.18653\/v1\/P19-1195","article-title":"Data-to-text generation with entity modeling","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Puduppully","year":"2019"},{"issue":"8","key":"2021060823403752900_bib45","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"2021060823403752900_bib46","first-page":"65","article-title":"A hierarchical model for data-to-text generation","volume-title":"European Conference on Information Retrieval","author":"Rebuffel","year":"2020"},{"key":"2021060823403752900_bib47","article-title":"NLG vs. templates","volume":"cmp-lg\/9504013v1","author":"Reiter","year":"1995","journal-title":"CoRR"},{"issue":"1","key":"2021060823403752900_bib48","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1017\/S1351324997001502","article-title":"Building applied natural language generation systems","volume":"3","author":"Reiter","year":"1997","journal-title":"Natural Language Engineering"},{"key":"2021060823403752900_bib49","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511519857","volume-title":"Building Natural Language Generation Systems","author":"Reiter","year":"2000"},{"key":"2021060823403752900_bib50","doi-asserted-by":"crossref","first-page":"273","DOI":"10.18653\/v1\/D19-5631","article-title":"Naver Labs Europe\u2019s systems for the document-level generation and translation task at WNGT 2019","volume-title":"Proceedings of the 3rd Workshop on Neural Generation and Translation","author":"Saleh","year":"2019"},{"key":"2021060823403752900_bib51","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.18653\/v1\/P16-1162","article-title":"Neural machine translation of rare words with subword units","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Sennrich","year":"2016"},{"key":"2021060823403752900_bib52","doi-asserted-by":"crossref","first-page":"3257","DOI":"10.18653\/v1\/D19-1321","article-title":"Long and diverse text generation with planning-based hierarchical variational model","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Shao","year":"2019"},{"key":"2021060823403752900_bib53","first-page":"3104","article-title":"Sequence to sequence learning with neural networks","volume-title":"Advances in Neural Information Processing Systems","author":"Sutskever","year":"2014"},{"key":"2021060823403752900_bib54","first-page":"55","article-title":"Controlling target features in neural machine translation via prefix constraints","volume-title":"Proceedings of the 4th Workshop on Asian Translation (WAT 2017)","author":"Takeno","year":"2017"},{"key":"2021060823403752900_bib55","article-title":"Sticking to the facts: Confident decoding for faithful data-to-text generation","author":"Tian","year":"2019","journal-title":"CoRR"},{"key":"2021060823403752900_bib56","first-page":"355","article-title":"Best practices for the human evaluation of automatically generated text","volume-title":"Proceedings of the 12th International Conference on Natural Language Generation","author":"Lee","year":"2019"},{"key":"2021060823403752900_bib57","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems 30","author":"Vaswani","year":"2017"},{"key":"2021060823403752900_bib58","first-page":"2692","article-title":"Pointer networks","volume-title":"Advances in Neural Information Processing Systems 28","author":"Vinyals","year":"2015"},{"key":"2021060823403752900_bib59","doi-asserted-by":"crossref","first-page":"5307","DOI":"10.18653\/v1\/D19-1534","article-title":"Do NLP models know numbers? Probing numeracy in embeddings","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Wallace","year":"2019"},{"issue":"4","key":"2021060823403752900_bib60","doi-asserted-by":"crossref","first-page":"490","DOI":"10.1162\/neco.1990.2.4.490","article-title":"An efficient gradient-based algorithm for on-line training of recurrent network trajectories","volume":"2","author":"Williams","year":"1990","journal-title":"Neural Computation"},{"key":"2021060823403752900_bib61","first-page":"2253","article-title":"Challenges in data-to-document generation","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Wiseman","year":"2017"},{"key":"2021060823403752900_bib62","article-title":"Google\u2019s neural machine translation system: Bridging the gap between human and machine translation","author":"Yonghui","year":"2016","journal-title":"CoRR"},{"key":"2021060823403752900_bib63","first-page":"1480","article-title":"Hierarchical attention networks for document classification","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Yang","year":"2016"},{"key":"2021060823403752900_bib64","doi-asserted-by":"crossref","first-page":"5696","DOI":"10.18653\/v1\/D19-1571","article-title":"Simple and effective noisy channel modeling for neural machine translation","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Yee","year":"2019"},{"key":"2021060823403752900_bib65","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1162\/tacl_a_00319","article-title":"Better document-level machine translation with Bayes\u2019 rule","volume":"8","author":"Lei","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"2","key":"2021060823403752900_bib66","first-page":"171","article-title":"Semantics of paragraphs","volume":"17","author":"Zadrozny","year":"1991","journal-title":"Computational Linguistics"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00381\/1924176\/tacl_a_00381.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00381\/1924176\/tacl_a_00381.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T09:46:25Z","timestamp":1623231985000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00381\/101876\/Data-to-text-Generation-with-Macro-Planning"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":66,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00381","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}