{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T21:23:07Z","timestamp":1778534587800,"version":"3.51.4"},"reference-count":50,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2022,1,28]],"date-time":"2022-01-28T00:00:00Z","timestamp":1643328000000},"content-version":"vor","delay-in-days":27,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,1,28]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Existing table question answering datasets contain abundant factual questions that primarily evaluate a QA system\u2019s comprehension of query and tabular data. However, restricted by their short-form answers, these datasets fail to include question\u2013answer interactions that represent more advanced and naturally occurring information needs: questions that ask for reasoning and integration of information pieces retrieved from a structured knowledge source. To complement the existing datasets and to reveal the challenging nature of the table-based question answering task, we introduce FeTaQA, a new dataset with 10K Wikipedia-based {table, question, free-form answer, supporting table cells} pairs. FeTaQA is collected from noteworthy descriptions of Wikipedia tables that contain information people tend to seek; generation of these descriptions requires advanced processing that humans perform on a daily basis: Understand the question and table, retrieve, integrate, infer, and conduct text planning and surface realization to generate an answer. We provide two benchmark methods for the proposed task: a pipeline method based on semantic parsing-based QA systems and an end-to-end method based on large pretrained text generation models, and show that FeTaQA poses a challenge for both methods.<\/jats:p>","DOI":"10.1162\/tacl_a_00446","type":"journal-article","created":{"date-parts":[[2022,1,28]],"date-time":"2022-01-28T19:23:00Z","timestamp":1643397780000},"page":"35-49","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":62,"title":["FeTaQA: Free-form Table Question Answering"],"prefix":"10.1162","volume":"10","author":[{"given":"Linyong","family":"Nan","sequence":"first","affiliation":[{"name":"Yale University, USA. linyong.nan@yale.edu"}]},{"given":"Chiachun","family":"Hsieh","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Ziming","family":"Mao","sequence":"additional","affiliation":[{"name":"Yale University, USA. ziming.mao@yale.edu"}]},{"given":"Xi Victoria","family":"Lin","sequence":"additional","affiliation":[{"name":"Salesforce Research, USA"}]},{"given":"Neha","family":"Verma","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Rui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Penn State University, USA"}]},{"given":"Wojciech","family":"Kry\u015bci\u0144ski","sequence":"additional","affiliation":[{"name":"Salesforce Research, USA"}]},{"given":"Hailey","family":"Schoelkopf","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Riley","family":"Kong","sequence":"additional","affiliation":[{"name":"Archbishop Mitty High School, USA"}]},{"given":"Xiangru","family":"Tang","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Mutethia","family":"Mutuma","sequence":"additional","affiliation":[{"name":"Yale University, USA. linyong.nan@yale.edu"}]},{"given":"Ben","family":"Rosand","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Isabel","family":"Trindade","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Renusree","family":"Bandaru","sequence":"additional","affiliation":[{"name":"Penn State University, USA"}]},{"given":"Jacob","family":"Cunningham","sequence":"additional","affiliation":[{"name":"Penn State University, USA"}]},{"given":"Caiming","family":"Xiong","sequence":"additional","affiliation":[{"name":"Salesforce Research, USA"}]},{"given":"Dragomir","family":"Radev","sequence":"additional","affiliation":[{"name":"Yale University, USA"}]},{"given":"Dragomir","family":"Radev","sequence":"additional","affiliation":[{"name":"Salesforce Research, USA"}]}],"member":"281","published-online":{"date-parts":[[2022,1,28]]},"reference":[{"issue":"CSCW","key":"2022042719122411000_bib1","doi-asserted-by":"publisher","DOI":"10.1145\/3274290","article-title":"With few eyes, all hoaxes are deep","volume":"2","author":"Asthana","year":"2018","journal-title":"Proceedings of the ACM on Human Computer Interaction"},{"key":"2022042719122411000_bib2","first-page":"65","article-title":"METEOR: An automatic metric for MT evaluation with improved correlation with human judgments","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee","year":"2005"},{"issue":"2","key":"2022042719122411000_bib3","first-page":"57","article-title":"Question answering from frequently asked question files: Experiences with the faq finder system","volume":"18","author":"Burke","year":"1997","journal-title":"AI Magazine"},{"key":"2022042719122411000_bib4","doi-asserted-by":"publisher","first-page":"1870","DOI":"10.18653\/v1\/P17-1171","article-title":"Reading Wikipedia to answer open-domain questions","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chen","year":"2017"},{"key":"2022042719122411000_bib5","article-title":"Open question answering over tables and text","volume-title":"Proceedings of ICLR 2021","author":"Chen","year":"2021"},{"key":"2022042719122411000_bib6","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.708","article-title":"Logical natural language generation from open-domain tables","volume-title":"ACL","author":"Chen","year":"2020"},{"key":"2022042719122411000_bib7","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.697","article-title":"Kgpt: Knowledge-grounded pre-training for data-to-text generation","volume-title":"EMNLP","author":"Chen","year":"2020"},{"key":"2022042719122411000_bib8","article-title":"Tabfact: A large-scale dataset for table-based fact verification","volume-title":"International Conference on Learning Representations (ICLR)","author":"Chen","year":"2020"},{"key":"2022042719122411000_bib9","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.91","article-title":"Hybridqa: A dataset of multi-hop question answering over tabular and textual data","author":"Chen","year":"2020","journal-title":"Findings of EMNLP 2020"},{"key":"2022042719122411000_bib10","doi-asserted-by":"publisher","first-page":"281","DOI":"10.18653\/v1\/2020.findings-emnlp.27","article-title":"Understanding tables with intermediate pre-training","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Eisenschlos","year":"2020"},{"key":"2022042719122411000_bib11","doi-asserted-by":"crossref","first-page":"3558","DOI":"10.18653\/v1\/P19-1346","article-title":"ELI5: Long form question answering","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Fan","year":"2019"},{"key":"2022042719122411000_bib12","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1033","article-title":"Improving text-to-SQL evaluation methodology","volume-title":"ACL 2018","author":"Finegan-Dollak","year":"2018"},{"key":"2022042719122411000_bib13","doi-asserted-by":"publisher","first-page":"124","DOI":"10.18653\/v1\/W17-3518","article-title":"The WebNLG challenge: Generating text from RDF data","volume-title":"Proceedings of the 10th International Conference on Natural Language Generation","author":"Gardent","year":"2017"},{"key":"2022042719122411000_bib14","doi-asserted-by":"publisher","first-page":"4320","DOI":"10.18653\/v1\/2020.acl-main.398","article-title":"TaPas: Weakly supervised table parsing via pre-training","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Herzig","year":"2020"},{"key":"2022042719122411000_bib15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.270","article-title":"Tabbie: Pretrained representations of tabular data","volume-title":"NAACL","author":"Iida","year":"2021"},{"issue":"3","key":"2022042719122411000_bib16","doi-asserted-by":"publisher","first-page":"599","DOI":"10.5715\/jnlp.27.599","article-title":"Learning to select, track, and generate for data-to-text","volume":"27","author":"Iso","year":"2020","journal-title":"Journal of Natural Language Processing"},{"key":"2022042719122411000_bib17","doi-asserted-by":"publisher","first-page":"1821","DOI":"10.18653\/v1\/P17-1167","article-title":"Search-based neural structured learning for sequential question answering","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Iyyer","year":"2017"},{"key":"2022042719122411000_bib18","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1147","article-title":"Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","author":"Joshi","year":"2017"},{"key":"2022042719122411000_bib19","doi-asserted-by":"crossref","first-page":"97","DOI":"10.18653\/v1\/2020.inlg-1.14","article-title":"Text-to-text pre-training for data-to-text tasks","volume-title":"Proceedings of the 13th International Conference on Natural Language Generation","author":"Kale","year":"2020"},{"key":"2022042719122411000_bib20","article-title":"The narrativeqa reading comprehension challenge","author":"Kocisk\u00fd","year":"2017","journal-title":"CoRR"},{"key":"2022042719122411000_bib21","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.393","article-title":"Hurdles to progress in long-form question answering","volume-title":"NAACL","author":"Krishna","year":"2021"},{"key":"2022042719122411000_bib22","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00276","article-title":"Natural questions: A benchmark for question answering research","author":"Kwiatkowski","year":"2019","journal-title":"Transactions of the Association of Computational Linguistics"},{"key":"2022042719122411000_bib23","doi-asserted-by":"publisher","first-page":"785","DOI":"10.18653\/v1\/D17-1082","article-title":"RACE: Large-scale ReAding comprehension dataset from examinations","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Lai","year":"2017"},{"key":"2022042719122411000_bib24","doi-asserted-by":"publisher","first-page":"1000","DOI":"10.18653\/v1\/2021.eacl-main.86","article-title":"Question and answer test-train overlap in open-domain question answering datasets","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Lewis","year":"2021"},{"key":"2022042719122411000_bib25","first-page":"10015","article-title":"Memory augmented policy optimization for program synthesis and semantic parsing","volume-title":"Advances in Neural Information Processing Systems 31","author":"Liang","year":"2018"},{"key":"2022042719122411000_bib26","first-page":"74","article-title":"ROUGE: A package for automatic evaluation of summaries","volume-title":"Text Summarization Branches Out","author":"Lin","year":"2004"},{"key":"2022042719122411000_bib27","first-page":"2267","article-title":"Step-by-step: Separating planning from realization in neural data-to-text generation","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Moryossef","year":"2019"},{"key":"2022042719122411000_bib28","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.naacl-main.37","article-title":"DART: Open-domain structured data record to text generation","volume-title":"NAACL","author":"Nan","year":"2021"},{"key":"2022042719122411000_bib29","article-title":"Ms marco: A human generated machine reading comprehension dataset","volume-title":"CoCo@ NIPS","author":"Nguyen","year":"2016"},{"key":"2022042719122411000_bib30","doi-asserted-by":"publisher","first-page":"1173","DOI":"10.18653\/v1\/2020.emnlp-main.89","article-title":"ToTTo: A controlled table-to-text generation dataset","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Parikh","year":"2020"},{"key":"2022042719122411000_bib31","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.3115\/v1\/P15-1142","article-title":"Compositional semantic parsing on semi-structured tables","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Pasupat","year":"2015"},{"key":"2022042719122411000_bib32","doi-asserted-by":"publisher","first-page":"2523","DOI":"10.18653\/v1\/2021.naacl-main.200","article-title":"KILT: A benchmark for knowledge intensive language tasks","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Petroni","year":"2021"},{"key":"2022042719122411000_bib33","doi-asserted-by":"publisher","first-page":"186","DOI":"10.18653\/v1\/W18-6319","article-title":"A call for clarity in reporting BLEU scores","volume-title":"Proceedings of the Third Conference on Machine Translation: Research Papers","author":"Post","year":"2018"},{"issue":"140","key":"2022042719122411000_bib34","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2022042719122411000_bib35","doi-asserted-by":"publisher","first-page":"2383","DOI":"10.18653\/v1\/D16-1264","article-title":"SQuAD: 100,000+ questions for machine comprehension of text","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Rajpurkar","year":"2016"},{"key":"2022042719122411000_bib36","article-title":"Free-marginal multirater kappa (multirater kfree): An alternative to fleiss fixed-marginal multirater kappa","volume":"4","author":"Randolph","year":"2010","journal-title":"Advances in Data Analysis and Classification"},{"key":"2022042719122411000_bib37","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1162\/tacl_a_00266","article-title":"Coqa: A conversational question answering challenge","volume":"7","author":"Reddy","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022042719122411000_bib38","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11332","article-title":"Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph","volume-title":"AAAI 2018","author":"Saha","year":"2018"},{"key":"2022042719122411000_bib39","doi-asserted-by":"publisher","first-page":"7881","DOI":"10.18653\/v1\/2020.acl-main.704","article-title":"BLEURT: Learning robust metrics for text generation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sellam","year":"2020"},{"key":"2022042719122411000_bib40","article-title":"MultimodalQA: complex question answering over text, tables and images","volume-title":"International Conference on Learning Representations","author":"Talmor","year":"2021"},{"key":"2022042719122411000_bib41","doi-asserted-by":"publisher","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Transformers: State-of-the-art natural language processing","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"2022042719122411000_bib42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1259","article-title":"HotpotQA: A dataset for diverse, explainable multi-hop question answering","volume-title":"Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Yang","year":"2018"},{"key":"2022042719122411000_bib43","doi-asserted-by":"publisher","first-page":"956","DOI":"10.3115\/v1\/P14-1090","article-title":"Information extraction over structured data: Question answering with Freebase","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguis tics (Volume 1: Long Papers)","author":"Yao","year":"2014"},{"key":"2022042719122411000_bib44","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2020.acl-main.745","article-title":"TaBERT: Pretraining for joint understanding of textual and tabular data","volume-title":"Annual Conference of the Association for Computational Linguistics (ACL)","author":"Yin","year":"2020"},{"key":"2022042719122411000_bib45","doi-asserted-by":"crossref","first-page":"8413","DOI":"10.18653\/v1\/2020.acl-main.745","article-title":"TaBERT: Pretraining for joint understanding of textual and tabular data","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Yin","year":"2020"},{"key":"2022042719122411000_bib46","article-title":"Grappa: Grammar-augmented pre-training for table semantic parsing","volume-title":"International Conference on Learning Representations","author":"Tao","year":"2021"},{"key":"2022042719122411000_bib47","article-title":"Score: Pre-training for context representation in conversational semantic parsing","volume-title":"ICLR","author":"Tao","year":"2021"},{"key":"2022042719122411000_bib48","doi-asserted-by":"publisher","first-page":"3911","DOI":"10.18653\/v1\/D18-1425","article-title":"Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Tao","year":"2018"},{"key":"2022042719122411000_bib49","article-title":"Bertscore: Evaluating text generation with bert","volume-title":"International Conference on Learning Representations","author":"Zhang","year":"2020"},{"key":"2022042719122411000_bib50","article-title":"Seq2sql: Generating structured queries from natural language using reinforcement learning","author":"Zhong","year":"2017","journal-title":"arXiv preprint arXiv:1709.00103"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00446\/2012156\/tacl_a_00446.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00446\/2012156\/tacl_a_00446.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,17]],"date-time":"2024-09-17T15:15:17Z","timestamp":1726586117000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00446\/109273\/FeTaQA-Free-form-Table-Question-Answering"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":50,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00446","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}