{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T02:40:56Z","timestamp":1781664056448,"version":"3.54.5"},"reference-count":109,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T00:00:00Z","timestamp":1678924800000},"content-version":"vor","delay-in-days":74,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,3,14]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the last few years, the natural language processing community has witnessed advances in neural representations of free texts with transformer-based language models (LMs). Given the importance of knowledge available in tabular data, recent research efforts extend LMs by developing neural representations for structured data. In this article, we present a survey that analyzes these efforts. We first abstract the different systems according to a traditional machine learning pipeline in terms of training data, input representation, model training, and supported downstream tasks. For each aspect, we characterize and compare the proposed solutions. Finally, we discuss future work directions.<\/jats:p>","DOI":"10.1162\/tacl_a_00544","type":"journal-article","created":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T14:37:32Z","timestamp":1678977452000},"page":"227-249","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":70,"title":["Transformers for Tabular Data Representation: A Survey of Models and Applications"],"prefix":"10.1162","volume":"11","author":[{"given":"Gilbert","family":"Badaro","sequence":"first","affiliation":[{"name":"EURECOM, France. gilbert.badaro@eurecom.fr"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammed","family":"Saeed","sequence":"additional","affiliation":[{"name":"EURECOM, France. mohammed.saeed@eurecom.fr"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Papotti","sequence":"additional","affiliation":[{"name":"EURECOM, France. paolo.papotti@eurecom.fr"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,3,14]]},"reference":[{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.fever-1.1","article-title":"FEVEROUS: Fact extraction and VERification over unstructured and structured information","volume-title":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)","author":"Aly","year":"2021"},{"key":"2023031614352457000_","first-page":"9","article-title":"AraBERT: Transformer-based model for arabic language understanding","volume-title":"Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection","author":"Antoun","year":"2020"},{"issue":"3","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3295662","article-title":"A survey of opinion mining in Arabic: A comprehensive system perspective covering challenges and advances in tools, resources, models, applications, and visualizations","volume":"18","author":"Badaro","year":"2019","journal-title":"ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"165","DOI":"10.3115\/v1\/W14-3623","article-title":"A large scale Arabic sentiment lexicon for arabic opinion mining","volume-title":"Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)","author":"Badaro","year":"2014"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"236","DOI":"10.18653\/v1\/S18-1036","article-title":"EMA at SemEval-2018 task 1: Emotion mining for Arabic","volume-title":"Proceedings of The 12th International Workshop on Semantic Evaluation","author":"Badaro","year":"2018"},{"issue":"6","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3404854","article-title":"A link prediction approach for accurately mapping a large-scale Arabic lexical resource to english wordnet","volume":"19","author":"Badaro","year":"2020","journal-title":"ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"86","DOI":"10.18653\/v1\/S18-2009","article-title":"EmoWordNet: Automatic expansion of emotion lexicon using English Wordnet","volume-title":"Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics","author":"Badaro","year":"2018"},{"key":"2023031614352457000_","first-page":"326","article-title":"ArSEL: A large scale Arabic sentiment and emotion lexicon","author":"Badaro","year":"2018","journal-title":"OSACT"},{"issue":"12","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"3746","DOI":"10.14778\/3554821.3554890","article-title":"Transformers for tabular data representation: A tutorial on models and applications","volume":"15","author":"Badaro","year":"2022","journal-title":"Proceedings of the VLDB Endowment"},{"key":"2023031614352457000_","article-title":"A survey of word embeddings evaluation methods","author":"Bakarov","year":"2018","journal-title":"arXiv preprint arXiv:1801.09536v1"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1415","DOI":"10.3115\/v1\/P14-1133","article-title":"Semantic parsing via paraphrasing","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Berant","year":"2014"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1007\/978-3-319-25007-6_25","article-title":"TabEL: Entity linking in web tables","volume-title":"International Semantic Web Conference","author":"Bhagavatula","year":"2015"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"7","DOI":"10.18653\/v1\/N19-3002","article-title":"Identifying and reducing gender bias in word-level language models","volume-title":"2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2019-Student Research Workshop, SRW 2019","author":"Bordia","year":"2019"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3229161","article-title":"Deep neural networks and tabular data: A survey","author":"Borisov","year":"2021","journal-title":"arXiv preprint arXiv:2110.01889v3"},{"key":"2023031614352457000_","article-title":"Language models are few-shot learners","author":"Brown","year":"2020","journal-title":"CoRR"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1145\/3318464.3389742","article-title":"Creating embeddings of heterogeneous relational datasets for data integration tasks","volume-title":"Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data","author":"Cappuzzo","year":"2020"},{"key":"2023031614352457000_","article-title":"A framework for automatically interpreting tabular data at orange","volume-title":"Proceedings of the 20th International Semantic Web Conference","author":"Chabot","year":"2021"},{"key":"2023031614352457000_","article-title":"Pre-training tasks for embedding-based large-scale retrieval","volume-title":"International Conference on Learning Representations","author":"Chang","year":"2020"},{"key":"2023031614352457000_","article-title":"Open question answering over tables and text","volume-title":"International Conference on Learning Representations","author":"Chen","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"7929","DOI":"10.18653\/v1\/2020.acl-main.708","article-title":"Logical natural language generation from open-domain tables","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Chen","year":"2020"},{"key":"2023031614352457000_","article-title":"TabFact: A large-scale dataset for table-based fact verification","volume-title":"International Conference on Learning Representations","author":"Chen","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.18653\/v1\/2020.findings-emnlp.91","article-title":"HybridQA: A dataset of multi-hop question answering over tabular and textual data","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Chen","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2096","DOI":"10.18653\/v1\/2020.findings-emnlp.190","article-title":"Logic2Text: High-fidelity natural language generation from logical forms","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Chen","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"183","DOI":"10.18653\/v1\/2020.acl-main.18","article-title":"Few-shot nlg with pre-trained language model","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Chen","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1150","DOI":"10.18653\/v1\/2022.acl-long.82","article-title":"FORTAP: Using formulas for numerical-reasoning-aware table pretraining","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Cheng","year":"2022"},{"key":"2023031614352457000_","article-title":"Transformers predicting the future. Applying attention in next-frame and time series forecasting","author":"Cholakov","year":"2021","journal-title":"arXiv preprint arXiv:2108.08224v1"},{"key":"2023031614352457000_","article-title":"Masked language modeling for proteins via linearly scalable long-context transformers","author":"Choromanski","year":"2020","journal-title":"arXiv preprint arXiv:2006.03555v3"},{"key":"2023031614352457000_","article-title":"ELECTRA: pre-training text encoders as discriminators rather than generators","volume-title":"8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020","author":"Clark","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02151-0","volume-title":"Recognizing Textual Entailment: Models and Applications","author":"Dagan","year":"2013"},{"issue":"1","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0217-0","article-title":"Big data in healthcare: Management, analysis and future prospects","volume":"6","author":"Dash","year":"2019","journal-title":"Journal of Big Data"},{"issue":"3","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"307","DOI":"10.14778\/3430915.3430921","article-title":"TURL: Table understanding through representation learning","volume":"14","author":"Deng","year":"2020","journal-title":"Proceedings of the VLDB Endowment"},{"key":"2023031614352457000_","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-1553541\/v1","article-title":"Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models","author":"Ding","year":"2022","journal-title":"arXiv preprint arXiv:2203.06904v2"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"5426","DOI":"10.24963\/ijcai.2022\/761","article-title":"Table pre-training: A survey on model architectures, pre-training objectives, and downstream tasks","volume-title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22","author":"Dong","year":"2022"},{"key":"2023031614352457000_","article-title":"Semantic structure extraction for spreadsheet tables with a multi-task learning architecture","volume-title":"Workshop on Document Intelligence at NeurIPS 2019","author":"Dong","year":"2019"},{"key":"2023031614352457000_","first-page":"2793","article-title":"Attention is not all you need: Pure attention loses rank doubly exponentially with depth","volume-title":"International Conference on Machine Learning","author":"Dong","year":"2021"},{"key":"2023031614352457000_","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","volume-title":"International Conference on Learning Representations","author":"Dosovitskiy","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1145\/3447548.3467228","article-title":"TabularNet: A neural network architecture for understanding semantic structures of tabular data","volume-title":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","author":"Lun","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"7606","DOI":"10.18653\/v1\/2021.emnlp-main.600","article-title":"MATE: Multi-view attention for table transformer efficiency","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Eisenschlos","year":"2021"},{"key":"2023031614352457000_","article-title":"TabVec: Table vectors for classification of web tables","author":"Ghasemi-Gol","year":"2018","journal-title":"CoRR"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.18653\/v1\/2021.naacl-main.96","article-title":"Capturing row and column semantics in transformer based question answering over tables","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Glass","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1978","DOI":"10.18653\/v1\/2020.coling-main.179","article-title":"TableGPT: Few-shot table-to-text generation with table structure reconstruction and content matching","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Gong","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2021-698","article-title":"AST: Audio spectrogram transformer","author":"Gong","year":"2021","journal-title":"arXiv preprint arXiv:2104.01778v3"},{"key":"2023031614352457000_","first-page":"18932","article-title":"Revisiting deep learning models for tabular data","volume":"34","author":"Gorishniy","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"4","key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.5120\/8406-2030","article-title":"A survey of text question answering techniques","volume":"53","author":"Gupta","year":"2012","journal-title":"International Journal of Computer Applications"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"512","DOI":"10.18653\/v1\/2021.naacl-main.43","article-title":"Open domain question answering over tables via dense retrieval","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Herzig","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"4320","DOI":"10.18653\/v1\/2020.acl-main.398","article-title":"TaPas: Weakly supervised table parsing via pre-training","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Herzig","year":"2020"},{"key":"2023031614352457000_","first-page":"1","article-title":"VizNet: Towards a large-scale visualization learning and benchmarking repository","volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems","author":"Kevin","year":"2019"},{"key":"2023031614352457000_","article-title":"ClinicalBERT: Modeling clinical notes and predicting hospital readmission","author":"Huang","year":"2019","journal-title":"arXiv preprint arXiv:1904.05342v3"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"3446","DOI":"10.18653\/v1\/2021.naacl-main.270","article-title":"TABBIE: Pretrained representations of tabular data","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Iida","year":"2021"},{"key":"2023031614352457000_","article-title":"TabMCQ: A dataset of general knowledge tables and multiple-choice questions","author":"Jauhar","year":"2016","journal-title":"arXiv preprint arXiv:1602.03960v1"},{"issue":"11","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2508","DOI":"10.14778\/3407790.3407841","article-title":"Scrutinizer: A mixed-initiative approach to large-scale, data-driven claim verification","volume":"13","author":"Karagiannis","year":"2020","journal-title":"Proceedings of the VLDB Endowment"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1145\/278459.258561","article-title":"Passage retrieval revisited","volume-title":"SIGIR","author":"Kaszkiel","year":"1997"},{"key":"2023031614352457000_","first-page":"5156","article-title":"Transformers are RNNs: Fast autoregressive transformers with linear attention","volume-title":"International Conference on Machine Learning","author":"Katharopoulos","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2846","DOI":"10.1145\/3448016.3457543","article-title":"A deep dive into deep learning approaches for text-to-SQL systems","volume-title":"Proceedings of the 2021 International Conference on Management of Data","author":"Katsogiannis-Meimarakis","year":"2021"},{"key":"2023031614352457000_","article-title":"Transformers in vision: A survey","author":"Khan","year":"2021","journal-title":"arXiv preprint arXiv:2101.01169"},{"key":"2023031614352457000_","article-title":"Reformer: The efficient transformer","volume-title":"International Conference on Learning Representations","author":"Kitaev","year":"2020"},{"key":"2023031614352457000_","first-page":"2284","article-title":"Text generation from knowledge graphs with graph transformers","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Koncel-Kedziorski","year":"2019"},{"issue":"6","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1002\/asi.24007","article-title":"Defining textual entailment","volume":"69","author":"Korman","year":"2018","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"82","DOI":"10.18653\/v1\/2021.mrqa-1.8","article-title":"Multi-modal retrieval of tables and texts using tri-encoder models","volume-title":"Proceedings of the 3rd Workshop on Machine Reading for Question Answering","author":"Kosti\u0107","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1162\/tacl_a_00276","article-title":"Natural questions: A benchmark for question answering research","volume":"7","author":"Kwiatkowski","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023031614352457000_","article-title":"ALBERT: A lite BERT for self-supervised learning of language representations","volume-title":"International Conference on Learning Representations","author":"Lan","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1145\/2872518.2889386","article-title":"A large public corpus of web tables containing time and context metadata","volume-title":"Proceedings of the 25th International Conference Companion on World Wide Web","author":"Lehmberg","year":"2016"},{"key":"2023031614352457000_","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.18653\/v1\/2021.findings-acl.136","article-title":"Few-shot knowledge graph-to-text generation with pretrained language models","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Li","year":"2021"},{"issue":"1","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"50","DOI":"10.14778\/3421424.3421431","article-title":"Deep entity matching with pre-trained language models","volume":"14","author":"Li","year":"2020","journal-title":"Proceedings of the VLDB Endowment"},{"key":"2023031614352457000_","article-title":"TAPEX: Table pre-training via learning a neural SQL executor","author":"Liu","year":"2021","journal-title":"arXiv preprint arXiv:2107.07653v3"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"14857","DOI":"10.1609\/aaai.v35i17.17744","article-title":"Mitigating political bias in language models through reinforced calibration","volume-title":"Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI","author":"Liu","year":"2021"},{"key":"2023031614352457000_","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"CoRR"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"5356","DOI":"10.18653\/v1\/2021.acl-long.416","article-title":"StereoSet: Measuring stereotypical bias in pretrained language models","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Nadeem","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"4551","DOI":"10.24963\/ijcai.2021\/619","article-title":"Automated fact-checking for assisting human fact-checkers","volume-title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21","author":"Nakov","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2799","DOI":"10.18653\/v1\/2021.naacl-main.224","article-title":"Incorporating external knowledge to enhance tabular reasoning","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics","author":"Neeraja","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"202","DOI":"10.18653\/v1\/2021.acl-demo.24","article-title":"CLTR: An end-to-end, transformer-based system for cell-level table retrieval and table question answering","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations","author":"Pan","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1173","DOI":"10.18653\/v1\/2020.emnlp-main.89","article-title":"ToTTo: A controlled table-to-text generation dataset","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Parikh","year":"2020"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.3115\/v1\/P15-1142","article-title":"Compositional semantic parsing on semi-structured tables","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Pasupat","year":"2015"},{"issue":"8","key":"2023031614352457000_","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"2023031614352457000_","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"issue":"1","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/2047-2501-2-3","article-title":"Big data analytics in healthcare: Promise and potential","volume":"2","author":"Raghupathi","year":"2014","journal-title":"Health Information Science and Systems"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"4902","DOI":"10.18653\/v1\/2020.acl-main.442","article-title":"Beyond accuracy: Behavioral testing of NLP models with CheckList","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Ribeiro","year":"2020"},{"key":"2023031614352457000_","first-page":"109","article-title":"Okapi at TREC-3","volume":"109","author":"Robertson","year":"1995","journal-title":"Nist Special Publication Sp"},{"issue":"12","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/3381831","article-title":"Green AI","volume":"63","author":"Schwartz","year":"2020","journal-title":"Communications of the ACM"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2931","DOI":"10.18653\/v1\/P19-1282","article-title":"Is attention interpretable?","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Serrano","year":"2019"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.11.011","article-title":"Tabular data: Deep learning is not all you need","volume-title":"8th ICML Workshop on Automated Machine Learning (AutoML)","author":"Shwartz-Ziv","year":"2021"},{"key":"2023031614352457000_","article-title":"SAINT: Improved neural networks for tabular data via row attention and contrastive pre-training","author":"Somepalli","year":"2021","journal-title":"arXiv preprint arXiv:2106.01342v1"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"13693","DOI":"10.1609\/aaai.v34i09.7123","article-title":"Energy and policy considerations for modern deep learning research","volume-title":"The Thirty-Fourth AAAI Conference on Artificial Intelligence","author":"Strubell","year":"2020"},{"key":"2023031614352457000_","first-page":"1451","article-title":"Towards table-to-text generation with numerical reasoning","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Suadaa","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1493","DOI":"10.1145\/3514221.3517906","article-title":"Annotating columns with pre-trained language models","volume-title":"Proceedings of the 2022 International Conference on Management of Data","author":"Suhara","year":"2022"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.neucom.2018.10.033","article-title":"Content-based table retrieval for web queries","volume":"349","author":"Sun","year":"2019","journal-title":"Neurocomputing"},{"key":"2023031614352457000_","first-page":"3104","article-title":"Sequence to sequence learning with neural networks","volume-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2","author":"Sutskever","year":"2014"},{"issue":"8","key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1254","DOI":"10.14778\/3457390.3457391","article-title":"RPT: Relational pre-trained transformer is almost all you need towards democratizing data preparation.","volume":"14","author":"Tang","year":"2021","journal-title":"Proceedings of the VLDB Endowment"},{"key":"2023031614352457000_","article-title":"Efficient transformers: A survey","author":"Yi","year":"2020","journal-title":"CoRR"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"809","DOI":"10.18653\/v1\/N18-1074","article-title":"FEVER: A large-scale dataset for fact extraction and verification","volume-title":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT","author":"Thorne","year":"2018"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"3091","DOI":"10.18653\/v1\/2021.acl-long.241","article-title":"Database reasoning over text","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Thorne","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1093\/oxfordhb\/9780198568971.013.0017","article-title":"Syntactic parsing","author":"Van Gompel","year":"2007","journal-title":"The Oxford handbook of psycholinguistics"},{"key":"2023031614352457000_","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2023031614352457000_","doi-asserted-by":"crossref","DOI":"10.1109\/ICDE55515.2023.00041","article-title":"Data ambiguity profiling for the generation of training examples","volume-title":"39th IEEE International Conference on Data Engineering, ICDE 2023, Anaheim, California, USA, April 3\u20137, 2023","author":"Veltri","year":"2023"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"2409","DOI":"10.1145\/3514221.3520164","article-title":"Pythia: Unsupervised generation of ambiguous textual claims from relational data","volume-title":"Proceedings of the 2022 International Conference on Management of Data","author":"Veltri","year":"2022"},{"key":"2023031614352457000_","first-page":"12388","article-title":"Investigating gender bias in language models using causal mediation analysis","volume":"33","author":"Vig","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2023031614352457000_","doi-asserted-by":"crossref","first-page":"1472","DOI":"10.1145\/3404835.3462909","article-title":"Retrieving complex tables with multi-granular graph representation learning","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang","year":"2021"},{"key":"2023031614352457000_","article-title":"Linformer: Self-attention with linear complexity","author":"Wang","year":"2020","journal-title":"arXiv preprint arXiv:2006.04768v3"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1780","DOI":"10.1145\/3447548.3467434","article-title":"TUTA: Tree-based transformers for generally structured table pre-training","volume-title":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","author":"Wang","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.suki-1.5","article-title":"Table retrieval may not necessitate table-specific model design","author":"Wang","year":"2022","journal-title":"arXiv preprint arXiv:2205.09843v1"},{"key":"2023031614352457000_","article-title":"UnifiedSKG: Unifying and multi-tasking structured knowledge grounding with text-to-text language models","author":"Xie","year":"2022","journal-title":"arXiv preprint arXiv:2201.05966v3"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"528","DOI":"10.18653\/v1\/2022.acl-long.40","article-title":"TableFormer: Robust transformer modeling for table-text encoding","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Yang","year":"2022"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.18653\/v1\/2021.findings-emnlp.90","article-title":"Exploring decomposition for table-based fact verification","volume-title":"EMNLP","author":"Yang","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"crossref","first-page":"8413","DOI":"10.18653\/v1\/2020.acl-main.745","article-title":"TaBERT: Pretraining for joint understanding of textual and tabular data","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Yin","year":"2020"},{"key":"2023031614352457000_","article-title":"GraPPa: Grammar-augmented pre-training for table semantic parsing","volume-title":"International Conference on Learning Representations","author":"Tao","year":"2021"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"3911","DOI":"10.18653\/v1\/D18-1425","article-title":"Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Tao","year":"2018"},{"key":"2023031614352457000_","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1145\/3077136.3080796","article-title":"EntiTables: Smart assistance for entity-focused tables","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhang","year":"2017"},{"key":"2023031614352457000_","article-title":"Seq2SQL: Generating structured queries from natural language using reinforcement learning","author":"Zhong","year":"2017","journal-title":"arXiv preprint arXiv:1709.00103v7"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00544\/2074873\/tacl_a_00544.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00544\/2074873\/tacl_a_00544.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,9]],"date-time":"2023-12-09T02:16:26Z","timestamp":1702088186000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00544\/115239\/Transformers-for-Tabular-Data-Representation-A"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":109,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00544","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}