{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T21:23:05Z","timestamp":1778534585571,"version":"3.51.4"},"reference-count":70,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T00:00:00Z","timestamp":1734998400000},"content-version":"vor","delay-in-days":358,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,12,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Fact verification on tabular evidence incentivizes the use of symbolic reasoning models where a logical form is constructed (e.g., a LISP-style program), providing greater verifiability than fully neural approaches. However, these logical forms typically rely on well-formed tables, restricting their use in many scenarios. An emerging symbolic reasoning paradigm for textual evidence focuses on natural logic inference, which constructs proofs by modeling set-theoretic relations between a claim and its evidence in natural language. This approach provides flexibility and transparency but is less compatible with tabular evidence since the relations do not extend to arithmetic functions. We propose a set-theoretic interpretation of numerals and arithmetic functions in the context of natural logic, enabling the integration of arithmetic expressions in deterministic proofs. We leverage large language models to generate arithmetic expressions by generating questions about salient parts of a claim which are answered by executing appropriate functions on tables. In a few-shot setting on FEVEROUS, we achieve an accuracy of 71.4, outperforming both fully neural and symbolic reasoning models by 3.4 points. When evaluated on TabFact without any further training, our method remains competitive with an accuracy lead of 0.5 points.<\/jats:p>","DOI":"10.1162\/tacl_a_00722","type":"journal-article","created":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T16:39:04Z","timestamp":1735058344000},"page":"1648-1671","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["TabVer: Tabular Fact Verification with Natural Logic"],"prefix":"10.1162","volume":"12","author":[{"given":"Rami","family":"Aly","sequence":"first","affiliation":[{"name":"University of Cambridge Department of Computer Science and Technology, UK rami.aly@cl.cam.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Vlachos","sequence":"additional","affiliation":[{"name":"University of Cambridge Department of Computer Science and Technology, UK andreas.vlachos@cl.cam.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,12,18]]},"reference":[{"key":"2024122416385549100_bib1","article-title":"RAFT: A real-world few-shot text classification benchmark","volume-title":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)","author":"Alex","year":"2021"},{"key":"2024122416385549100_bib2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.fever-1.1","article-title":"FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information","volume-title":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)","author":"Aly","year":"2021"},{"key":"2024122416385549100_bib3","doi-asserted-by":"publisher","first-page":"8376","DOI":"10.18653\/v1\/2023.emnlp-main.521","article-title":"QA-NatVer: Question Answering for Natural Logic-based Fact Verification","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Aly","year":"2023"},{"key":"2024122416385549100_bib4","doi-asserted-by":"publisher","first-page":"442","DOI":"10.18653\/v1\/P16-1042","article-title":"Combining natural logic and shallow reasoning for question answering","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Angeli","year":"2016"},{"key":"2024122416385549100_bib5","doi-asserted-by":"publisher","first-page":"534","DOI":"10.3115\/v1\/D14-1059","article-title":"NaturalLI: Natural Logic Inference for common sense reasoning","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","author":"Angeli","year":"2014"},{"key":"2024122416385549100_bib6","doi-asserted-by":"publisher","first-page":"283","DOI":"10.18653\/v1\/2023.acl-short.25","article-title":"Faithfulness tests for natural language explanations","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Atanasova","year":"2023"},{"key":"2024122416385549100_bib7","article-title":"Language models are few-shot learners","volume-title":"Advances in Neural Information Processing Systems 33 (NeurIPS 2020)","author":"Brown","year":"2020"},{"issue":"10","key":"2024122416385549100_bib8","doi-asserted-by":"publisher","first-page":"10482","DOI":"10.1609\/aaai.v36i10.21291","article-title":"LOREN: Logic- Regularized Reasoning for Interpretable Fact Verification","volume":"36","author":"Chen","year":"2022","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2024122416385549100_bib9","article-title":"Evaluating large language models trained on code","author":"Chen","year":"2021"},{"key":"2024122416385549100_bib10","doi-asserted-by":"publisher","first-page":"1120","DOI":"10.18653\/v1\/2023.findings-eacl.83","article-title":"Large language models are few(1)-shot table reasoners","volume-title":"Findings of the Association for Computational Linguistics: EACL 2023","author":"Chen","year":"2023"},{"key":"2024122416385549100_bib11","article-title":"TabFact: A large-scale dataset for table-based fact verification","volume-title":"8th International Conference on Learning Representations, ICLR 2020","author":"Chen","year":"2020"},{"key":"2024122416385549100_bib12","article-title":"Binding language models in symbolic languages","volume-title":"The Eleventh International Conference on Learning Representations (2023)","author":"Cheng","year":"2023"},{"key":"2024122416385549100_bib13","first-page":"35","article-title":"InstructEval: Towards holistic evaluation of instruction-tuned large language models","volume-title":"Proceedings of the First edition of the Workshop on the Scaling Behavior of Large Language Models (SCALE-LLM 2024)","author":"Chia","year":"2024"},{"issue":"70","key":"2024122416385549100_bib14","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung","year":"2024","journal-title":"Journal of Machine Learning Research"},{"key":"2024122416385549100_bib15","doi-asserted-by":"publisher","first-page":"7147","DOI":"10.18653\/v1\/2020.emnlp-main.580","article-title":"Generating fact checking briefs","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Fan","year":"2020"},{"key":"2024122416385549100_bib16","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.25","article-title":"BlendSQL: A scalable dialect for unifying hybrid question answering in relational algebra","author":"Glenn","year":"2024"},{"key":"2024122416385549100_bib17","doi-asserted-by":"publisher","first-page":"86859","DOI":"10.1109\/ACCESS.2023.3304915","article-title":"Double graph attention network reasoning method based on filtering and program-like evidence for table-based fact verification","volume":"11","author":"Gong","year":"2023","journal-title":"IEEE Access"},{"key":"2024122416385549100_bib18","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1163\/9789004368811_003","article-title":"Logic and conversation","volume-title":"Speech Acts","author":"Grice","year":"1975"},{"key":"2024122416385549100_bib19","doi-asserted-by":"publisher","first-page":"4971","DOI":"10.18653\/v1\/2022.emnlp-main.331","article-title":"PASTA: Table-operations aware fact verification via sentence-table cloze pre-training","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Zihui","year":"2022"},{"key":"2024122416385549100_bib20","article-title":"DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing","volume-title":"The Eleventh International Conference on Learning Representations (2023)","author":"He","year":"2023"},{"key":"2024122416385549100_bib21","doi-asserted-by":"publisher","first-page":"4320","DOI":"10.18653\/v1\/2020.acl-main.398","article-title":"TaPas: Weakly supervised table parsing via pre-training","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Herzig","year":"2020"},{"issue":"8","key":"2024122416385549100_bib22","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2024122416385549100_bib23","article-title":"LoRA: Low-Rank Adaptation of Large Language Models","volume-title":"International Conference on Learning Representations (2022)","author":"Edward","year":"2022"},{"key":"2024122416385549100_bib24","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1162\/tacl_a_00367","article-title":"Aligning faithful interpretations with their social attribution","volume":"9","author":"Jacovi","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"3","key":"2024122416385549100_bib25","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1076\/jqul.8.3.187.4095","article-title":"On round numbers: Pragmatic aspects of numerical expressions","volume":"8","author":"Jansen","year":"2001","journal-title":"Journal of Quantitative Linguistics"},{"key":"2024122416385549100_bib26","doi-asserted-by":"publisher","first-page":"8690","DOI":"10.18653\/v1\/2020.acl-main.768","article-title":"Are natural language inference models IMPPRESsive? Learning IMPlicature and PRESupposition","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Jeretic","year":"2020"},{"key":"2024122416385549100_bib27","article-title":"Mistral 7B","author":"Jiang","year":"2023"},{"key":"2024122416385549100_bib28","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1007\/978-981-19-7596-7_14","article-title":"A survey on table question answering: Recent advances","volume-title":"Knowledge Graph and Semantic Computing: Knowledge Graph Empowers the Digital Economy","author":"Jin","year":"2022"},{"key":"2024122416385549100_bib29","doi-asserted-by":"publisher","first-page":"52","DOI":"10.18653\/v1\/D19-6609","article-title":"Unsupervised question answering for fact-checking","volume-title":"Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)","author":"Jobanputra","year":"2019"},{"key":"2024122416385549100_bib30","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1007\/978-3-319-18111-0_23","article-title":"From natural logic to natural reasoning","volume-title":"Computational Linguistics and Intelligent Text Processing","author":"Karttunen","year":"2015"},{"key":"2024122416385549100_bib31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-319-44330-0_1","article-title":"The quantifier questionnaire","volume-title":"Handbook of Quantifiers in Natural Language: Volume II","author":"Keenan","year":"2017"},{"key":"2024122416385549100_bib32","article-title":"OpenTab: Advancing large language models as open-domain table reasoners","volume-title":"The twelfth International Conference on Learning Representations (ICLR 2024)","author":"Kong","year":"2024"},{"key":"2024122416385549100_bib33","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1162\/tacl_a_00503","article-title":"ProoFVer: Natural logic theorem proving for fact verification","volume":"10","author":"Krishna","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"3","key":"2024122416385549100_bib34","doi-asserted-by":"publisher","first-page":"522","DOI":"10.2307\/417059","article-title":"Pragmatic halos","volume":"75","author":"Lasersohn","year":"1999","journal-title":"Language"},{"key":"2024122416385549100_bib35","first-page":"389","article-title":"On the pragmatics of pragmatic slack","volume-title":"Proceedings of SINn und BEDEUtung","author":"Lauer","year":"2012"},{"key":"2024122416385549100_bib36","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2024122416385549100_bib37","doi-asserted-by":"publisher","first-page":"23","DOI":"10.18653\/v1\/P17-1003","article-title":"Neural symbolic machines: Learning semantic parsers on freebase with weak supervision","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Liang","year":"2017"},{"key":"2024122416385549100_bib38","article-title":"Memory augmented policy optimization for program synthesis and semantic parsing","volume-title":"Advances in Neural Information Processing Systems 31 (2018)","author":"Liang","year":"2018"},{"key":"2024122416385549100_bib39","article-title":"Unsupervised cross-task generalization via retrieval augmentation","volume-title":"Advances in Neural Information Processing Systems 35 (2022)","author":"Lin","year":"2022"},{"key":"2024122416385549100_bib40","article-title":"Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning","volume-title":"Advances in Neural Information Processing Systems","author":"Liu","year":"2022"},{"key":"2024122416385549100_bib41","article-title":"TAPEX: Table Pre-training via Learning a Neural SQL Executor","volume-title":"International Conference on Learning Representations (2022)","author":"Liu","year":"2022"},{"key":"2024122416385549100_bib42","article-title":"From zero to hero: Examining the power of symbolic tasks in instruction tuning","author":"Liu","year":"2023"},{"key":"2024122416385549100_bib43","article-title":"Decoupled weight decay regularization","volume-title":"International Conference on Learning Representations (2019)","author":"Loshchilov","year":"2019"},{"key":"2024122416385549100_bib44","article-title":"Natural language inference","author":"MacCartney","year":"2009"},{"key":"2024122416385549100_bib45","doi-asserted-by":"publisher","first-page":"193","DOI":"10.3115\/1654536.1654575","article-title":"Natural logic for textual inference","volume-title":"Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing","author":"MacCartney","year":"2007"},{"key":"2024122416385549100_bib46","doi-asserted-by":"publisher","first-page":"140","DOI":"10.3115\/1693756.1693772","article-title":"An extended model of natural logic","volume-title":"Proceedings of the Eight International Conference on Computational Semantics","author":"MacCartney","year":"2009"},{"key":"2024122416385549100_bib47","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/978-94-007-7284-7_8","article-title":"Natural logic and natural language inference","volume-title":"Computing Meaning: Volume 4","author":"MacCartney","year":"2014"},{"key":"2024122416385549100_bib48","doi-asserted-by":"publisher","first-page":"7624","DOI":"10.18653\/v1\/2022.acl-long.525","article-title":"Learning to generate programs for table fact verification via structure-aware semantic parsing","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Suixin","year":"2022"},{"issue":"4","key":"2024122416385549100_bib49","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1016\/j.jml.2009.07.005","article-title":"On the role of entailment patterns and scalar implicatures in the processing of numerals","volume":"61","author":"Panizza","year":"2009","journal-title":"Journal of Memory and Language"},{"key":"2024122416385549100_bib50","doi-asserted-by":"publisher","first-page":"10421","DOI":"10.18653\/v1\/2023.acl-long.581","article-title":"FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Rani","year":"2023"},{"key":"2024122416385549100_bib51","doi-asserted-by":"publisher","first-page":"5335","DOI":"10.18653\/v1\/2020.coling-main.466","article-title":"Learn to combine linguistic and symbolic information for table-based fact verification","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Qi","year":"2020"},{"issue":"5","key":"2024122416385549100_bib52","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1111\/lnc3.12018","article-title":"Bare numerals and scalar implicatures","volume":"7","author":"Spector","year":"2013","journal-title":"Language and Linguistics Compass"},{"key":"2024122416385549100_bib53","first-page":"1451","article-title":"Towards table-to-text generation with numerical reasoning","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Suadaa","year":"2021"},{"key":"2024122416385549100_bib54","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron","year":"2023"},{"key":"2024122416385549100_bib55","doi-asserted-by":"publisher","first-page":"2596","DOI":"10.18653\/v1\/D15-1312","article-title":"Identification and verification of simple claims about statistical properties","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Vlachos","year":"2015"},{"key":"2024122416385549100_bib56","doi-asserted-by":"publisher","first-page":"6288","DOI":"10.18653\/v1\/2023.findings-emnlp.416","article-title":"Explainable claim verification via knowledge-grounded reasoning with large language models","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2023","author":"Wang","year":"2023"},{"key":"2024122416385549100_bib57","article-title":"Chain-of-table: Evolving tables in the reasoning chain for table understanding","author":"Wang","year":"2024"},{"key":"2024122416385549100_bib58","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume-title":"Advances in Neural Information Processing Systems 36","author":"Wei","year":"2022"},{"issue":"1","key":"2024122416385549100_bib59","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1515\/cllt-2022-0082","article-title":"Large-scale patterns of number use in spoken and written English","volume":"20","author":"Woodin","year":"2024","journal-title":"Corpus Linguistics and Linguistic Theory"},{"key":"2024122416385549100_bib60","article-title":"ProTrix: Building models for planning and reasoning over tables with sentence context","author":"Zirui","year":"2024"},{"key":"2024122416385549100_bib61","doi-asserted-by":"publisher","first-page":"7810","DOI":"10.18653\/v1\/2020.emnlp-main.628","article-title":"Program enhanced fact verification with verbalization and graph attention network","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Yang","year":"2020"},{"key":"2024122416385549100_bib62","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.18653\/v1\/2021.findings-emnlp.90","article-title":"Exploring decomposition for table-based fact verification","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Yang","year":"2021"},{"key":"2024122416385549100_bib63","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1145\/3539618.3591708","article-title":"Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning","volume-title":"Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Ye","year":"2023"},{"key":"2024122416385549100_bib64","doi-asserted-by":"publisher","first-page":"8413","DOI":"10.18653\/v1\/2020.acl-main.745","article-title":"TaBERT: Pretraining for joint understanding of textual and tabular data","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Yin","year":"2020"},{"key":"2024122416385549100_bib65","doi-asserted-by":"crossref","DOI":"10.1109\/CSCWD61410.2024.10580146","article-title":"Are large language models table-based fact-checkers?","author":"Zhang","year":"2024"},{"key":"2024122416385549100_bib66","doi-asserted-by":"publisher","first-page":"6024","DOI":"10.18653\/v1\/2024.naacl-long.335","article-title":"TableLlama: Towards open large generalist models for tables","volume-title":"Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)","author":"Zhang","year":"2024"},{"issue":"1","key":"2024122416385549100_bib67","doi-asserted-by":"publisher","DOI":"10.14778\/3659437.3659452","article-title":"ReAcTable: Enhancing ReAct for table question answering","volume":"14","author":"Zhang","year":"2024","journal-title":"Proceedings of the VLDB"},{"key":"2024122416385549100_bib68","first-page":"12564","article-title":"PACAR: Automated fact-checking with planning and customized action reasoning using large language models","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)","author":"Zhao","year":"2024"},{"key":"2024122416385549100_bib69","doi-asserted-by":"publisher","first-page":"6053","DOI":"10.18653\/v1\/2020.acl-main.539","article-title":"LogicalFactChecker: Leveraging logical operations for fact checking with graph module network","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Zhong","year":"2020"},{"key":"2024122416385549100_bib70","article-title":"StructLM: Towards building generalist models for structured knowledge grounding","volume-title":"First Conference on Language Modeling","author":"Zhuang","year":"2024"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00722\/2487364\/tacl_a_00722.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00722\/2487364\/tacl_a_00722.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T16:39:12Z","timestamp":1735058352000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00722\/125986\/TabVer-Tabular-Fact-Verification-with-Natural"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":70,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00722","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}