{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T05:36:14Z","timestamp":1775280974688,"version":"3.50.1"},"reference-count":46,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":76,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,3,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples. In this work, we propose a model that computes a representation and denotation for all question spans in a bottom-up, compositional manner using a CKY-style parser. Our model induces latent trees, driven by end-to-end (the answer) supervision only. We show that this inductive bias towards tree structures dramatically improves systematic generalization to out-of- distribution examples, compared to strong baselines on an arithmetic expressions benchmark as well as on C losure, a dataset that focuses on systematic generalization for grounded question answering. On this challenging dataset, our model reaches an accuracy of 96.1%, significantly higher than prior models that almost perfectly solve the task on a random, in-distribution split.<\/jats:p>","DOI":"10.1162\/tacl_a_00361","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T21:16:07Z","timestamp":1616102167000},"page":"195-210","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":8,"title":["Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering"],"prefix":"10.1162","volume":"9","author":[{"given":"Ben","family":"Bogin","sequence":"first","affiliation":[{"name":"Tel-Aviv University, Israel. ben.bogin@cs.tau.ac.il"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sanjay","family":"Subramanian","sequence":"additional","affiliation":[{"name":"Allen Institute for AI, United States. sanjays@allenai.org"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matt","family":"Gardner","sequence":"additional","affiliation":[{"name":"Allen Institute for AI, United States. mattg@allenai.org"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Berant","sequence":"additional","affiliation":[{"name":"Tel-Aviv University, Israel."},{"name":"Allen Institute for AI, United States. joberant@cs.tau.ac.il"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2021,3,11]]},"reference":[{"key":"2021060823410037800_bib1","doi-asserted-by":"crossref","first-page":"6077","DOI":"10.1109\/CVPR.2018.00636","article-title":"Bottom-up and top-down attention for image captioning and visual question answering","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Anderson","year":"2018"},{"key":"2021060823410037800_bib2","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/CVPR.2016.12","article-title":"Neural module networks","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Andreas","year":"2016"},{"key":"2021060823410037800_bib3","article-title":"Layer normalization","author":"Ba","year":"2016","journal-title":"CoRR"},{"key":"2021060823410037800_bib4","article-title":"Systematic generalization: What is required and can it be learned?","volume-title":"International Conference on Learning Representations","author":"Bahdanau","year":"2019"},{"key":"2021060823410037800_bib5","article-title":"CLOSURE: Assessing systematic generalization of CLEVR models","author":"Bahdanau","year":"2019","journal-title":"CoRR"},{"key":"2021060823410037800_bib6","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-58577-8_7","article-title":"UNITER: Universal image-text representation learning","volume-title":"ECCV","author":"Chen","year":"2020"},{"key":"2021060823410037800_bib7","doi-asserted-by":"crossref","DOI":"10.1515\/9783112316009","volume-title":"Syntactic Structures","author":"Chomsky","year":"1957"},{"key":"2021060823410037800_bib8","volume-title":"Programming Languages and Their Compilers: Preliminary Notes","author":"Cocke","year":"1969"},{"key":"2021060823410037800_bib9","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2021060823410037800_bib10","first-page":"1129","article-title":"Unsupervised latent tree induction with deep inside-outside recursive auto-encoders","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Drozdov","year":"2019"},{"key":"2021060823410037800_bib11","doi-asserted-by":"crossref","first-page":"351","DOI":"10.18653\/v1\/P18-1033","article-title":"Improving text-to-SQL evaluation methodology","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Finegan-Dollak","year":"2018"},{"key":"2021060823410037800_bib12","first-page":"371","volume-title":"Connectionism and Cognitive Architecture: A Critical Analysis","author":"Fodor","year":"1988"},{"key":"2021060823410037800_bib13","doi-asserted-by":"crossref","first-page":"946","DOI":"10.18653\/v1\/2020.acl-main.89","article-title":"Injecting numerical reasoning skills into language models","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Geva","year":"2020"},{"key":"2021060823410037800_bib14","doi-asserted-by":"crossref","first-page":"2152","DOI":"10.18653\/v1\/D18-1239","article-title":"Neural compositional denotational semantics for question answering","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Gupta","year":"2018"},{"key":"2021060823410037800_bib15","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2021060823410037800_bib16","article-title":"Explainable neural computation via stack neural module networks","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Ronghang","year":"2018"},{"key":"2021060823410037800_bib17","article-title":"Compositional attention networks for machine reasoning","volume-title":"International Conference on Learning Representations","author":"Hudson","year":"2018"},{"key":"2021060823410037800_bib18","doi-asserted-by":"crossref","first-page":"2726","DOI":"10.18653\/v1\/P19-1262","article-title":"Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop QA","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jiang","year":"2019"},{"key":"2021060823410037800_bib19","doi-asserted-by":"crossref","first-page":"1988","DOI":"10.1109\/CVPR.2017.215","article-title":"CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Johnson","year":"2017"},{"key":"2021060823410037800_bib20","doi-asserted-by":"crossref","first-page":"3008","DOI":"10.1109\/ICCV.2017.325","article-title":"Inferring and executing programs for visual reasoning","volume-title":"2017 IEEE International Conference on Computer Vision (ICCV)","author":"Johnson","year":"2017"},{"key":"2021060823410037800_bib21","article-title":"Neural GPUs learn algorithms","author":"Kaiser","year":"2016","journal-title":"International Conference on Learning Representations"},{"key":"2021060823410037800_bib22","article-title":"An efficient recognition and syntax analysis algorithm for context-free languages","author":"Kasami","year":"1965"},{"key":"2021060823410037800_bib23","article-title":"Measuring compositional generalization: A comprehensive method on realistic data","volume-title":"International Conference on Learning Representations","author":"Keysers","year":"2020"},{"key":"2021060823410037800_bib24","article-title":"Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks","volume-title":"ICML","author":"Lake","year":"2018"},{"key":"2021060823410037800_bib25","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2018","journal-title":"Behavioral and Brain Sciences"},{"key":"2021060823410037800_bib26","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.18653\/v1\/D15-1137","article-title":"The forest convolutional network: Compositional distributional semantics with a neural chart and without binarization","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Le","year":"2015"},{"issue":"2","key":"2021060823410037800_bib27","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1162\/COLI_a_00127","article-title":"Learning dependency-based compositional semantics","volume":"39","author":"Liang","year":"2013","journal-title":"Computational Linguistics"},{"key":"2021060823410037800_bib28","doi-asserted-by":"crossref","first-page":"5210","DOI":"10.18653\/v1\/2020.acl-main.465","article-title":"How can we accelerate progress towards human-like linguistic generalization?","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Linzen","year":"2020"},{"key":"2021060823410037800_bib29","doi-asserted-by":"crossref","first-page":"1554","DOI":"10.18653\/v1\/D18-1184","article-title":"Structured alignment networks for matching sentences","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2018"},{"issue":"4","key":"2021060823410037800_bib30","doi-asserted-by":"crossref","first-page":"433449","DOI":"10.1017\/S1351324919000184","article-title":"Jointly learning sentence embeddings and syntax with unsupervised tree-LSTMs","volume":"25","author":"Maillard","year":"2019","journal-title":"Natural Language Engineering"},{"key":"2021060823410037800_bib31","article-title":"The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision","volume-title":"International Conference on Learning Representations","author":"Mao","year":"2019"},{"issue":"3","key":"2021060823410037800_bib32","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1111\/j.1755-2567.1970.tb00434.x","article-title":"Universal grammar","volume":"36","author":"Montague","year":"1970","journal-title":"Theoria"},{"key":"2021060823410037800_bib33","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","article-title":"GloVe: Global vectors for word representation","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington","year":"2014"},{"key":"2021060823410037800_bib34","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11671","article-title":"FiLM: Visual reasoning with a general conditioning layer","volume-title":"AAAI Conference on Artificial Intelligence","author":"Perez","year":"2018"},{"key":"2021060823410037800_bib35","first-page":"91","article-title":"Faster R-CNN: Towards real- time object detection with region proposal networks","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1","author":"Ren","year":"2015"},{"key":"2021060823410037800_bib36","article-title":"A benchmark for systematic generalization in grounded language understanding","author":"Ruis","year":"2020","journal-title":"CoRR"},{"key":"2021060823410037800_bib37","doi-asserted-by":"crossref","first-page":"5594","DOI":"10.18653\/v1\/2020.acl-main.495","article-title":"Obtaining faithful interpretations from compositional neural networks","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Subramanian","year":"2020"},{"key":"2021060823410037800_bib38","article-title":"Analyzing compositionality in visual question answering.","volume-title":"NeurIPS Workshop on Visually Grounded Interaction and Language","author":"Subramanian","year":"2019"},{"key":"2021060823410037800_bib39","doi-asserted-by":"crossref","first-page":"5100","DOI":"10.18653\/v1\/D19-1514","article-title":"LXMERT: Learning cross-modality encoder representations from transformers","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Tan","year":"2019"},{"key":"2021060823410037800_bib40","article-title":"On the value of out-of-distribution testing: An example of goodhart\u2019s law","author":"Teney","year":"2020","journal-title":"CoRR"},{"key":"2021060823410037800_bib41","first-page":"8035","article-title":"Neural arithmetic logic units","volume-title":"Advances in Neural Information Processing Systems","author":"Trask","year":"2018"},{"key":"2021060823410037800_bib42","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems 30","author":"Vaswani","year":"2017"},{"key":"2021060823410037800_bib43","first-page":"1031","article-title":"Neural-symbolic VQA: Disentangling reasoning from vision and language understanding","volume-title":"Advances in Neural Information Processing Systems","author":"Yi","year":"2018"},{"issue":"2","key":"2021060823410037800_bib44","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/S0019-9958(67)80007-X","article-title":"Recognition and parsing of context-free languages in time n3","volume":"10","author":"Younger","year":"1967","journal-title":"Information and Control"},{"key":"2021060823410037800_bib45","article-title":"Learning to execute","author":"Zaremba","year":"2014","journal-title":"CoRR"},{"key":"2021060823410037800_bib46","article-title":"Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars","volume-title":"UAI","author":"Zettlemoyer","year":"2005"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00361\/1924225\/tacl_a_00361.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00361\/1924225\/tacl_a_00361.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T22:31:41Z","timestamp":1671661901000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00361\/98090\/Latent-Compositional-Representations-Improve"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":46,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00361","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}