{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T00:29:55Z","timestamp":1779928195486,"version":"3.53.1"},"reference-count":30,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2021,9,23]],"date-time":"2021-09-23T00:00:00Z","timestamp":1632355200000},"content-version":"vor","delay-in-days":265,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,9,21]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can ever \u201cunderstand\u201d raw text without access to some form of grounding. We formally investigate the abilities of ungrounded systems to acquire meaning. Our analysis focuses on the role of \u201cassertions\u201d: textual contexts that provide indirect clues about the underlying semantics. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation of languages that satisfy a strong notion of semantic transparency. However, for classes of languages where the same expression can take different values in different contexts, we show that emulation can become uncomputable. Finally, we discuss differences between our formal model and natural language, exploring how our results generalize to a modal setting and other semantic relations. Together, our results suggest that assertions in code or language do not provide sufficient signal to fully emulate semantic representations. We formalize ways in which ungrounded language models appear to be fundamentally limited in their ability to \u201cunderstand\u201d.<\/jats:p>","DOI":"10.1162\/tacl_a_00412","type":"journal-article","created":{"date-parts":[[2021,11,8]],"date-time":"2021-11-08T22:22:42Z","timestamp":1636410162000},"page":"1047-1060","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":16,"title":["Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?"],"prefix":"10.1162","volume":"9","author":[{"given":"William","family":"Merrill","sequence":"first","affiliation":[{"name":"Allen Institute for AI, United States. willm@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yoav","family":"Goldberg","sequence":"additional","affiliation":[{"name":"Allen Institute for AI, United States"},{"name":"Bar Ilan University, Israel. yoavg@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roy","family":"Schwartz","sequence":"additional","affiliation":[{"name":"Bar Ilan University, Israel. roys@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noah A.","family":"Smith","sequence":"additional","affiliation":[{"name":"Allen Institute for AI, United States"},{"name":"University of Washington, United States. noah@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2021,9,21]]},"reference":[{"key":"2021110822025289900_bib1","article-title":"Fine- grained analysis of sentence embeddings using auxiliary prediction tasks","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings","author":"Adi","year":"2017"},{"issue":"2","key":"2021110822025289900_bib2","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/0890-5401(87)90052-6","article-title":"Learning regular sets from queries and counterexamples","volume":"75","author":"Angluin","year":"1987","journal-title":"Information and Computation"},{"key":"2021110822025289900_bib3","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1162\/tacl_a_00254","article-title":"Analysis methods in neural language processing: A survey","volume":"7","author":"Belinkov","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2021110822025289900_bib4","doi-asserted-by":"publisher","first-page":"5185","DOI":"10.18653\/v1\/2020.acl-main.463","article-title":"Climbing towards NLU: On meaning, form, and understanding in the age of data","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Bender","year":"2020"},{"key":"2021110822025289900_bib5","unstructured":"Tom B. Brown , BenjaminMann, NickRyder, MelanieSubbiah, JaredKaplan, PrafullaDhariwal, ArvindNeelakantan, PranavShyam, GirishSastry, AmandaAskell, SandhiniAgarwal, ArielHerbert-Voss, GretchenKrueger, TomHenighan, RewonChild, AdityaRamesh, Daniel M.Ziegler, JeffreyWu, ClemensWinter, ChristopherHesse, MarkChen, EricSigler, MateuszLitwin, ScottGray, BenjaminChess, JackClark, ChristopherBerner, SamMcCandlish, AlecRadford, IlyaSutskever, and DarioAmodei. 2020. Language models are few-shot learners. The arXiv is: 2005.14165."},{"key":"2021110822025289900_bib6","volume-title":"Aspects of the Theory of Syntax","author":"Chomsky","year":"1965"},{"key":"2021110822025289900_bib7","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1007\/978-3-642-13089-2_2","article-title":"Three learnable models for the description of language","volume-title":"Language and Automata Theory and Applications","author":"Clark","year":"2010"},{"key":"2021110822025289900_bib8","doi-asserted-by":"publisher","first-page":"2126","DOI":"10.18653\/v1\/P18-1198","article-title":"What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Conneau","year":"2018"},{"key":"2021110822025289900_bib9","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2021110822025289900_bib10","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1163\/9789004368811_003","article-title":"Logic and conversation","volume-title":"Speech Acts","author":"Grice","year":"1975"},{"issue":"2\u20133","key":"2021110822025289900_bib11","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1080\/00437956.1954.11659520","article-title":"Distributional structure","volume":"10","author":"Harris","year":"1954","journal-title":"WORD"},{"key":"2021110822025289900_bib12","volume-title":"Semantics in Generative Grammar","author":"Heim","year":"1998"},{"key":"2021110822025289900_bib13","doi-asserted-by":"publisher","first-page":"2733","DOI":"10.18653\/v1\/D19-1275","article-title":"Designing and interpreting probes with control tasks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Hewitt","year":"2019"},{"key":"2021110822025289900_bib14","article-title":"Negation","volume-title":"The Stanford Encyclopedia of Philosophy","author":"Horn","year":"2020"},{"key":"2021110822025289900_bib15","doi-asserted-by":"publisher","first-page":"5617","DOI":"10.24963\/ijcai.2018\/796","article-title":"Visualisation and \u2019diagnostic classifiers\u2019 reveal how recurrent and recursive neural networks process hierarchical structure (extended abstract)","volume-title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18","author":"Hupkes","year":"2018"},{"key":"2021110822025289900_bib16","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2014.03.007","article-title":"On our best behaviour","volume":"212","author":"Levesque","year":"2014","journal-title":"Artificial Intelligence"},{"key":"2021110822025289900_bib17","unstructured":"Julian Michael . 2020. To dissect an octopus: Making sense of the form\/meaning debate."},{"key":"2021110822025289900_bib18","doi-asserted-by":"publisher","DOI":"10.1305\/ndjfl\/1153858644","article-title":"Is it possible for language models to achieve understanding?","author":"Potts","year":"2020"},{"issue":"2","key":"2021110822025289900_bib19","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1305\/ndjfl\/1153858644","article-title":"More fragments of language","volume":"47","author":"Pratt-Hartmann","year":"2006","journal-title":"Notre Dame Journal of Formal Logic"},{"key":"2021110822025289900_bib20","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer. The arXiv: 1910.10683","author":"Raffel","year":"2019"},{"key":"2021110822025289900_bib21","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00349","article-title":"A primer in BERTology: What we know about how BERT works","author":"Rogers","year":"2020"},{"key":"2021110822025289900_bib22","doi-asserted-by":"publisher","first-page":"pages 4593\u2013pages 4601","DOI":"10.18653\/v1\/P19-1452","article-title":"BERT rediscovers the classical NLP pipeline","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Tenney","year":"2019"},{"issue":"345\u2013363","key":"2021110822025289900_bib23","first-page":"5","article-title":"On computable numbers, with an application to the Entscheidungsproblem","volume":"58","author":"Turing","year":"1936","journal-title":"Journal of Math"},{"issue":"236","key":"2021110822025289900_bib24","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1093\/mind\/LIX.236.433","article-title":"Computing machinery and intelligence","volume":"LIX","author":"Turing","year":"1950","journal-title":"Mind"},{"key":"2021110822025289900_bib25","article-title":"Intensional semantics","author":"Fintel","year":"2011","journal-title":"Unpublished Lecture Notes"},{"key":"2021110822025289900_bib26","doi-asserted-by":"publisher","first-page":"353","DOI":"10.18653\/v1\/W18-5446","article-title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Wang","year":"2018"},{"key":"2021110822025289900_bib27","volume-title":"Principia Mathematica","author":"Whitehead","year":"1925\u20131927"},{"key":"2021110822025289900_bib28","doi-asserted-by":"crossref","DOI":"10.1515\/9780748677771","volume-title":"Elements of Formal Semantics: An Introduction to the Mathematical Theory of Meaning in Natural Language","author":"Winter","year":"2016"},{"key":"2021110822025289900_bib29","article-title":"Learning and evaluating general linguistic intelligence. arXiv: 1901 .11373","author":"Yogatama","year":"2019"},{"key":"2021110822025289900_bib30","doi-asserted-by":"publisher","first-page":"5736","DOI":"10.18653\/v1\/2020.acl-main.508","article-title":"WinoWhy: A deep diagnosis of essential commonsense knowledge for answering Winograd schema challenge","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Zhang","year":"2020"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00412\/1963983\/tacl_a_00412.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00412\/1963983\/tacl_a_00412.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,14]],"date-time":"2023-01-14T18:29:45Z","timestamp":1673720985000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00412\/107385\/Provable-Limitations-of-Acquiring-Meaning-from"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":30,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00412","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}