{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T22:34:49Z","timestamp":1777934089361,"version":"3.51.4"},"reference-count":52,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T00:00:00Z","timestamp":1683504000000},"content-version":"vor","delay-in-days":127,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,5,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing. However, there is still a lack of understanding of the limitations and weaknesses of various types of probes. In this work, we suggest a strategy for input-level intervention on naturalistic sentences. Using our approach, we intervene on the morpho-syntactic features of a sentence, while keeping the rest of the sentence unchanged. Such an intervention allows us to causally probe pre-trained models. We apply our naturalistic causal probing framework to analyze the effects of grammatical gender and number on contextualized representations extracted from three pre-trained models in Spanish, the multilingual versions of BERT, RoBERTa, and GPT-2. Our experiments suggest that naturalistic interventions lead to stable estimates of the causal effects of various linguistic properties. Moreover, our experiments demonstrate the importance of naturalistic causal probing when analyzing pre-trained models.<\/jats:p><jats:p>https:\/\/github.com\/rycolab\/naturalistic-causal-probing<\/jats:p>","DOI":"10.1162\/tacl_a_00554","type":"journal-article","created":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T21:23:12Z","timestamp":1683580992000},"page":"384-403","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":6,"title":["Naturalistic Causal Probing for Morpho-Syntax"],"prefix":"10.1162","volume":"11","author":[{"given":"Afra","family":"Amini","sequence":"first","affiliation":[{"name":"ETH Z\u00fcrich, Switzerland. afra.amini@inf.ethz.ch"},{"name":"ETH AI Center, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tiago","family":"Pimentel","sequence":"additional","affiliation":[{"name":"University of Cambridge, UK. tp472@cam.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Clara","family":"Meister","sequence":"additional","affiliation":[{"name":"ETH Z\u00fcrich, Switzerland. clara.meister@inf.ethz.ch"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ryan","family":"Cotterell","sequence":"additional","affiliation":[{"name":"ETH Z\u00fcrich, Switzerland. ryan.cotterell@inf.ethz.ch"},{"name":"ETH AI Center, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,5,9]]},"reference":[{"key":"2023050821230630600_bib1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2017.2702858","article-title":"Fine-grained analysis of sentence embeddings using auxiliary prediction tasks","volume-title":"5th International Conference on Learning Representations (Conference Track)","author":"Adi","year":"2017"},{"key":"2023050821230630600_bib2","article-title":"Understanding intermediate layers using linear classifier probes","volume-title":"5th International Conference on Learning Representations (Workshop Track)","author":"Alain","year":"2017"},{"key":"2023050821230630600_bib3","first-page":"1","article-title":"Unmasking contextual stereotypes: Measuring and mitigating BERT\u2019s gender bias","volume-title":"Proceedings of the Second Workshop on Gender Bias in Natural Language Processing","author":"Bartl","year":"2020"},{"key":"2023050821230630600_bib4","article-title":"Probing classifiers: Promises, shortcomings, and alternatives","author":"Belinkov","year":"2021","journal-title":"arXiv preprint arXiv:2102.12452"},{"key":"2023050821230630600_bib5","first-page":"1","article-title":"Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks","volume-title":"Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Belinkov","year":"2017"},{"key":"2023050821230630600_bib6","doi-asserted-by":"publisher","DOI":"10.4000\/books.aaccademia.8280","article-title":"Gender bias in italian word embeddings","volume-title":"Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020: Bologna","author":"Biasion","year":"2020"},{"key":"2023050821230630600_bib7","article-title":"Man is to computer programmer as woman is to homemaker? Debiasing word embeddings","volume-title":"Advances in Neural Information Processing Systems","author":"Bolukbasi","year":"2016"},{"issue":"2","key":"2023050821230630600_bib8","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1214\/ss\/1009213286","article-title":"Interval estimation for a binomial proportion","volume":"16","author":"Brown","year":"2001","journal-title":"Statistical Science"},{"key":"2023050821230630600_bib9","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2023050821230630600_bib10","doi-asserted-by":"publisher","first-page":"8440","DOI":"10.18653\/v1\/P18-1198","article-title":"Unsupervised cross-lingual representation learning at scale","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Conneau","year":"2020"},{"key":"2023050821230630600_bib11","doi-asserted-by":"crossref","first-page":"2126","DOI":"10.18653\/v1\/P18-1198","article-title":"What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Conneau","year":"2018"},{"key":"2023050821230630600_bib12","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023050821230630600_bib13","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00359","article-title":"Amnesic probing: Behavioral explanation with amnesic counterfactuals","author":"Elazar","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023050821230630600_bib14","doi-asserted-by":"crossref","first-page":"134","DOI":"10.18653\/v1\/W16-2524","article-title":"Probing for semantic evidence of composition by means of simple classification tasks","volume-title":"Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP","author":"Ettinger","year":"2016"},{"issue":"2","key":"2023050821230630600_bib15","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1162\/coli_a_00404","article-title":"CausaLM: Causal Model Explanation Through Counterfactual Language Models","volume":"47","author":"Feder","year":"2021","journal-title":"Computational Linguistics"},{"key":"2023050821230630600_bib16","doi-asserted-by":"publisher","first-page":"1828","DOI":"10.18653\/v1\/2021.acl-long.144","article-title":"Causal analysis of syntactic agreement mechanisms in neural language models","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Finlayson","year":"2021"},{"key":"2023050821230630600_bib17","doi-asserted-by":"publisher","first-page":"240","DOI":"10.18653\/v1\/W18-5426","article-title":"Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Giulianelli","year":"2018"},{"key":"2023050821230630600_bib18","doi-asserted-by":"publisher","first-page":"463","DOI":"10.18653\/v1\/K19-1043","article-title":"How does grammatical gender affect noun representations in gender-marking languages?","volume-title":"Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)","author":"Gonen","year":"2019"},{"key":"2023050821230630600_bib19","doi-asserted-by":"crossref","first-page":"67","DOI":"10.18653\/v1\/2022.repl4nlp-1.8","article-title":"Analyzing gender representation in multilingual models","volume-title":"Proceedings of the 7th Workshop on Representation Learning for NLP","author":"Gonen","year":"2022"},{"key":"2023050821230630600_bib20","first-page":"2525","article-title":"Multilingual central repository version 3.0","volume-title":"Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC\u201912)","author":"Gonzalez-Agirre","year":"2012"},{"key":"2023050821230630600_bib21","volume-title":"Causal inference: What if","author":"Hern\u00e1n","year":"2020"},{"key":"2023050821230630600_bib22","doi-asserted-by":"publisher","first-page":"1626","DOI":"10.18653\/v1\/2021.emnlp-main.122","article-title":"Conditional probing: Measuring usable information beyond a baseline","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Hewitt","year":"2021"},{"key":"2023050821230630600_bib23","doi-asserted-by":"crossref","first-page":"2733","DOI":"10.18653\/v1\/D19-1275","article-title":"Designing and interpreting probes with control tasks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Hewitt","year":"2019"},{"key":"2023050821230630600_bib24","doi-asserted-by":"publisher","first-page":"907","DOI":"10.1613\/jair.1.11196","article-title":"Visualisation and \u2018diagnostic classifiers\u2019 reveal how recurrent and recursive neural networks process hierarchical structure","volume":"61","author":"Hupkes","year":"2018","journal-title":"Journal of Artificial Intelligence Research"},{"key":"2023050821230630600_bib25","doi-asserted-by":"publisher","first-page":"5803","DOI":"10.18653\/v1\/D19-1588","article-title":"BERT for coreference resolution: Baselines and analysis","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Joshi","year":"2019"},{"key":"2023050821230630600_bib26","doi-asserted-by":"publisher","first-page":"12","DOI":"10.18653\/v1\/W19-4203","article-title":"Cross-lingual lemmatization and morphology tagging with two-stage multilingual BERT fine-tuning","volume-title":"Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology","author":"Kondratyuk","year":"2019"},{"key":"2023050821230630600_bib27","doi-asserted-by":"publisher","first-page":"8818","DOI":"10.18653\/v1\/2022.acl-long.603","article-title":"Probing for the usage of grammatical number","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Lasri","year":"2022"},{"key":"2023050821230630600_bib28","doi-asserted-by":"crossref","first-page":"5502","DOI":"10.18653\/v1\/2020.acl-main.488","article-title":"Towards debiasing sentence representations","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Liang","year":"2020"},{"key":"2023050821230630600_bib29","first-page":"92","article-title":"Universal Dependency annotation for multilingual parsing","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"McDonald","year":"2013"},{"key":"2023050821230630600_bib30","first-page":"4034","article-title":"Universal Dependencies v2: An evergrowing multilingual treebank collection","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Nivre","year":"2020"},{"key":"2023050821230630600_bib31","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1214\/09-SS057","article-title":"Causal inference in statistics: An overview","volume":"3","author":"Pearl","year":"2009","journal-title":"Statistics Surveys"},{"key":"2023050821230630600_bib32","doi-asserted-by":"crossref","first-page":"2869","DOI":"10.18653\/v1\/2021.emnlp-main.229","article-title":"A Bayesian framework for information-theoretic probing","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Pimentel","year":"2021"},{"key":"2023050821230630600_bib33","doi-asserted-by":"publisher","first-page":"3138","DOI":"10.18653\/v1\/2020.emnlp-main.254","article-title":"Pareto probing: Trading off accuracy for complexity","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pimentel","year":"2020"},{"key":"2023050821230630600_bib34","doi-asserted-by":"crossref","first-page":"4609","DOI":"10.18653\/v1\/2020.acl-main.420","article-title":"Information-theoretic probing for linguistic structure","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Pimentel","year":"2020"},{"issue":"8","key":"2023050821230630600_bib35","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"2023050821230630600_bib36","doi-asserted-by":"publisher","first-page":"7237","DOI":"10.18653\/v1\/2020.acl-main.647","article-title":"Null it out: Guarding protected attributes by iterative nullspace projection","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Ravfogel","year":"2020"},{"key":"2023050821230630600_bib37","doi-asserted-by":"crossref","first-page":"194","DOI":"10.18653\/v1\/2021.conll-1.15","article-title":"Counterfactual interventions reveal the causal effect of relative clause representations on agreement prediction","volume-title":"Proceedings of the 25th Conference on Computational Natural Language Learning","author":"Ravfogel","year":"2021"},{"key":"2023050821230630600_bib38","article-title":"Linear relaxed adversarial concept erasure","author":"Ravfogel","year":"2022"},{"key":"2023050821230630600_bib39","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2022.emnlp-main.405","article-title":"Adversarial concept erasure in kernel space","author":"Ravfogel","year":"2022"},{"key":"2023050821230630600_bib40","doi-asserted-by":"publisher","first-page":"3363","DOI":"10.18653\/v1\/2021.eacl-main.295","article-title":"Probing the probing paradigm: Does probing accuracy entail task relevance?","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Ravichander","year":"2021"},{"issue":"1","key":"2023050821230630600_bib41","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1093\/biomet\/70.1.41","article-title":"The central role of the propensity score in observational studies for causal effects","volume":"70","author":"Rosenbaum","year":"1983","journal-title":"Biometrika"},{"key":"2023050821230630600_bib42","doi-asserted-by":"publisher","first-page":"8","DOI":"10.18653\/v1\/N18-2002","article-title":"Gender bias in coreference resolution","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)","author":"Rudinger","year":"2018"},{"key":"2023050821230630600_bib43","first-page":"3257","article-title":"Understanding learning dynamics of language models with SVCCA","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Saphra","year":"2019"},{"key":"2023050821230630600_bib44","doi-asserted-by":"crossref","first-page":"1679","DOI":"10.18653\/v1\/P19-1164","article-title":"Evaluating gender bias in machine translation","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Stanovsky","year":"2019"},{"key":"2023050821230630600_bib45","article-title":"AnCora: Multilevel annotated corpora for Catalan and Spanish","volume-title":"Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC\u201908)","author":"Taul\u00e9","year":"2008"},{"key":"2023050821230630600_bib46","doi-asserted-by":"publisher","first-page":"862","DOI":"10.18653\/v1\/2021.findings-acl.76","article-title":"What if this modified that? Syntactic interventions with counterfactual embeddings","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Tucker","year":"2021"},{"key":"2023050821230630600_bib47","first-page":"12388","article-title":"Investigating gender bias in language models using causal mediation analysis","volume-title":"Advances in Neural Information Processing Systems","author":"Vig","year":"2020"},{"key":"2023050821230630600_bib48","doi-asserted-by":"crossref","first-page":"183","DOI":"10.18653\/v1\/2020.emnlp-main.14","article-title":"Information-theoretic probing with minimum description length","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Voita","year":"2020"},{"key":"2023050821230630600_bib49","doi-asserted-by":"publisher","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Transformers: State-of-the-art natural language processing","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"2023050821230630600_bib50","doi-asserted-by":"publisher","first-page":"4791","DOI":"10.18653\/v1\/P19-1472","article-title":"HellaSwag: Can a machine really finish your sentence?","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Zellers","year":"2019"},{"key":"2023050821230630600_bib51","doi-asserted-by":"publisher","first-page":"15","DOI":"10.18653\/v1\/N18-2003","article-title":"Gender bias in coreference resolution: Evaluation and debiasing methods","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)","author":"Zhao","year":"2018"},{"key":"2023050821230630600_bib52","doi-asserted-by":"publisher","first-page":"1651","DOI":"10.18653\/v1\/P19-1161","article-title":"Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Zmigrod","year":"2019"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00554\/2086305\/tacl_a_00554.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00554\/2086305\/tacl_a_00554.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,12]],"date-time":"2023-12-12T10:54:26Z","timestamp":1702378466000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00554\/115895\/Naturalistic-Causal-Probing-for-Morpho-Syntax"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":52,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00554","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}