{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T16:24:04Z","timestamp":1781022244255,"version":"3.54.1"},"reference-count":88,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T00:00:00Z","timestamp":1714089600000},"content-version":"vor","delay-in-days":116,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,4,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Multiword expressions (MWEs) are composed of multiple words and exhibit variable degrees of compositionality. As such, their meanings are notoriously difficult to model, and it is unclear to what extent this issue affects transformer architectures. Addressing this gap, we provide the first in-depth survey of MWE processing with transformer models. We overall find that they capture MWE semantics inconsistently, as shown by reliance on surface patterns and memorized information. MWE meaning is also strongly localized, predominantly in early layers of the architecture. Representations benefit from specific linguistic properties, such as lower semantic idiosyncrasy and ambiguity of target expressions. Our findings overall question the ability of transformer models to robustly capture fine-grained semantics. Furthermore, we highlight the need for more directly comparable evaluation setups.<\/jats:p>","DOI":"10.1162\/tacl_a_00657","type":"journal-article","created":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T18:14:02Z","timestamp":1714155242000},"page":"593-612","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":6,"title":["Semantics of Multiword Expressions in Transformer-Based Models: A Survey"],"prefix":"10.1162","volume":"12","author":[{"given":"Filip","family":"Mileti\u0107","sequence":"first","affiliation":[{"name":"Institute for Natural Language Processing, University of Stuttgart, Germany. filip.miletic@ims.uni-stuttgart.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sabine Schulte im","family":"Walde","sequence":"additional","affiliation":[{"name":"Institute for Natural Language Processing, University of Stuttgart, Germany. schulte@ims.uni-stuttgart.de"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,4,30]]},"reference":[{"key":"2024042618135744000_bib1","article-title":"Designing a Russian idiom-annotated corpus","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Aharodnik","year":"2018"},{"key":"2024042618135744000_bib2","doi-asserted-by":"publisher","first-page":"195","DOI":"10.18653\/v1\/2022.naacl-srw.25","article-title":"Probe-less probing of BERT\u2019s layer-wise linguistic knowledge with masked word prediction","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop","author":"Aoyama","year":"2022"},{"key":"2024042618135744000_bib3","doi-asserted-by":"publisher","first-page":"505","DOI":"10.18653\/v1\/N19-1050","article-title":"Big BiRD: A large, fine-grained, bigram relatedness dataset for examining semantic composition","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Asaadi","year":"2019"},{"key":"2024042618135744000_bib4","first-page":"267","article-title":"Multiword expressions","volume-title":"Handbook of Natural Language Processing","author":"Baldwin","year":"2010"},{"key":"2024042618135744000_bib5","doi-asserted-by":"publisher","first-page":"3682","DOI":"10.18653\/v1\/2023.eacl-main.267","article-title":"Automatic evaluation and analysis of idioms in neural machine translation","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Baziotis","year":"2023"},{"key":"2024042618135744000_bib6","first-page":"21","article-title":"Distributional semantics and compositionality 2011: Shared task description and results","volume-title":"Proceedings of the Workshop on Distributional Semantics and Compositionality","author":"Biemann","year":"2011"},{"key":"2024042618135744000_bib7","doi-asserted-by":"publisher","first-page":"61","DOI":"10.18653\/v1\/2023.bsnlp-1.8","article-title":"Dispersing the clouds of doubt: Can cosine similarity of word embeddings help identify relation-level metaphors in Slovene?","volume-title":"Proceedings of the 9th Workshop on Slavic Natural Language Processing 2023 (SlavicNLP 2023)","author":"Brglez","year":"2023"},{"key":"2024042618135744000_bib8","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2024042618135744000_bib9","doi-asserted-by":"publisher","first-page":"2230","DOI":"10.18653\/v1\/2023.eacl-main.163","article-title":"A psycholinguistic analysis of BERT\u2019s representations of compounds","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Buijtelaar","year":"2023"},{"key":"2024042618135744000_bib10","doi-asserted-by":"publisher","first-page":"4558","DOI":"10.18653\/v1\/2022.naacl-main.338","article-title":"Using paraphrases to study properties of contextual embeddings","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Burdick","year":"2022"},{"key":"2024042618135744000_bib11","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1162\/tacl_a_00478","article-title":"It\u2019s not rocket science: Interpreting figurative language in narratives","volume":"10","author":"Chakrabarty","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618135744000_bib12","article-title":"Visualizing and measuring the geometry of BERT","volume-title":"Advances in Neural Information Processing Systems","author":"Coenen","year":"2019"},{"key":"2024042618135744000_bib13","doi-asserted-by":"publisher","first-page":"3538","DOI":"10.18653\/v1\/2022.findings-emnlp.259","article-title":"McPhraSy: Multi-context phrase similarity and clustering","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2022","author":"Cohen","year":"2022"},{"key":"2024042618135744000_bib14","doi-asserted-by":"publisher","first-page":"2698","DOI":"10.18653\/v1\/2023.findings-acl.169","article-title":"From chocolate bunny to chocolate crocodile: Do language models understand noun compounds?","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Coil","year":"2023"},{"key":"2024042618135744000_bib15","doi-asserted-by":"publisher","first-page":"8440","DOI":"10.18653\/v1\/2020.acl-main.747","article-title":"Unsupervised cross-lingual representation learning at scale","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Conneau","year":"2020"},{"issue":"4","key":"2024042618135744000_bib16","doi-asserted-by":"publisher","first-page":"837","DOI":"10.1162\/COLI_a_00302","article-title":"Multiword expression processing: A survey","volume":"43","author":"Constant","year":"2017","journal-title":"Computational Linguistics"},{"key":"2024042618135744000_bib17","first-page":"19","article-title":"The VNC-tokens dataset","volume-title":"Proceedings of the LREC Workshop Towards a Shared Task for Multiword Expressions (MWE 2008)","author":"Cook","year":"2008"},{"issue":"1","key":"2024042618135744000_bib18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/coli_a_00341","article-title":"Unsupervised compositionality prediction of nominal compounds","volume":"45","author":"Cordeiro","year":"2019","journal-title":"Computational Linguistics"},{"key":"2024042618135744000_bib19","doi-asserted-by":"publisher","first-page":"3608","DOI":"10.18653\/v1\/2022.acl-long.252","article-title":"Can transformer be too compositional? Analysing idiom processing in neural machine translation","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Dankers","year":"2022"},{"key":"2024042618135744000_bib20","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","article-title":"BERT: Pre-training of deep bidirectional Transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2024042618135744000_bib21","doi-asserted-by":"publisher","first-page":"1406","DOI":"10.18653\/v1\/2021.eacl-main.120","article-title":"Evaluating language models for the retrieval and categorization of lexical collocations","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Anke","year":"2021"},{"key":"2024042618135744000_bib22","doi-asserted-by":"publisher","first-page":"55","DOI":"10.18653\/v1\/D19-1006","article-title":"How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Ethayarajh","year":"2019"},{"key":"2024042618135744000_bib23","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1162\/tacl_a_00298","article-title":"What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models","volume":"8","author":"Ettinger","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618135744000_bib24","doi-asserted-by":"publisher","first-page":"23","DOI":"10.18653\/v1\/2021.mwe-1.4","article-title":"Contextualized embeddings encode monolingual and cross-lingual knowledge of idiomaticity","volume-title":"Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)","author":"Fakharian","year":"2021"},{"key":"2024042618135744000_bib25","first-page":"239","article-title":"Automatic classification of attributes in German adjective-noun phrases","volume-title":"Proceedings of the 14th International Conference on Computational Semantics (IWCS)","author":"Falk","year":"2021"},{"key":"2024042618135744000_bib26","doi-asserted-by":"publisher","first-page":"1258","DOI":"10.5220\/0010391812581265","article-title":"Comparing dependency-based compositional models with contextualized word embeddings","volume-title":"Proceedings of the 13th International Conference on Agents and Artificial Intelligence","author":"Gamallo","year":"2021"},{"key":"2024042618135744000_bib27","doi-asserted-by":"publisher","first-page":"153","DOI":"10.26342\/2022-69-13","article-title":"Evaluating contextualized vectors from both large language models and compositional strategies","author":"Gamallo","year":"2022","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"2024042618135744000_bib28","doi-asserted-by":"publisher","first-page":"2730","DOI":"10.18653\/v1\/2021.acl-long.212","article-title":"Assessing the representations of idiomaticity in vector models with a noun compound dataset labeled at type and token levels","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Garcia","year":"2021"},{"key":"2024042618135744000_bib29","doi-asserted-by":"publisher","first-page":"3551","DOI":"10.18653\/v1\/2021.eacl-main.310","article-title":"Probing for idiomaticity in vector space models","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Garcia","year":"2021"},{"key":"2024042618135744000_bib30","first-page":"279","article-title":"MAGPIE: A large corpus of potentially idiomatic expressions","volume-title":"Proceedings of the Twelfth Language Resources and Evaluation Conference","author":"Haagsma","year":"2020"},{"key":"2024042618135744000_bib31","unstructured":"Matthias\n              Hartung\n            \n          . 2015. Distributional Semantic Models of Attribute Meaning in Adjectives and Nouns. Ph.D. thesis, Heidelberg University."},{"key":"2024042618135744000_bib32","doi-asserted-by":"publisher","first-page":"992","DOI":"10.3115\/1613715.1613844","article-title":"Construction of an idiom corpus and its application to idiom identification based on WSD incorporating idiom-specific features","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","author":"Hashimoto","year":"2008"},{"key":"2024042618135744000_bib33","doi-asserted-by":"publisher","first-page":"248","DOI":"10.18653\/v1\/2023.eacl-main.19","article-title":"Understanding transformer memorization recall through idioms","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Haviv","year":"2023"},{"key":"2024042618135744000_bib34","article-title":"DeBERTa: Decoding-enhanced BERT with disentangled attention","volume-title":"Proceedings of the Ninth International Conference on Learning Representations (ICLR 2021)","author":"He","year":"2021"},{"key":"2024042618135744000_bib35","first-page":"138","article-title":"SemEval-2013 task 4: Free paraphrases of noun compounds","volume-title":"Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)","author":"Hendrickx","year":"2013"},{"key":"2024042618135744000_bib36","doi-asserted-by":"publisher","first-page":"3651","DOI":"10.18653\/v1\/P19-1356","article-title":"What does BERT learn about the structure of language?","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jawahar","year":"2019"},{"key":"2024042618135744000_bib37","doi-asserted-by":"publisher","first-page":"16131","DOI":"10.18653\/v1\/2023.emnlp-main.1002","article-title":"To split or not to split: Composing compounds in contextual vector spaces","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Jenkins","year":"2023"},{"key":"2024042618135744000_bib38","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1162\/tacl_a_00300","article-title":"SpanBERT: Improving pre-training by representing and predicting spans","volume":"8","author":"Joshi","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"4","key":"2024042618135744000_bib39","doi-asserted-by":"publisher","first-page":"1004","DOI":"10.3758\/s13428-014-0523-6","article-title":"A database of 629 English compound words: Ratings of familiarity, lexeme meaning dominance, semantic transparency, age of acquisition, imageability, and sensory experience","volume":"47","author":"Juhasz","year":"2015","journal-title":"Behavior Research Methods"},{"key":"2024042618135744000_bib40","doi-asserted-by":"publisher","first-page":"45","DOI":"10.18653\/v1\/2023.mwe-1.8","article-title":"Idioms, probing and dangerous things: Towards structural probing for idiomaticity in vector space","volume-title":"Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023)","author":"Klubi\u010dka","year":"2023"},{"key":"2024042618135744000_bib41","article-title":"ALBERT: A lite BERT for self-supervised learning of language representations","volume-title":"Proceedings of the Eighth International Conference on Learning Representations (ICLR 2020)","author":"Lan","year":"2020"},{"issue":"3","key":"2024042618135744000_bib42","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1515\/ling-2019-0013","article-title":"Systematicity in the semantics of noun compounds: The role of artifacts vs. natural kinds","volume":"57","author":"Levin","year":"2019","journal-title":"Linguistics"},{"key":"2024042618135744000_bib43","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2024042618135744000_bib44","doi-asserted-by":"publisher","first-page":"717","DOI":"10.18653\/v1\/2022.findings-emnlp.50","article-title":"Systematicity in GPT-3\u2019s interpretation of novel English noun compounds","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2022","author":"Li","year":"2022"},{"key":"2024042618135744000_bib45","doi-asserted-by":"publisher","first-page":"4437","DOI":"10.18653\/v1\/2022.naacl-main.330","article-title":"Testing the ability of language models to interpret figurative language","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Liu","year":"2022"},{"key":"2024042618135744000_bib46","doi-asserted-by":"publisher","first-page":"9053","DOI":"10.18653\/v1\/2022.emnlp-main.617","article-title":"Are representations built from the ground up? An empirical examination of local composition in language models","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2022"},{"key":"2024042618135744000_bib47","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1907.11692","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv: 1907.11692"},{"key":"2024042618135744000_bib48","first-page":"279","article-title":"What do you mean, BERT? Assessing BERT as a distributional semantics model","volume-title":"Proceedings of the Society for Computation in Linguistics 2020","author":"Mickus","year":"2020"},{"key":"2024042618135744000_bib49","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.18653\/v1\/2023.eacl-main.110","article-title":"A systematic search for compound semantics in pretrained BERT architectures","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Mileti\u0107","year":"2023"},{"key":"2024042618135744000_bib50","first-page":"236","article-title":"Vector-based models of semantic composition","volume-title":"Proceedings of ACL-08: HLT","author":"Mitchell","year":"2008"},{"key":"2024042618135744000_bib51","article-title":"LIdioms: A multilingual linked idioms data set","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Moussallem","year":"2018"},{"key":"2024042618135744000_bib52","doi-asserted-by":"publisher","first-page":"27","DOI":"10.18653\/v1\/W19-2004","article-title":"How well do embedding models capture non-compositionality? A view from multiword expressions","volume-title":"Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP","author":"Nandakumar","year":"2019"},{"key":"2024042618135744000_bib53","doi-asserted-by":"publisher","first-page":"57","DOI":"10.18653\/v1\/2021.mwe-1.7","article-title":"Finding BERT\u2019s idiomatic key","volume-title":"Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)","author":"Nedumpozhimana","year":"2021"},{"key":"2024042618135744000_bib54","doi-asserted-by":"publisher","first-page":"813967","DOI":"10.3389\/frai.2022.813967","article-title":"Shapley idioms: Analysing BERT sentence embeddings for general idiom token identification","volume":"5","author":"Nedumpozhimana","year":"2022","journal-title":"Frontiers in Artificial Intelligence"},{"key":"2024042618135744000_bib55","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3115\/1613704.1613712","article-title":"Co-occurrence contexts for noun compound interpretation","volume-title":"Proceedings of the Workshop on A Broader Perspective on Multiword Expressions","author":"S\u00e9aghdha","year":"2007"},{"key":"2024042618135744000_bib56","doi-asserted-by":"publisher","first-page":"425","DOI":"10.3115\/v1\/P15-2070","article-title":"PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Pavlick","year":"2015"},{"key":"2024042618135744000_bib57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2023.eacl-main.1","article-title":"PiC: A phrase-in-context dataset for phrase understanding and semantic search","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Pham","year":"2023"},{"key":"2024042618135744000_bib58","doi-asserted-by":"publisher","first-page":"6509","DOI":"10.18653\/v1\/2020.coling-main.572","article-title":"NYTWIT: A dataset of novel words in the New York Times","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Pinter","year":"2020"},{"key":"2024042618135744000_bib59","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.18653\/v1\/2020.findings-emnlp.138","article-title":"Will it unblend?","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Pinter","year":"2020"},{"key":"2024042618135744000_bib60","article-title":"Language models are unsupervised multitask learners","author":"Radford","year":"2019"},{"issue":"1","key":"2024042618135744000_bib61","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2024042618135744000_bib62","doi-asserted-by":"publisher","first-page":"87","DOI":"10.18653\/v1\/2023.mwe-1.13","article-title":"Are frequent phrases directly retrieved like idioms? An investigation with self-paced reading and language models","volume-title":"Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023)","author":"Rambelli","year":"2023"},{"key":"2024042618135744000_bib63","first-page":"210","article-title":"An empirical study on compositionality in compound nouns","volume-title":"Proceedings of 5th International Joint Conference on Natural Language Processing","author":"Reddy","year":"2011"},{"key":"2024042618135744000_bib64","doi-asserted-by":"publisher","first-page":"3982","DOI":"10.18653\/v1\/D19-1410","article-title":"Sentence-BERT: Sentence embeddings using Siamese BERT-Networks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Reimers","year":"2019"},{"key":"2024042618135744000_bib65","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1162\/tacl_a_00349","article-title":"A primer in BERTology: What we know about how BERT works","volume":"8","author":"Rogers","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618135744000_bib66","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/3-540-45715-1_1","article-title":"Multiword expressions: A pain in the neck for NLP","volume-title":"Computational Linguistics and Intelligent Text Processing","author":"Sag","year":"2002"},{"key":"2024042618135744000_bib67","article-title":"DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter","volume-title":"5th Workshop on Energy Efficient Machine Learning and Cognitive Computing \u2013 NeurIPS 2019","author":"Sanh","year":"2019"},{"key":"2024042618135744000_bib68","doi-asserted-by":"publisher","first-page":"31","DOI":"10.18653\/v1\/W17-1704","article-title":"The PARSEME shared task on automatic identification of verbal multiword expressions","volume-title":"Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017)","author":"Savary","year":"2017"},{"key":"2024042618135744000_bib69","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/978-3-030-58323-1_9","article-title":"EPIE dataset: A corpus for possible idiomatic expressions","volume-title":"Text, Speech, and Dialogue","author":"Saxena","year":"2020"},{"key":"2024042618135744000_bib70","article-title":"Collecting and investigating features of compositionality ratings","volume-title":"Multiword Expressions in Lexical Resources. Linguistic, Lexicographic and Computational Perspectives","author":"Schulte im Walde","year":"."},{"key":"2024042618135744000_bib71","article-title":"Bidirectional attention flow for machine comprehension","volume-title":"Proceedings of the Fifth International Conference on Learning Representations (ICLR 2017)","author":"Seo","year":"2017"},{"key":"2024042618135744000_bib72","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1162\/tacl_a_00277","article-title":"Still a pain in the neck: Evaluating text representations on lexical composition","volume":"7","author":"Shwartz","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618135744000_bib73","doi-asserted-by":"publisher","first-page":"2888","DOI":"10.18653\/v1\/2021.emnlp-main.230","article-title":"Masked language modeling and the distributional hypothesis: Order word matters pre-training for little","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Sinha","year":"2021"},{"key":"2024042618135744000_bib74","doi-asserted-by":"publisher","first-page":"107606","DOI":"10.1016\/j.knosys.2021.107606","article-title":"MICE: Mining idioms with contextual embeddings","volume":"235","author":"\u0160kvorc","year":"2022","journal-title":"Knowledge-Based Systems"},{"key":"2024042618135744000_bib75","first-page":"4368","article-title":"All that glitters is not gold: A gold standard of adjective-noun collocations for German","volume-title":"Proceedings of the Twelfth Language Resources and Evaluation Conference","author":"Strakatova","year":"2020"},{"key":"2024042618135744000_bib76","doi-asserted-by":"publisher","first-page":"234","DOI":"10.18653\/v1\/2022.starsem-1.21","article-title":"Leveraging three types of embeddings from masked language models in idiom token classification","volume-title":"Proceedings of the 11th Joint Conference on Lexical and Computational Semantics","author":"Takahashi","year":"2022"},{"key":"2024042618135744000_bib77","doi-asserted-by":"publisher","first-page":"1312","DOI":"10.18653\/v1\/2020.coling-main.113","article-title":"A BERT-based dual embedding model for Chinese idiom prediction","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Tan","year":"2020"},{"key":"2024042618135744000_bib78","doi-asserted-by":"publisher","first-page":"1397","DOI":"10.26615\/978-954-452-072-4_156","article-title":"Does BERT understand idioms? A probing-based empirical study of BERT encodings of idioms","volume-title":"Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)","author":"Tan","year":"2021"},{"issue":"6","key":"2024042618135744000_bib79","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3453185","article-title":"A BERT-based two-stage model for Chinese chengyu recommendation","volume":"20","author":"Tan","year":"2021","journal-title":"ACM Transactions on Asian and Low-Resource Language Information Processing"},{"key":"2024042618135744000_bib80","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2024042618135744000_bib81","doi-asserted-by":"publisher","first-page":"7222","DOI":"10.18653\/v1\/2020.emnlp-main.586","article-title":"Probing pretrained language models for lexical semantics","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Vuli\u0107","year":"2020"},{"key":"2024042618135744000_bib82","doi-asserted-by":"publisher","first-page":"10837","DOI":"10.18653\/v1\/2021.emnlp-main.846","article-title":"Phrase-BERT: Improved phrase embeddings from BERT with an application to corpus exploration","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Wang","year":"2021"},{"key":"2024042618135744000_bib83","first-page":"161","article-title":"Does BERT make any sense? Interpretable word sense disambiguation with contextualized embeddings","volume-title":"Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers","author":"Wiedemann","year":"2019"},{"key":"2024042618135744000_bib84","article-title":"XLNet: Generalized autoregressive pretraining for language understanding","volume-title":"Advances in Neural Information Processing Systems","author":"Yang","year":"2019"},{"key":"2024042618135744000_bib85","doi-asserted-by":"publisher","first-page":"4896","DOI":"10.18653\/v1\/2020.emnlp-main.397","article-title":"Assessing phrasal representation and composition in Transformers","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Lang","year":"2020"},{"key":"2024042618135744000_bib86","doi-asserted-by":"publisher","first-page":"2279","DOI":"10.18653\/v1\/2021.findings-acl.201","article-title":"On the interplay between fine-tuning and composition in Transformers","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Lang","year":"2021"},{"key":"2024042618135744000_bib87","doi-asserted-by":"publisher","first-page":"1546","DOI":"10.1162\/tacl_a_00442","article-title":"Idiomatic expression identification using semantic compatibility","volume":"9","author":"Zeng","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024042618135744000_bib88","doi-asserted-by":"publisher","first-page":"1120","DOI":"10.1162\/tacl_a_00510","article-title":"Getting BART to ride the idiomatic train: Learning to represent idiomatic expressions","volume":"10","author":"Zeng","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00657\/2367417\/tacl_a_00657.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00657\/2367417\/tacl_a_00657.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,26]],"date-time":"2024-04-26T18:14:27Z","timestamp":1714155267000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00657\/120831\/Semantics-of-Multiword-Expressions-in-Transformer"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":88,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00657","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}