{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T13:58:35Z","timestamp":1781704715314,"version":"3.54.5"},"reference-count":58,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":167,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>While existing concreteness norms primarily target words in isolation, little attention has been paid to concreteness in context. To address this, we systematically collect multilingual concreteness ratings using Best-Worst Scaling (BWS) for 5,814 verb-direct object noun expressions in three languages with different degrees of resource availability: English, German, and Slovene. We identify consistent patterns where the concreteness of verb-noun combinations is more strongly influenced by the nominal object than the verb. Through comparative analyses on an English subset, we demonstrate that BWS guarantees more reliable concreteness judgments than traditional rating scales. Expanding beyond our human-generated data, we use traditional and LLM-based automatic extrapolation methods to generate a large-scale multilingual resource of over 430,000 expressions. Additionally, we conduct a study examining the interaction between concreteness and literal vs. figurative judgments for a subset of 1,800 expressions in all three languages, along with example usage sentences. Our findings show that lower concreteness ratings correlate with figurative language, thus reinforcing the link between abstractness and figurativeness. All resources are available from https:\/\/github.com\/urbikn\/multilingual-concreteness-vo.<\/jats:p>","DOI":"10.1162\/tacl.a.703","type":"journal-article","created":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T13:28:12Z","timestamp":1781702892000},"page":"1205-1224","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Literally Concrete or Figuratively Abstract? Multilingual Concreteness Norms for Verb-Object Expressions"],"prefix":"10.1162","volume":"14","author":[{"given":"Urban","family":"Knuple\u0161","sequence":"first","affiliation":[{"name":"Institute for Natural Language Processing, University of Stuttgart, urban.knuples@ims.uni-stuttgart.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diego","family":"Frassinelli","sequence":"additional","affiliation":[{"name":"Center for Information and Language Processing, LMU Munich, frassinelli@cis.lmu.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Fraser","sequence":"additional","affiliation":[{"name":"School of Computation, Information and Technology, TU Munich, alexander.fraser@tum.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sabine Schulte im","family":"Walde","sequence":"additional","affiliation":[{"name":"Institute for Natural Language Processing, University of Stuttgart, schulte@ims.uni-stuttgart.de"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2026,6,1]]},"reference":[{"key":"2026061709280569000_bib1","doi-asserted-by":"crossref","first-page":"204","DOI":"10.18653\/v1\/2020.figlang-1.28","article-title":"Augmenting neural metaphor detection with concreteness","volume-title":"Proceedings of the 2nd Workshop on Figurative Language Processing","author":"Alnafesah","year":"2020"},{"key":"2026061709280569000_bib2","first-page":"7924","article-title":"\u201cYou are an expert annotator\u201d: Automatic best\u2013worst-scaling annotations for emotion intensity modeling","volume-title":"Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Bagdon","year":"2024"},{"key":"2026061709280569000_bib3","doi-asserted-by":"crossref","first-page":"2366","DOI":"10.3758\/s13428-018-1014-y","article-title":"Concreteness norms for 1,659 French words: Relationships with other psycholinguistic variables and word recognition times","volume":"50","author":"Bonin","year":"2018","journal-title":"Behavior Research Methods"},{"key":"2026061709280569000_bib4","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.actpsy.2014.04.010","article-title":"Norms of age of acquisition and concreteness for 30,000 Dutch words","volume":"150","author":"Brysbaert","year":"2014","journal-title":"Acta Psychologica"},{"key":"2026061709280569000_bib5","doi-asserted-by":"crossref","first-page":"904","DOI":"10.3758\/s13428-013-0403-5","article-title":"Concreteness ratings for 40 thousand generally known English word lemmas","volume":"64","author":"Brysbaert","year":"2014","journal-title":"Behavior Research Methods"},{"issue":"1752","key":"2026061709280569000_bib6","doi-asserted-by":"crossref","first-page":"20170131","DOI":"10.1098\/rstb.2017.0131","article-title":"A review of abstract concept learning in embodied agents and robots","volume":"373","author":"Cangelosi","year":"2018","journal-title":"Philosophical Transactions of the Royal Society B: Biological Sciences"},{"key":"2026061709280569000_bib7","doi-asserted-by":"crossref","first-page":"6788","DOI":"10.18653\/v1\/2020.coling-main.598","article-title":"German\u2019s next language model","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Chan","year":"2020"},{"key":"2026061709280569000_bib8","article-title":"Predicting the concreteness of German words","volume-title":"Proceedings of the 5th Swiss Text Analytics Conference (SwissText) & 16th Conference on Natural Language Processing (KONVENS), CEUR Workshop Proceedings Vol. 2624","author":"Charbonnier","year":"2020"},{"issue":"1","key":"2026061709280569000_bib9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5334\/joc.482","article-title":"Updating the German psycholinguistic word toolbox with AI-generated estimates of concreteness, valence, arousal, age of acquisition, and familiarity","volume":"9","author":"Conde","year":"2026","journal-title":"Journal of Cognition"},{"key":"2026061709280569000_bib10","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1007\/978-3-642-40722-2_6","article-title":"SdeWaC \u2013 A corpus of parsable sentences from the web","volume-title":"Language Processing and Knowledge in the Web","author":"Faa\u00df","year":"2013"},{"key":"2026061709280569000_bib11","first-page":"38","article-title":"Distributional interaction of concreteness and abstractness in verb\u2013noun subcategorisation","volume-title":"Proceedings of the 13th International Conference on Computational Semantics","author":"Frassinelli","year":"2019"},{"key":"2026061709280569000_bib12","volume-title":"CONcreTEXT @ EVALITA2020: The concreteness in context task","author":"Gregori","year":"2020"},{"key":"2026061709280569000_bib13","first-page":"22","article-title":"Cross-lingual metaphor detection for low- to high-resource languages","volume-title":"Proceedings of the 4th Workshop on Figurative Language Processing","author":"H\u00fclsing","year":"2024"},{"key":"2026061709280569000_bib14","first-page":"287","article-title":"Do as I can, not as I say: Grounding language in robotic affordances","volume-title":"Proceedings of the 6th Conference on Robot Learning","author":"Ichter","year":"2023"},{"issue":"4","key":"2026061709280569000_bib15","doi-asserted-by":"crossref","first-page":"987","DOI":"10.3758\/BRM.42.4.987","article-title":"Leipzig affective norms for German: A reliability study","volume":"42","author":"Kanske","year":"2010","journal-title":"Behavior Research Methods"},{"issue":"8","key":"2026061709280569000_bib16","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1080\/17470218.2015.1051065","article-title":"Megastudies, crowdsourcing, and large datasets in psycholinguistics: An overview of recent developments","volume":"68","author":"Keuleers","year":"2015","journal-title":"Quarterly Journal of Experimental Psychology (2006)"},{"key":"2026061709280569000_bib17","first-page":"15","article-title":"Comparison of image generation models for abstract and concrete event descriptions","volume-title":"Proceedings of the 4th Workshop on Figurative Language Processing","author":"Khaliq","year":"2024"},{"key":"2026061709280569000_bib18","doi-asserted-by":"crossref","first-page":"465","DOI":"10.18653\/v1\/P17-2074","article-title":"Best-worst scaling more reliable than rating scales: A case study on sentiment intensity annotation","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Kiritchenko","year":"2017"},{"key":"2026061709280569000_bib19","doi-asserted-by":"crossref","first-page":"70","DOI":"10.18653\/v1\/2023.conll-1.6","article-title":"Investigating the nature of disagreements on mid-scale ratings: A case study on the abstractness-concreteness continuum","volume-title":"Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)","author":"Knuple\u0161","year":"2023"},{"key":"2026061709280569000_bib20","first-page":"353","article-title":"Distinguishing literal and non-literal usage of German particle verbs","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"K\u00f6per","year":"2016"},{"key":"2026061709280569000_bib21","first-page":"2595","article-title":"Automatically generated affective norms of abstractness, arousal, imageability and valence for 350 000 German lemmas","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation","author":"K\u00f6per","year":"2016"},{"issue":"1","key":"2026061709280569000_bib22","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3758\/BRM.41.1.13","article-title":"Using the World-Wide Web to obtain large-scale word norms: 190,212 ratings on a set of 2,654 German nouns","volume":"41","author":"Lahl","year":"2009","journal-title":"Behavior Research Methods"},{"key":"2026061709280569000_bib23","volume-title":"Metaphors We Live By","author":"Lakoff","year":"1980"},{"key":"2026061709280569000_bib24","article-title":"Roberta: A robustly optimized BERT pretraining approach","volume-title":"CoRR, abs\/1907.11692","author":"Liu","year":"2019"},{"key":"2026061709280569000_bib25","doi-asserted-by":"crossref","first-page":"217","DOI":"10.18653\/v1\/W18-3028","article-title":"Predicting concreteness and imageability of words within and across languages via word embeddings","volume-title":"Proceedings of the Third Workshop on Representation Learning for NLP","author":"Ljube\u0161i\u0107","year":"2018"},{"key":"2026061709280569000_bib26","first-page":"3271","article-title":"CLASSLA-web: Comparable web corpora of South Slavic languages enriched with linguistic and genre annotation","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation","author":"Ljube\u0161i\u0107","year":"2024"},{"key":"2026061709280569000_bib27","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107337855","volume-title":"Best-Worst Scaling: Theory, Methods and Applications","author":"Louviere","year":"2015"},{"key":"2026061709280569000_bib28","volume-title":"Best-worst scaling: A model for the largest difference judgments","author":"Louviere","year":"1991"},{"issue":"8","key":"2026061709280569000_bib29","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1080\/17470218.2014.988735","article-title":"How useful are corpus-based methods for extrapolating psycholinguistic variables?","volume":"68","author":"Mandera","year":"2015","journal-title":"Quarterly Journal of Experimental Psychology"},{"issue":"1","key":"2026061709280569000_bib30","first-page":"1","article-title":"Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal","volume":"57","author":"Mart\u00ednez","year":"2025","journal-title":"Behavior Research Methods"},{"key":"2026061709280569000_bib31","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.procs.2025.02.260","article-title":"EuroLLM: Multilingual language models for Europe","volume":"255","author":"Martins","year":"2025","journal-title":"Procedia Computer Science"},{"key":"2026061709280569000_bib32","doi-asserted-by":"crossref","first-page":"221","DOI":"10.18653\/v1\/2020.figlang-1.30","article-title":"Metaphor detection using context and concreteness","volume-title":"Proceedings of the 2nd Workshop on Figurative Language Processing","author":"Maudslay","year":"2020"},{"issue":"3","key":"2026061709280569000_bib33","doi-asserted-by":"crossref","first-page":"887","DOI":"10.3758\/s13428-013-0405-3","article-title":"The adaptation of the affective norms for English words (ANEW) for Italian","volume":"46","author":"Montefinese","year":"2014","journal-title":"Behavior Research Methods"},{"issue":"10","key":"2026061709280569000_bib34","doi-asserted-by":"crossref","first-page":"e0293031","DOI":"10.1371\/journal.pone.0293031","article-title":"CONcre-TEXT norms: Concreteness ratings for Italian and English words in context","volume":"18","author":"Montefinese","year":"2023","journal-title":"PLOS ONE"},{"issue":"5","key":"2026061709280569000_bib35","doi-asserted-by":"crossref","first-page":"2522","DOI":"10.3758\/s13428-022-01912-6","article-title":"Concreteness ratings for 62,000 English multiword expressions","volume":"55","author":"Muraki","year":"2023","journal-title":"Behavior Research Methods"},{"key":"2026061709280569000_bib36","volume-title":"Maxdiff analysis: Simple counting, individual-level logit, and HB","author":"Orme","year":"2009"},{"issue":"1","key":"2026061709280569000_bib37","article-title":"Addressing the elephant in the middle: Implications of the midscale disagreement problem through the lens of body-object interaction ratings","volume":"9","author":"Paisios","year":"2023","journal-title":"Collabra: Psychology"},{"issue":"4","key":"2026061709280569000_bib38","doi-asserted-by":"crossref","first-page":"1799","DOI":"10.3758\/s13428-020-01533-x","article-title":"The Croatian psycholinguistic database: Estimates for 6000 nouns, verbs, adjectives and adverbs","volume":"53","author":"Peti-Stanti\u0107","year":"2021","journal-title":"Behavior Research Methods"},{"key":"2026061709280569000_bib39","doi-asserted-by":"crossref","DOI":"10.63317\/2prfsvwba97g","article-title":"Features of perceived metaphoricity on the discourse level: Abstractness and emotionality","volume-title":"Proceedings of the 13th International Conference on Language Resources and Evaluation","author":"Piccirilli","year":"2022"},{"key":"2026061709280569000_bib40","doi-asserted-by":"crossref","first-page":"299","DOI":"10.18653\/v1\/2022.starsem-1.26","article-title":"What Drives the use of metaphorical language? Negative insights from abstractness, affect, discourse coherence and contextualized word representations","volume-title":"Proceedings of the 11th Joint Conference on Lexical and Computational Semantics","author":"Piccirilli","year":"2022"},{"issue":"3","key":"2026061709280569000_bib41","doi-asserted-by":"crossref","first-page":"1198","DOI":"10.3758\/s13428-017-0938-y","article-title":"Statistical and methodological problems with concreteness and other semantic variables: A list memory experiment case study","volume":"50","author":"Pollock","year":"2018","journal-title":"Behavior Research Methods"},{"issue":"1","key":"2026061709280569000_bib42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/10926480709336752","article-title":"MIP: A method for identifying metaphorically used words in discourse","volume":"22","author":"Pragglejaz Group","year":"2007","journal-title":"Metaphor and Symbol"},{"key":"2026061709280569000_bib43","doi-asserted-by":"crossref","DOI":"10.3758\/s13428-023-02257-4","article-title":"Concreteness ratings for 36,000 Estonian words","volume-title":"Behavior Research Methods","author":"Proos","year":"2023"},{"issue":"3","key":"2026061709280569000_bib44","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1007\/s10462-017-9608-9","article-title":"Extension of grounding mechanism for abstract words: Computational methods insights","volume":"50","author":"Rasheed","year":"2018","journal-title":"Artificial Intelligence Review"},{"key":"2026061709280569000_bib45","first-page":"28","article-title":"Processing and querying large web corpora with the COW14 architecture","volume-title":"Proceedings of the 3rd Workshop on Challenges in the Management of Large Corpora","author":"Sch\u00e4fer","year":"2015"},{"key":"2026061709280569000_bib46","article-title":"A compact but linguistically detailed database for German verb subcategorisation relying on dependency parses from web corpora: Tool, guidelines and resource","volume-title":"Proceedings of the 8th Web as Corpus Workshop","author":"Scheible","year":"2013"},{"key":"2026061709280569000_bib47","first-page":"22","article-title":"Assessing meaning components in German complex verbs: A collection of source\u2013target domains and directionality","volume-title":"Proceedings of the 7th Joint Conference on Lexical and Computational Semantics","author":"im Walde","year":"2018"},{"key":"2026061709280569000_bib48","first-page":"3255","article-title":"Metaphor corpus annotated for source\u2013target domain mappings","volume-title":"Proceedings of the 7th International Conference on Language Resources and Evaluation","author":"Shutova","year":"2010"},{"key":"2026061709280569000_bib49","volume-title":"Non-parametric Statistics for the Behavioral Sciences","author":"Siegel","year":"1988"},{"key":"2026061709280569000_bib50","article-title":"CLASSLA-Stanza: The next step for linguistic processing of South Slavic languages","volume-title":"ArXiv preprint arXiv:2308.04255","author":"Ter\u010don","year":"2023"},{"key":"2026061709280569000_bib51","volume-title":"The falcon 3 family of open models","author":"Team","year":"2024"},{"key":"2026061709280569000_bib52","first-page":"248","article-title":"Metaphor detection with cross-lingual model transfer","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics","author":"Tsvetkov","year":"2014"},{"key":"2026061709280569000_bib53","first-page":"45","article-title":"Cross-lingual metaphor detection using common semantic features","volume-title":"Proceedings of the First Workshop on Metaphor in NLP","author":"Tsvetkov","year":"2013"},{"key":"2026061709280569000_bib54","first-page":"680","article-title":"Literal and metaphorical sense identification through concrete and abstract context","volume-title":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing","author":"Turney","year":"2011"},{"issue":"4","key":"2026061709280569000_bib55","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1145\/944012.944013","article-title":"Measuring praise and criticism: Inference of semantic orientation from association","volume":"21","author":"Turney","year":"2003","journal-title":"ACM Transactions on Information Systems"},{"key":"2026061709280569000_bib56","first-page":"17","article-title":"SloBERTa: Slovene monolingual large pre-trained masked language model","volume-title":"Proceedings of Data Mining and Data Warehousing, SiKDD","author":"Ul\u010dar","year":"2021"},{"key":"2026061709280569000_bib57","first-page":"485","article-title":"Generative model for less-resourced language with 1 billion parameters","volume-title":"Language Technologies and Digital Humanities Conference","author":"Vre\u0161","year":"2024"},{"issue":"4","key":"2026061709280569000_bib58","doi-asserted-by":"crossref","first-page":"1374","DOI":"10.3758\/s13428-016-0793-2","article-title":"Norms of valence, arousal, concreteness, familiarity, imageability, and context availability for 1,100 Chinese words","volume":"49","author":"Yao","year":"2017","journal-title":"Behavior Research Methods"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/TACL.a.703\/2607046\/tacl.a.703.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/TACL.a.703\/2607046\/tacl.a.703.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T13:28:23Z","timestamp":1781702903000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/TACL.a.703\/137200\/Literally-Concrete-or-Figuratively-Abstract"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":58,"URL":"https:\/\/doi.org\/10.1162\/tacl.a.703","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]}}}