{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T05:08:47Z","timestamp":1790744927897,"version":"4.1.0"},"reference-count":160,"publisher":"MIT Press","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2017,12]]},"abstract":"<jats:p>We introduce HyperLex\u2014a data set and evaluation resource that quantifies the extent of the semantic category membership, that is, type-of relation, also known as hyponymy\u2013hypernymy or lexical entailment (LE) relation between 2,616 concept pairs. Cognitive psychology research has established that typicality and category\/class membership are computed in human semantic memory as a gradual rather than binary relation. Nevertheless, most NLP research and existing large-scale inventories of concept category membership (WordNet, DBPedia, etc.) treat category membership and LE as binary. To address this, we asked hundreds of native English speakers to indicate typicality and strength of category membership between a diverse range of concept pairs on a crowdsourcing platform. Our results confirm that category membership and LE are indeed more gradual than binary. We then compare these human judgments with the predictions of automatic systems, which reveals a huge gap between human performance and state-of-the-art LE, distributional and representation learning models, and substantial differences between the models themselves. We discuss a pathway for improving semantic models to overcome this discrepancy, and indicate future application areas for improved graded LE systems.<\/jats:p>","DOI":"10.1162\/coli_a_00301","type":"journal-article","created":{"date-parts":[[2017,9,11]],"date-time":"2017-09-11T15:00:35Z","timestamp":1505142035000},"page":"781-835","source":"Crossref","is-referenced-by-count":35,"title":["HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment"],"prefix":"10.1162","volume":"43","author":[{"given":"Ivan","family":"Vuli\u0107","sequence":"first","affiliation":[{"name":"Language Technology Lab (LTL), Department of Theoretical and Applied Linguistics, University of Cambridge, 9 West Road, CB3 9DP Cambridge, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniela","family":"Gerz","sequence":"additional","affiliation":[{"name":"Language Technology Lab (LTL), Department of Theoretical and Applied Linguistics, University of Cambridge, 9 West Road, CB3 9DP Cambridge, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Douwe","family":"Kiela","sequence":"additional","affiliation":[{"name":"Facebook AI Research, 770 Broadway, NY 10003, New York City, NY, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Hill","sequence":"additional","affiliation":[{"name":"Google DeepMind, 7 Pancras Square, London NC14AG, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Korhonen","sequence":"additional","affiliation":[{"name":"Language Technology Lab (LTL), Department of Theoretical and Applied Linguistics, University of Cambridge, 9 West Road, CB3 9DP Cambridge, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"bib1","unstructured":"Agi\u0107, \u017deljko and Natalie Schluter. 2017. Baselines and test data for cross-lingual inference. CoRR, abs\/1704.05347."},{"key":"bib2","doi-asserted-by":"crossref","unstructured":"Agirre, Eneko, Enrique Alfonseca, Keith B. Hall, Jana Kravalova, Marius Pasca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and WordNet-based approaches. In Proceedings of NAACL-HLT, pages 19\u201327, Boulder, CO.","DOI":"10.3115\/1620754.1620758"},{"key":"bib3","unstructured":"Al-Rfou, Rami, Bryan Perozzi, and Steven Skiena. 2013. Polyglot: Distributed word representations for multilingual NLP. In Proceedings of CoNLL, pages 183\u2013192, Sofia."},{"key":"bib4","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2985"},{"key":"bib5","doi-asserted-by":"crossref","unstructured":"Astudillo, Ram\u00f3n, Silvio Amir, Wang Ling, Mario Silva, and Isabel Trancoso. 2015. Learning word representations from scarce and noisy data with embedding subspaces. In Proceedings of ACL, pages 1074\u20131084, Beijing.","DOI":"10.3115\/v1\/P15-1104"},{"key":"bib6","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"bib7","unstructured":"Bankova, Desislava, Bob Coecke, Martha Lewis, and Daniel Marsden. 2016. Graded entailment for compositional distributional semantics. CoRR, abs\/1601.04908."},{"key":"bib9","unstructured":"Baroni, Marco, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of EACL, pages 23\u201332, Avignon."},{"key":"bib10","doi-asserted-by":"crossref","unstructured":"Baroni, Marco, Georgiana Dinu, and Germ\u00e1n Kruszewski. 2014. Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of ACL, pages 238\u2013247, Baltimore, MD.","DOI":"10.3115\/v1\/P14-1023"},{"key":"bib11","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00016"},{"key":"bib12","unstructured":"Baroni, Marco and Alessandro Lenci. 2011. How we BLESSed distributional semantic evaluation. In Proceedings of the Workshop on Geometrical Models of Natural Language Semantics (GEMS), pages 1\u201310, Edinburgh."},{"key":"bib13","doi-asserted-by":"crossref","unstructured":"Batchkarov, Miroslav, Thomas Kober, Jeremy Reffin, Julie Weeds, and David Weir. 2016. A critique of word similarity as a method for evaluating distributional semantic models. In Proceedings of REPEVAL, pages 7\u201312, Berlin.","DOI":"10.18653\/v1\/W16-2502"},{"key":"bib14","unstructured":"Beckwith, Richard, Christiane Fellbaum, Derek Gross, and George A. Miller. 1991. WordNet: A lexical database organized on psycholinguistic principles. Lexical acquisition: Exploiting on-line resources to build a lexicon, pages 211\u2013231."},{"key":"bib16","unstructured":"Beltagy, Islam, Cuong Chau, Gemma Boleda, Dan Garrette, Katrin Erk, and Raymond Mooney. 2013. Montague meets Markov: Deep semantics with probabilistic logical form. In Proceedings of *SEM, pages 11\u201321, Atlanta, GA."},{"key":"bib18","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4900"},{"key":"bib19","unstructured":"Biran, Or and Kathleen McKeown. 2013. Classifying taxonomic relations between pairs of Wikipedia articles. In Proceedings of IJCNLP, pages 788\u2013794, Nagoya."},{"key":"bib20","doi-asserted-by":"crossref","unstructured":"Blutner, Reinhard, Emmanuel M. Pothos, and Peter Bruza. 2013. A quantum probability perspective on borderline vagueness. Topics in Cognitive Science, 5(4):711\u2013736.","DOI":"10.1111\/tops.12041"},{"key":"bib21","doi-asserted-by":"crossref","unstructured":"Bordes, Antoine, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Proceedings of AAAI, pages 301\u2013306, San Francisco, CA.","DOI":"10.1609\/aaai.v25i1.7917"},{"key":"bib22","doi-asserted-by":"crossref","unstructured":"Bos, Johan and Katja Markert. 2005. Recognising textual entailment with logical inference. In Proceedings of EMNLP, pages 628\u2013635, Vancouver.","DOI":"10.3115\/1220575.1220654"},{"key":"bib23","doi-asserted-by":"crossref","unstructured":"Bowman, Samuel R., Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of EMNLP, pages 632\u2013642, Lisbon.","DOI":"10.18653\/v1\/D15-1075"},{"key":"bib24","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4135"},{"key":"bib25","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193020"},{"key":"bib27","doi-asserted-by":"crossref","unstructured":"Camacho-Collados, Jos\u00e9, Mohammad Taher Pilehvar, and Roberto Navigli. 2015. A framework for the construction of monolingual and cross-lingual word similarity datasets. In Proceedings of ACL, pages 1\u20137, Beijing.","DOI":"10.3115\/v1\/P15-2001"},{"key":"bib28","doi-asserted-by":"publisher","DOI":"10.3115\/1705415.1705430"},{"key":"bib29","doi-asserted-by":"publisher","DOI":"10.1353\/lan.1981.0002"},{"key":"bib30","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-5371(69)80069-1"},{"key":"bib31","unstructured":"Collins, Allan M. and Ross M. Quillian. 1972. Experiments on semantic memory and language comprehension. Cognition in Learning and Memory."},{"key":"bib32","unstructured":"Dagan, Ido, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges, pages 177\u2013190."},{"key":"bib33","doi-asserted-by":"publisher","DOI":"10.2200\/S00509ED1V01Y201305HLT023"},{"key":"bib34","doi-asserted-by":"publisher","DOI":"10.1111\/nous.12003"},{"key":"bib35","doi-asserted-by":"crossref","unstructured":"Deng, Jia, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of CVPR, pages 248\u2013255, Miami, FL.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"bib36","doi-asserted-by":"crossref","unstructured":"Deselaers, Thomas and Vittorio Ferrari. 2011. Visual and semantic similarity in ImageNet. In Proceedings of CVPR, pages 1777\u20131784, Colorado Springs, CO.","DOI":"10.1109\/CVPR.2011.5995474"},{"key":"bib38","doi-asserted-by":"publisher","DOI":"10.1515\/cog-2013-0008"},{"key":"bib39","unstructured":"Do, Quang and Dan Roth. 2010. Constraints based taxonomic relation classification. In Proceedings of EMNLP, pages 1099\u20131109, Cambridge, MA."},{"key":"bib40","doi-asserted-by":"publisher","DOI":"10.1016\/j.fss.2011.09.005"},{"key":"bib41","unstructured":"Evert, Stefan. 2008. Corpora and collocations. Corpus Linguistics, 2:223\u2013233."},{"key":"bib42","doi-asserted-by":"crossref","unstructured":"Faruqui, Manaal, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015. Retrofitting word vectors to semantic lexicons. In Proceedings of NAACL-HLT, pages 1606\u20131615, Denver, CO.","DOI":"10.3115\/v1\/N15-1184"},{"key":"bib43","doi-asserted-by":"crossref","unstructured":"Faruqui, Manaal and Chris Dyer. 2015. Non-distributional word vector representations. In Proceedings of ACL, pages 464\u2013469, Beijing.","DOI":"10.3115\/v1\/P15-2076"},{"key":"bib44","doi-asserted-by":"crossref","unstructured":"Faruqui, Manaal, Yulia Tsvetkov, Pushpendre Rastogi, and Chris Dyer. 2016. Problems with evaluation of word embeddings using word similarity tasks. In Proceedings of REPEVAL, pages 30\u201335, Berlin.","DOI":"10.18653\/v1\/W16-2506"},{"key":"bib46","doi-asserted-by":"publisher","DOI":"10.1145\/503104.503110"},{"key":"bib48","doi-asserted-by":"crossref","unstructured":"Fu, Ruiji, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Learning semantic hierarchies via word embeddings. In Proceedings of ACL, pages 1199\u20131209, Baltimore, MD.","DOI":"10.3115\/v1\/P14-1113"},{"key":"bib49","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2014.2377580"},{"key":"bib51","doi-asserted-by":"publisher","DOI":"10.3115\/1219840.1219854"},{"key":"bib52","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog0702_3"},{"key":"bib54","doi-asserted-by":"crossref","unstructured":"Gerz, Daniela, Ivan Vuli\u0107, Felix Hill, Roi Reichart, and Anna Korhonen. 2016. SimVerb-3500: A large-scale evaluation set of verb similarity. In Proceedings of EMNLP, pages 2173\u20132182, Austin, TX.","DOI":"10.18653\/v1\/D16-1235"},{"key":"bib55","unstructured":"Gheorghita, Inga and Jean-Marie Pierrel. 2012. Towards a methodology for automatic identification of hypernyms in the definitions of large-scale dictionary. In Proceedings of LREC, pages 2614\u20132618, Istanbul."},{"key":"bib56","doi-asserted-by":"crossref","unstructured":"Gladkova, Anna and Aleksandr Drozd. 2016. Intrinsic evaluations of word embeddings: What can we do better? In Proceedings of REPEVAL, pages 36\u201342, Berlin.","DOI":"10.18653\/v1\/W16-2507"},{"key":"bib57","unstructured":"Gulordava, Kristina and Marco Baroni. 2011. A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. In Proceedings of the GEMS Workshop on GEometrical Models of Natural Language Semantics, pages 67\u201371, Edinburgh."},{"key":"bib58","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-7421(06)46003-5"},{"key":"bib59","doi-asserted-by":"publisher","DOI":"10.1080\/15326900701326402"},{"key":"bib60","doi-asserted-by":"publisher","DOI":"10.1080\/00437956.1954.11659520"},{"key":"bib61","doi-asserted-by":"crossref","unstructured":"Hashimoto, Tatsunori B., David Alvarez-Melis, and Tommi S. Jaakkola. 2016. Word embeddings as metric recovery in semantic spaces. Transactions of the ACL, 4:273\u2013286.","DOI":"10.1162\/tacl_a_00098"},{"key":"bib62","doi-asserted-by":"publisher","DOI":"10.1145\/2806416.2806502"},{"key":"bib63","doi-asserted-by":"crossref","unstructured":"Hearst, Marti A. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of COLING, pages 539\u2013545, Nantes.","DOI":"10.3115\/992133.992154"},{"key":"bib64","doi-asserted-by":"crossref","unstructured":"Hendrickx, Iris, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid \u00d3 S\u00e9aghdha, Sebastian Pad\u00f3, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010. SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of SEMEVAL, pages 33\u201338, Uppsala.","DOI":"10.3115\/1621969.1621986"},{"key":"bib65","unstructured":"Herbelot, Aur\u00e9lie and Mohan Ganesalingam. 2013. Measuring semantic content in distributional vectors. In Proceedings of ACL, pages 440\u2013445, Sofia."},{"key":"bib66","doi-asserted-by":"publisher","DOI":"10.1111\/cogs.12076"},{"key":"bib67","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00237"},{"key":"bib68","doi-asserted-by":"publisher","DOI":"10.1613\/jair.3994"},{"key":"bib71","doi-asserted-by":"crossref","unstructured":"Jauhar, Sujay Kumar, Chris Dyer, and Eduard Hovy. 2015. Ontologically grounded multi-sense representation learning for semantic vector space models. In Proceedings of NAACL-HLT, pages 683\u2013693, Denver, CO.","DOI":"10.3115\/v1\/N15-1070"},{"key":"bib72","doi-asserted-by":"crossref","unstructured":"Jia, Yangqing, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross B. Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of ACM Multimedia, pages 675\u2013678, Orlando, FL.","DOI":"10.1145\/2647868.2654889"},{"key":"bib73","unstructured":"Jurgens, David, Saif Mohammad, Peter Turney, and Keith Holyoak. 2012. SemEval-2012 task 2: Measuring degrees of relational similarity. In Proceedings of SEMEVAL, pages 356\u2013364, Montreal."},{"key":"bib74","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(94)00659-9"},{"key":"bib75","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1645960"},{"key":"bib76","doi-asserted-by":"crossref","unstructured":"Kiela, Douwe and L\u00e9on Bottou. 2014. Learning image embeddings using convolutional neural networks for improved multi-modal semantics. In Proceedings of EMNLP, pages 36\u201345, Doha.","DOI":"10.3115\/v1\/D14-1005"},{"key":"bib77","doi-asserted-by":"crossref","unstructured":"Kiela, Douwe, Felix Hill, and Stephen Clark. 2015. Specializing word embeddings for similarity or relatedness. In Proceedings of EMNLP, pages 2044\u20132048, Lisbon.","DOI":"10.18653\/v1\/D15-1242"},{"key":"bib78","doi-asserted-by":"crossref","unstructured":"Kiela, Douwe, Felix Hill, Anna Korhonen, and Stephen Clark. 2014. Improving multi-modal representations using image dispersion: Why less is sometimes more. In Proceedings of ACL, pages 835\u2013841, Baltimore, MD.","DOI":"10.3115\/v1\/P14-2135"},{"key":"bib79","doi-asserted-by":"crossref","unstructured":"Kiela, Douwe, Laura Rimell, Ivan Vuli\u0107, and Stephen Clark. 2015. Exploiting image generality for lexical entailment detection. In Proceedings of ACL, pages 119\u2013124, Beijing.","DOI":"10.3115\/v1\/P15-2020"},{"key":"bib80","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-007-9048-2"},{"key":"bib81","doi-asserted-by":"crossref","unstructured":"Kopliku, Arlind, Karen Pinel-Sauvagnat, and Mohand Boughanem. 2011. Retrieving attributes using Web tables. In Proceedings of JCDL, pages 397\u2013398, Ottawa.","DOI":"10.1145\/1998076.1998153"},{"key":"bib82","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2010.0039"},{"key":"bib83","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324910000124"},{"key":"bib84","unstructured":"Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of NIPS, pages 1106\u20131114, South Lake Tahoe, NV."},{"key":"bib85","unstructured":"Lake, Brenden M., Ruslan Salakhutdinov, Jason Gross, and Joshua B. Tenenbaum. 2011. One shot learning of simple visual concepts. In Proceedings of CogSci, pages 2568\u20132573, Boston, MA."},{"key":"bib87","unstructured":"Nghia2015. Combining language and vision with a multimodal skip-gram model. In Proceedings of NAACL-HLT, pages 153\u2013163, Denver, CO."},{"key":"bib88","doi-asserted-by":"crossref","unstructured":"Leacock, Claudia and Martin Chodorow. 1998. Combining local context and WordNet similarity for word sense identification. WordNet: An electronic lexical database, 49(2):265\u2013283.","DOI":"10.7551\/mitpress\/7287.003.0018"},{"key":"bib89","unstructured":"Lenci, Alessandro and Giulia Benotto. 2012. Identifying hypernyms in distributional semantic spaces. In Proceedings of *SEM, pages 75\u201379, Montreal."},{"key":"bib90","unstructured":"Leviant, Ira and Roi Reichart. 2015. Separated by an un-common language: Towards judgment language informed vector space modeling. CoRR, abs\/1508.00106."},{"key":"bib92","doi-asserted-by":"crossref","unstructured":"Levy, Omer, Ido Dagan, and Jacob Goldberger. 2014. Focused entailment graphs for open IE propositions. In Proceedings of CoNLL, pages 87\u201397, Baltimore, MD.","DOI":"10.3115\/v1\/W14-1610"},{"key":"bib93","doi-asserted-by":"crossref","unstructured":"Levy, Omer and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of ACL, pages 302\u2013308, Baltimore, MD.","DOI":"10.3115\/v1\/P14-2050"},{"key":"bib94","doi-asserted-by":"crossref","unstructured":"Levy, Omer, Steffen Remus, Chris Biemann, and Ido Dagan. 2015. Do supervised distributional methods really learn lexical inference relations? In Proceedings of NAACL-HLT, pages 970\u2013976, Denver, CO.","DOI":"10.3115\/v1\/N15-1098"},{"key":"bib95","doi-asserted-by":"crossref","unstructured":"Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. In Proceedings of ACL, pages 768\u2013774, Montreal.","DOI":"10.3115\/980432.980696"},{"key":"bib96","doi-asserted-by":"crossref","unstructured":"Lin, Yankai, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of AAAI, pages 2181\u20132187, Austin, TX.","DOI":"10.1609\/aaai.v29i1.9491"},{"key":"bib97","doi-asserted-by":"crossref","unstructured":"Liu, Quan, Hui Jiang, Si Wei, Zhen-Hua Ling, and Yu Hu. 2015. Learning semantic word embeddings based on ordinal knowledge constraints. In Proceedings of ACL, pages 1501\u20131511, Beijing.","DOI":"10.3115\/v1\/P15-1145"},{"key":"bib98","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339754"},{"key":"bib99","unstructured":"Luong, Thang, Richard Socher, and Christopher Manning. 2013. Better word representations with recursive neural networks for morphology. In Proceedings of CoNLL, pages 104\u2013113, Sofia."},{"key":"bib100","doi-asserted-by":"publisher","DOI":"10.1037\/0278-7393.23.1.54"},{"key":"bib101","doi-asserted-by":"publisher","DOI":"10.3758\/BF03192726"},{"key":"bib102","unstructured":"Medin, Douglas L., Mark W. Altom, and Timothy D. Murphy. 1984. Given versus induced category representations: Use of prototype and exemplar information in classification. Journal of Experimental Psychology, 10(3):333\u2013352."},{"key":"bib103","unstructured":"Mikolov, Tomas, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. In Proceedings of ICLR: Workshop Papers, Scottsdale, AZ. http:\/\/arxiv.org\/abs\/1301.3781"},{"key":"bib104","unstructured":"Mikolov, Tomas, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Proceedings of the 27th Annual Conference on Advances in Neural Information Processing Systems (NIPS), pages 3111\u20133119, South Lake Tahoe, NV."},{"key":"bib105","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"bib106","unstructured":"Mohler, Michael, David Bracewell, Marc Tomlinson, and David Hinote. 2013. Semantic signatures for example-based linguistic metaphor detection. In Proceedings of the First Workshop on Metaphor in NLP, pages 27\u201335, Atlanta, GA."},{"key":"bib107","doi-asserted-by":"crossref","unstructured":"Mrk\u0161i\u0107, Nikola, Diarmuid \u00d3 S\u00e9aghdha, Blaise Thomson, Milica Ga\u0161i\u0107, Lina Maria Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. Counter-fitting word vectors to linguistic constraints. In Proceedings of NAACL-HLT, pages 142\u2013148, San Diego, CA.","DOI":"10.18653\/v1\/N16-1018"},{"key":"bib108","doi-asserted-by":"crossref","unstructured":"Mrk\u0161i\u0107, Nikola, Diarmuid \u00d3 S\u00e9aghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve J. Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of ACL, pages 1777\u20131788, Vancouver.","DOI":"10.18653\/v1\/P17-1163"},{"key":"bib111","doi-asserted-by":"crossref","unstructured":"Neelakantan, Arvind, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. 2014. Efficient non-parametric estimation of multiple embeddings per word in vector space. In Proceedings of EMNLP, pages 1059\u20131069, Doha.","DOI":"10.3115\/v1\/D14-1113"},{"key":"bib112","doi-asserted-by":"publisher","DOI":"10.3758\/BF03195588"},{"key":"bib114","doi-asserted-by":"crossref","unstructured":"Ono, Masataka, Makoto Miwa, and Yutaka Sasaki. 2015. Word embedding-based antonym detection using thesauri and distributional information. In Proceedings of NAACL-HLT, pages 984\u2013989, Denver, CO.","DOI":"10.3115\/v1\/N15-1100"},{"key":"bib115","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(81)90013-5"},{"key":"bib116","doi-asserted-by":"publisher","DOI":"10.1016\/S0010-0277(97)00025-5"},{"key":"bib117","unstructured":"Pad\u00f3, Sebastian, Ulrike Pad\u00f3, and Katrin Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of EMNLP-CoNLL, pages 400\u2013409, Prague."},{"key":"bib118","doi-asserted-by":"publisher","DOI":"10.1037\/h0084295"},{"key":"bib119","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220190"},{"key":"bib120","doi-asserted-by":"crossref","unstructured":"Parikh, Ankur, Oscar T\u00e4ckstr\u00f6m, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of EMNLP, pages 2249\u20132255, Austin, TX.","DOI":"10.18653\/v1\/D16-1244"},{"key":"bib121","doi-asserted-by":"crossref","unstructured":"Pedersen, Ted, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet:Similarity - Measuring the relatedness of concepts. In Proceedings of AAAI, pages 1024\u20131025, San Jose, CA.","DOI":"10.3115\/1614025.1614037"},{"key":"bib122","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.142"},{"key":"bib123","unstructured":"Pulman, Stephan Guy. 1983. Word Meaning and Belief. Routledge."},{"key":"bib124","doi-asserted-by":"publisher","DOI":"10.1002\/bs.3830120511"},{"key":"bib125","doi-asserted-by":"crossref","unstructured":"Rei, Marek and Ted Briscoe. 2014. Looking for hyponyms in vector space. In Proceedings of CoNLL, pages 68\u201377, Baltimore, MD.","DOI":"10.3115\/v1\/W14-1608"},{"key":"bib126","unstructured":"Reisinger, Joseph and Raymond J. Mooney. 2010a. A mixture model with sharing for lexical semantics. In Proceedings of EMNLP, pages 1173\u20131182, Cambridge, MA."},{"key":"bib127","unstructured":"Reisinger, Joseph and Raymond J. Mooney. 2010b. Multi-prototype vector-space models of word meaning. In Proceedings of NAACL-HLT, pages 109\u2013117, Los Angeles, CA."},{"key":"bib128","unstructured":"Resnik, Philip. 1995. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of IJCAI, pages 448\u2013453, Montreal."},{"key":"bib129","unstructured":"Riedel, Sebastian, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In Proceedings of NAACL-HLT, pages 74\u201384, Atlanta, GA."},{"key":"bib130","doi-asserted-by":"crossref","unstructured":"Rimell, Laura. 2014. Distributional lexical entailment by topic coherence. In Proceedings of EACL, pages 511\u2013519, Gothenburg.","DOI":"10.3115\/v1\/E14-1054"},{"key":"bib131","doi-asserted-by":"crossref","unstructured":"Roller, Stephen and Katrin Erk. 2016. Relations such as hypernymy: Identifying and exploiting Hearst patterns in distributional vectors for lexical entailment. In Proceedings of EMNLP, pages 2163\u20132172, Austin, TX.","DOI":"10.18653\/v1\/D16-1234"},{"key":"bib132","unstructured":"Roller, Stephen, Katrin Erk, and Gemma Boleda. 2014. Inclusive yet selective: Supervised distributional hypernymy detection. In Proceedings of COLING, pages 1025\u20131036, Dublin."},{"key":"bib133","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(73)90017-0"},{"key":"bib134","doi-asserted-by":"publisher","DOI":"10.1037\/0096-3445.104.3.192"},{"key":"bib135","doi-asserted-by":"crossref","unstructured":"Rubinstein, Dana, Effi Levi, Roy Schwartz, and Ari Rappoport. 2015. How well do distributional models capture different types of semantic knowledge? In Proceedings of ACL, pages 726\u2013730, Beijing.","DOI":"10.3115\/v1\/P15-2119"},{"key":"bib136","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"bib137","unstructured":"Sacaleanu, Bogdan, Constantin Orasan, Christian Spurk, Shiyan Ou, Oscar Ferrandez, Milen Kouylekov, and Matteo Negri. 2008. Entailment-based question answering for structured data. In Proceedings of COLING, pages 173\u2013176, Manchester."},{"key":"bib138","doi-asserted-by":"crossref","unstructured":"Santus, Enrico, Alessandro Lenci, Qin Lu, and Sabine Schulte im Walde. 2014. Chasing hypernyms in vector spaces with entropy. In Proceedings of EACL, pages 38\u201342, Gothenburg.","DOI":"10.3115\/v1\/E14-4008"},{"key":"bib139","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-4208"},{"key":"bib140","doi-asserted-by":"crossref","unstructured":"Schnabel, Tobias, Igor Labutov, David M. Mimno, and Thorsten Joachims. 2015. Evaluation methods for unsupervised word embeddings. In Proceedings of EMNLP, pages 298\u2013307, Lisbon.","DOI":"10.18653\/v1\/D15-1036"},{"key":"bib141","doi-asserted-by":"crossref","unstructured":"Schwartz, Roy, Roi Reichart, and Ari Rappoport. 2015. Symmetric pattern based word embeddings for improved word similarity prediction. In Proceedings of CoNLL, pages 258\u2013267, Beijing.","DOI":"10.18653\/v1\/K15-1026"},{"key":"bib142","doi-asserted-by":"crossref","unstructured":"Shwartz, Vered, Yoav Goldberg, and Ido Dagan. 2016. Improving hypernymy detection with an integrated path-based and distributional method. In Proceedings of ACL, pages 2389\u20132398, Berlin.","DOI":"10.18653\/v1\/P16-1226"},{"key":"bib143","doi-asserted-by":"crossref","unstructured":"Shwartz, Vered, Omer Levy, Ido Dagan, and Jacob Goldberger. 2015. Learning to exploit structured resources for lexical inference. In Proceedings of CoNLL, pages 175\u2013184, Beijing.","DOI":"10.18653\/v1\/K15-1018"},{"key":"bib144","doi-asserted-by":"crossref","unstructured":"Shwartz, Vered, Enrico Santus, and Dominik Schlechtweg. 2017. Hypernyms under siege: Linguistically-motivated artillery for hypernymy detection. In Proceedings of EACL, pages 65\u201375, Valencia.","DOI":"10.18653\/v1\/E17-1007"},{"key":"bib145","unstructured":"Silberer, Carina and Mirella Lapata. 2012. Grounded models of semantic representation. In Proceedings of EMNLP, pages 1423\u20131433, Jeju."},{"key":"bib146","doi-asserted-by":"crossref","unstructured":"Silberer, Carina and Mirella Lapata. 2014. Learning grounded meaning representations with autoencoders. In Proceedings of ACL, pages 721\u2013732, Baltimore, MD.","DOI":"10.3115\/v1\/P14-1068"},{"key":"bib147","unstructured":"Simonyan, Karen and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of ICLR, San Diego, CA. http:\/\/arxiv.org\/abs\/1409.1556"},{"key":"bib148","unstructured":"Snow, Rion, Daniel Jurafsky, and Andrew Y. Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In Proceedings of NIPS, pages 1297\u20131304, Vancouver."},{"key":"bib149","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220276"},{"key":"bib150","unstructured":"Socher, Richard, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of NIPS, pages 926\u2013934, South Lake Tahoe, NV."},{"key":"bib151","doi-asserted-by":"crossref","unstructured":"Socher, Richard, Andrej Karpathy, Quoc V. Le, Christopher D. Manning, and Andrew Y. Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. Transactions of the ACL, 2:207\u2013218, Baltimore, MD.","DOI":"10.1162\/tacl_a_00177"},{"key":"bib152","unstructured":"Song, Yangqiu, Haixun Wang, Zhongyuan Wang, Hongsong Li, and Weizhu Chen. 2011. Short text conceptualization using a probabilistic knowledgebase. In Proceedings of AAAI, pages 2330\u20132336, San Francisco, CA."},{"key":"bib153","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242667"},{"key":"bib154","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2874809"},{"key":"bib156","doi-asserted-by":"crossref","unstructured":"Tsvetkov, Yulia, Manaal Faruqui, Wang Ling, Guillaume Lample, and Chris Dyer. 2015. Evaluation of word vector representations by subspace alignment. In Proceedings of EMNLP, pages 2049\u20132054, Lisbon.","DOI":"10.18653\/v1\/D15-1243"},{"key":"bib157","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2006.32.3.379"},{"key":"bib158","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324913000387"},{"key":"bib159","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2934"},{"key":"bib160","unstructured":"Vendrov, Ivan, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. 2016. Order-embeddings of images and language. In Proceedings of ICLR, San Juan, PR."},{"key":"bib161","unstructured":"Vilnis, Luke and Andrew McCallum. 2015. Word representations via Gaussian embedding. In Proceedings of ICLR, San Diego, CA."},{"key":"bib162","doi-asserted-by":"crossref","unstructured":"\u0160uster, Simon, Ivan Titov, and Gertjan van Noord. 2016. Bilingual learning of multi-sense embeddings with discrete autoencoders. In Proceedings of NAACL-HLT, pages 1346\u20131356, San Diego, CA.","DOI":"10.18653\/v1\/N16-1160"},{"key":"bib163","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, Ivan and Anna Korhonen. 2016. Is \u201cuniversal syntax\u201d universally useful for learning distributed word representations? In Proceedings of ACL, pages 518\u2013524, Berlin.","DOI":"10.18653\/v1\/P16-2084"},{"key":"bib164","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, Ivan, Nikola Mrk\u0161i\u0107, Roi Reichart, Diarmuid \u00d3 S\u00e9aghdha, Steve Young, and Anna Korhonen. 2017. Morph-fitting: Fine-tuning word vector spaces with simple language-specific rules. In Proceedings of ACL, pages 56\u201368, Vancouver.","DOI":"10.18653\/v1\/P17-1006"},{"key":"bib165","doi-asserted-by":"crossref","unstructured":"Vylomova, Ekaterina, Laura Rimell, Trevor Cohn, and Timothy Baldwin. 2016. Take and took, gaggle and goose, book and read: Evaluating the utility of vector differences for lexical relation learning. In Proceedings of ACL, pages 1671\u20131682, Berlin.","DOI":"10.18653\/v1\/P16-1158"},{"key":"bib166","unstructured":"Weeds, Julie, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. 2014. Learning to distinguish hypernyms and co-hyponyms. In Proceedings of COLING, pages 2249\u20132259, Dublin."},{"key":"bib167","doi-asserted-by":"crossref","unstructured":"Weeds, Julie and David Weir. 2003. A general framework for distributional similarity. In Proceedings of EMNLP, pages 81\u201388, Sapporo.","DOI":"10.3115\/1119355.1119366"},{"key":"bib168","doi-asserted-by":"publisher","DOI":"10.3115\/1220355.1220501"},{"key":"bib169","doi-asserted-by":"crossref","unstructured":"Wieting, John, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. From paraphrase database to compositional paraphrase model and back. Transactions of the ACL, 3:345\u2013358.","DOI":"10.1162\/tacl_a_00143"},{"key":"bib170","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213891"},{"key":"bib171","doi-asserted-by":"crossref","unstructured":"Wu, Zhibiao and Martha Palmer. 1994. Verb semantics and lexical selection. In Proceedings of ACL, pages 133\u2013138, Las Cruces, NM.","DOI":"10.3115\/981732.981751"},{"key":"bib172","doi-asserted-by":"crossref","unstructured":"Yaghoobzadeh, Yadollah and Hinrich Sch\u00fctze. 2016. Intrinsic subspace evaluation of word embedding representations. In Proceedings of ACL, pages 236\u2013246, Berlin.","DOI":"10.18653\/v1\/P16-1023"},{"key":"bib173","unstructured":"Yih, Scott Wen-Tau, Geoffrey Zweig, and John C. Platt. 2012. Polarity inducing latent semantic analysis. In Proceedings of EMNLP, pages 1212\u20131222, Jeju."},{"key":"bib174","doi-asserted-by":"crossref","unstructured":"Young, Peter, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the ACL, 2:67\u201378.","DOI":"10.1162\/tacl_a_00166"},{"key":"bib175","doi-asserted-by":"crossref","unstructured":"Yu, Mo and Mark Dredze. 2014. Improving lexical embeddings with semantic knowledge. In Proceedings of ACL, pages 545\u2013550, Baltimore, MD.","DOI":"10.3115\/v1\/P14-2089"},{"key":"bib176","unstructured":"Zhila, Alisa, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, and Tomas Mikolov. 2013. Combining heterogeneous models for measuring relational similarity. In Proceedings of NAACL-HLT, pages 1000\u20131009, Atlanta, GA."},{"key":"bib177","doi-asserted-by":"publisher","DOI":"10.1162\/coli.08-032-R1-06-96"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/COLI_a_00301","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T17:42:25Z","timestamp":1750873345000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/43\/4\/781-835\/1582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12]]},"references-count":160,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["10.1162\/COLI_a_00301"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00301","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12]]}}}