{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T09:36:04Z","timestamp":1774604164533,"version":"3.50.1"},"reference-count":57,"publisher":"Cambridge University Press (CUP)","issue":"6","license":[{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>As a ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words, but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget\u2019s Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. Manual human evaluation results have also been presented to further verify that the proposed method increases interpretability. We also demonstrate the preservation of semantic coherence of the resulting vector space using word-analogy\/word-similarity tests and a downstream task. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests.<\/jats:p>","DOI":"10.1017\/s1351324920000315","type":"journal-article","created":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T08:57:32Z","timestamp":1591693052000},"page":"721-746","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":6,"title":["Imparting interpretability to word embeddings while preserving semantic structure"],"prefix":"10.1017","volume":"27","author":[{"given":"L\u00fctfi Kerem","family":"\u015eenel","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u0130hsan","family":"Utlu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Furkan","family":"\u015eahinu\u00e7","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haldun M.","family":"Ozaktas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aykut","family":"Ko\u00e7","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2020,6,9]]},"reference":[{"key":"S1351324920000315_ref43","volume-title":"Roget\u2019s Thesaurus of English Words and Phrases","author":"Roget","year":"1911"},{"key":"S1351324920000315_ref51","unstructured":"Socher, R. , Perelygin, A. , Wu, J. , Chuang, J. , Manning, C.D. , Ng, A.Y. and Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). Seattle, WA, USA: Association for Computational Linguistics, pp. 1631\u20131642."},{"key":"S1351324920000315_ref4","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11259"},{"key":"S1351324920000315_ref24","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1164"},{"key":"S1351324920000315_ref54","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2662038"},{"key":"S1351324920000315_ref40","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1041"},{"key":"S1351324920000315_ref33","unstructured":"Mikolov, T. , Sutskever, I. , Chen, K. , Corrado, G.S. and Dean, J. (2013c). Distributed representations of words and phrases and their compositionality. In Burges C. J. C., Bottou L., Welling M., Ghahramani Z. and Weinberger K. Q. (eds.), Advances in Neural Information Processing Systems, pp. 3111\u20133119. Curran Associates, Inc."},{"key":"S1351324920000315_ref38","unstructured":"Murphy, B. , Talukdar, P.P. and Mitchell, T.M. (2012). Learning effective and interpretable semantic models using non-negative sparse embedding. In Proceedings of COLING 2012. Mumbai, India: The COLING 2012 Organizing Committee, pp. 1933\u20131950."},{"key":"S1351324920000315_ref44","unstructured":"Roget, P.M. (2008). Roget\u2019s International Thesaurus, 3\/E. New Delhi: Oxford & IBH Publishing Company Pvt. Limited."},{"key":"S1351324920000315_ref25","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1104"},{"key":"S1351324920000315_ref11","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5004"},{"key":"S1351324920000315_ref52","doi-asserted-by":"crossref","unstructured":"Subramanian, A. , Pruthi, D. , Jhamtani, H. , Berg-Kirkpatrick, T. and Hovy, E. (2018). SPINE: sparse interpretable neural embeddings. In Proceedings of the Thirty Second AAAI Conference on Artificial Intelligence New Orleans, LA, USA: Association for the Advancement of Artificial Intelligence (AAAI), pp. 4921\u20134928.","DOI":"10.1609\/aaai.v32i1.11935"},{"key":"S1351324920000315_ref41","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1351324920000315_ref55","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1056"},{"key":"S1351324920000315_ref48","unstructured":"Sien\u010dnik, S.K. (2015). Adapting word2vec to named entity recognition. In Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015). Vilnius, Lithuania: Link\u00f6ping University Electronic Press, Sweden, pp. 239\u2013243."},{"key":"S1351324920000315_ref14","volume-title":"Papers in Linguistics, 1934-1951","author":"Firth","year":"1957"},{"key":"S1351324920000315_ref29","doi-asserted-by":"crossref","unstructured":"Liu, Q. , Jiang, H. , Wei, S. , Ling, Z.-H. and Hu, Y. (2015). Learning semantic word embeddings based on ordinal knowledge constraints. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Beijing, China: Association for Computational Linguistics, pp. 1501\u20131511.","DOI":"10.3115\/v1\/P15-1145"},{"key":"S1351324920000315_ref45","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2018.2837384"},{"key":"S1351324920000315_ref2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"S1351324920000315_ref21","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1003"},{"key":"S1351324920000315_ref22","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5442"},{"key":"S1351324920000315_ref3","doi-asserted-by":"crossref","unstructured":"Bollagela, D. , Alsuhaibani, M. , Maehara, T. and Kawarabayashi, K. (2016). Joint word representation learning using a corpus and a semantic lexicon. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix, AZ, USA: Association for the Advancement of Artificial Intelligence (AAAI), pp. 2690\u20132696.","DOI":"10.1609\/aaai.v30i1.10340"},{"key":"S1351324920000315_ref31","unstructured":"Mikolov, T. , Chen, K. , Corrado, G. and Dean, J. (2013a). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781."},{"key":"S1351324920000315_ref20","doi-asserted-by":"publisher","DOI":"10.1080\/00437956.1954.11659520"},{"key":"S1351324920000315_ref50","unstructured":"Socher, R. , Pennington, J. , Huang, E.H. , Ng, A.Y. and Manning, C.D. (2011). Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP). Edinburgh, Scotland, UK: Association for Computational Linguistics, pp. 151\u2013161."},{"key":"S1351324920000315_ref47","doi-asserted-by":"crossref","unstructured":"Senel, L.K. , Yucesoy, V. , Ko\u00e7, A. and Cukur, T. (2017). Measuring cross-lingual semantic similarity across European languages. In 40th International Conference on Telecommunications and Signal Processing (TSP). Barcelona, Spain: IEEE, pp. 359\u2013363.","DOI":"10.1109\/TSP.2017.8076005"},{"key":"S1351324920000315_ref17","doi-asserted-by":"crossref","unstructured":"Glava\u0161, G. and Vuli\u0107, I. (2018). Explicit retrofitting of distributional word vectors. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne, Australia: Association for Computational Linguistics, pp. 34\u201345.","DOI":"10.18653\/v1\/P18-1004"},{"key":"S1351324920000315_ref37","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00063"},{"key":"S1351324920000315_ref30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1196"},{"key":"S1351324920000315_ref26","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1085"},{"key":"S1351324920000315_ref9","unstructured":"De Vine, L. , Kholgi, M. , Zuccon, G. , Sitbon, L. and Nguyen, A. (2015). Analysis of word embeddings and sequence features for clinical information extraction. In Proceedings of the Australasian Language Technology Association Workshop 2015. Parramatta, Australia, pp. 21\u201330."},{"key":"S1351324920000315_ref39","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1570"},{"key":"S1351324920000315_ref12","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1144"},{"key":"S1351324920000315_ref7","unstructured":"Chen, X. , Duan, Y. , Houthooft, R. , Schulman, J. , Sutskever, I. and Abbeel, P. (2016). Infogan: interpretable representation learning by information maximizing generative adversarial nets. In Lee D. D., Sugiyama M., Luxburg U. V., Guyon I. and Garnett R. (eds.), Advances in Neural Information Processing Systems, pp. 2172\u20132180. Curran Associates, Inc."},{"key":"S1351324920000315_ref13","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1184"},{"key":"S1351324920000315_ref6","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1082"},{"key":"S1351324920000315_ref34","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"S1351324920000315_ref46","doi-asserted-by":"publisher","DOI":"10.1109\/SIU.2018.8404244"},{"key":"S1351324920000315_ref1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00034"},{"key":"S1351324920000315_ref8","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1077"},{"key":"S1351324920000315_ref15","first-page":"1930","volume-title":"Studies in Linguistic Analysis","author":"Firth","year":"1957"},{"key":"S1351324920000315_ref35","unstructured":"Moody, C.E. (2016). Mixing dirichlet topic models and word embeddings to make lda2vec. arXiv preprint arXiv:1605.02019."},{"key":"S1351324920000315_ref28","doi-asserted-by":"crossref","unstructured":"Liu, Y. , Liu, Z. , Chua, T.-S. and Sun, M. (2015). Topical word embeddings. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Austin, TX, USA: Association for the Advancement of Artificial Intelligence (AAAI), pp. 2418\u20132424.","DOI":"10.1609\/aaai.v29i1.9522"},{"key":"S1351324920000315_ref32","unstructured":"Mikolov, T. , Le, Q.V. and Sutskever, I. (2013b). Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168."},{"key":"S1351324920000315_ref16","doi-asserted-by":"crossref","unstructured":"Fyshe, A. , Talukdar, P.P. , Murphy, B. and Mitchell, T.M. (2014). Interpretable semantic vectors from a joint model of brain-and text-based meaning. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Baltimore, MD, USA: Association for Computational Linguistics, pp. 489\u2013499.","DOI":"10.3115\/v1\/P14-1046"},{"key":"S1351324920000315_ref5","unstructured":"Chang, J. , Gerrish, S. , Wang, C. , Boyd-Graber, J.L. and Blei, D.M. (2009). Reading tea leaves: how humans interpret topic models. In Bengio Y., Schuurmans D., Lafferty J. D., Williams C. K. I. and Culotta A. (eds.), Advances in Neural Information Processing Systems, pp. 288\u2013296. Curran Associates, Inc."},{"key":"S1351324920000315_ref36","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1018"},{"key":"S1351324920000315_ref49","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080806"},{"key":"S1351324920000315_ref42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1026"},{"key":"S1351324920000315_ref27","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2050"},{"key":"S1351324920000315_ref57","doi-asserted-by":"crossref","unstructured":"Zobnin, A. (2017). Rotations and interpretability of word embeddings: the case of the Russian language. In International Conference on Analysis of Images, Social Networks and Texts. Moscow, Russia: Springer International Publishing, pp. 116\u2013128.","DOI":"10.1007\/978-3-319-73013-4_11"},{"key":"S1351324920000315_ref18","volume-title":"In Synthesis Lectures on Human Language Technologies","author":"Goldberg","year":"2017"},{"key":"S1351324920000315_ref53","unstructured":"Turian, J. , Ratinov, L.-A. and Bengio, Y. (2010). Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Uppsala, Sweden: Association for Computational Linguistics, pp. 384\u2013394."},{"key":"S1351324920000315_ref23","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1070"},{"key":"S1351324920000315_ref19","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i3.2741"},{"key":"S1351324920000315_ref56","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2089"},{"key":"S1351324920000315_ref10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1111"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324920000315","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T08:38:22Z","timestamp":1666859902000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324920000315\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,9]]},"references-count":57,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["S1351324920000315"],"URL":"https:\/\/doi.org\/10.1017\/s1351324920000315","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,9]]},"assertion":[{"value":"\u00a9 The Author(s), 2020. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}