{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T22:00:23Z","timestamp":1747173623439,"version":"3.40.5"},"reference-count":74,"publisher":"Cambridge University Press (CUP)","issue":"4","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the aid of recently proposed word embedding algorithms, the study of semantic relatedness has progressed rapidly. However, word-level representations are still lacking for many natural language processing tasks. Various sense-level embedding learning algorithms have been proposed to address this issue. In this paper, we present a generalized model derived from existing sense retrofitting models. In this generalization, we take into account semantic relations between the senses, relation strength, and semantic strength. Experimental results show that the generalized model outperforms previous approaches on four tasks: semantic relatedness, contextual word similarity, semantic difference, and synonym selection. Based on the generalized sense retrofitting model, we also propose a standardization process on the dimensions with four settings, a neighbor expansion process from the nearest neighbors, and combinations of these two approaches. Finally, we propose a Procrustes analysis approach that inspired from bilingual mapping models for learning representations that outside of the ontology. The experimental results show the advantages of these approaches on semantic relatedness tasks.<\/jats:p>","DOI":"10.1017\/s1351324922000523","type":"journal-article","created":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T09:45:46Z","timestamp":1680255946000},"page":"1097-1125","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":0,"title":["On generalization of the sense retrofitting model"],"prefix":"10.1017","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6279-4414","authenticated-orcid":false,"given":"Yang-Yin","family":"Lee","sequence":"first","affiliation":[]},{"given":"Ting-Yu","family":"Yen","sequence":"additional","affiliation":[]},{"given":"Hen-Hsen","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Yow-Ting","family":"Shiue","sequence":"additional","affiliation":[]},{"given":"Hsin-Hsi","family":"Chen","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2023,3,31]]},"reference":[{"key":"S1351324922000523_ref37","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3133152"},{"key":"S1351324922000523_ref70","doi-asserted-by":"publisher","DOI":"10.3115\/981732.981751"},{"key":"S1351324922000523_ref42","doi-asserted-by":"publisher","DOI":"10.1145\/502512.502559"},{"key":"S1351324922000523_ref63","unstructured":"Santos, J. , Consoli, B. and Vieira, R. (2020). Word embedding evaluation in downstream tasks and semantic analogies. In Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC), Marseille, France. European Language Resources Association, pp. 4828\u20134834."},{"key":"S1351324922000523_ref17","doi-asserted-by":"publisher","DOI":"10.3115\/1220355.1220406"},{"key":"S1351324922000523_ref52","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1018"},{"key":"S1351324922000523_ref46","unstructured":"Luong, T. , Socher, R. and Manning, C. (2013). Better word representations with recursive neural networks for morphology. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning (CoNLL), Sofia, Bulgaria. Association for Computational Linguistics, pp. 104\u2013113."},{"key":"S1351324922000523_ref6","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10340"},{"key":"S1351324922000523_ref64","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6402"},{"key":"S1351324922000523_ref34","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.104.2.211"},{"key":"S1351324922000523_ref41","unstructured":"Lin, D. (1998). An information-theoretic definition of similarity. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML), vol. 98, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc., pp. 296\u2013304."},{"key":"S1351324922000523_ref55","doi-asserted-by":"crossref","unstructured":"Peters, M.E. , Neumann, M. , Iyyer, M. , Gardner, M. , Clark, C. , Lee, K. and Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (NAACL), New Orleans, Louisiana. Association for Computational Linguistics.","DOI":"10.18653\/v1\/N18-1202"},{"key":"S1351324922000523_ref74","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2089"},{"key":"S1351324922000523_ref43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-48051-0_12"},{"key":"S1351324922000523_ref2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2011.108"},{"key":"S1351324922000523_ref9","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4135"},{"key":"S1351324922000523_ref4","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44848-9_9"},{"key":"S1351324922000523_ref25","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1141"},{"key":"S1351324922000523_ref29","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1070"},{"key":"S1351324922000523_ref44","unstructured":"Liu, Y. , Ott, M. , Goyal, N. , Du, J. , Joshi, M. , Chen, D. , Levy, O. , Lewis, M. , Zettlemoyer, L. and Stoyanov, V. (2019). Roberta: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692."},{"key":"S1351324922000523_ref8","unstructured":"Brunet, M.-E. , Alkalay-Houlihan, C. , Anderson, A. and Zemel, R. (2019). Understanding the origins of bias in word embeddings. In Proceedings of the 36th International Conference on Machine Learning (ICML), Long Beach, CA, USA. PMLR, pp. 803\u2013811."},{"key":"S1351324922000523_ref57","unstructured":"Quirk, C. , Brockett, C. and Dolan, W.B. (2004). Monolingual machine translation for paraphrase generation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), Barcelona, Spain. Association for Computational Linguistics, pp. 142\u2013149."},{"key":"S1351324922000523_ref60","unstructured":"Reisinger, J. and Mooney, R.J. (2010). Multi-prototype vector-space models of word meaning. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), Los Angeles, CA, USA. Association for Computational Linguistics, pp. 109\u2013117."},{"key":"S1351324922000523_ref36","doi-asserted-by":"publisher","DOI":"10.1145\/2872518.2889395"},{"key":"S1351324922000523_ref56","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1018"},{"key":"S1351324922000523_ref38","unstructured":"Lee, Y.-Y. , Yen, T.-Y. , Huang, H.-H. , Shiue, Y.-T. and Chen, H.-H. (2018). Gensense: A generalized sense retrofitting model. In Proceedings of the 27th International Conference on Computational Linguistics (COLING), Santa Fe, NM, USA. Association for Computational Linguistics, pp. 1662\u20131671."},{"key":"S1351324922000523_ref11","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal4230"},{"key":"S1351324922000523_ref19","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1006"},{"key":"S1351324922000523_ref27","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1010"},{"key":"S1351324922000523_ref13","doi-asserted-by":"crossref","unstructured":"Camacho-Collados, J. , Pilehvar, M.T. and Navigli, R. (2015). Nasari: A novel approach to a semantically-aware representation of items. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Denver, CO, USA. Association for Computational Linguistics, pp. 567\u2013577.","DOI":"10.3115\/v1\/N15-1059"},{"key":"S1351324922000523_ref23","unstructured":"Ganitkevitch, J. , Van Durme, B. and Callison-Burch, C. (2013). PPDB: The paraphrase database. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA, USA. Association for Computational Linguistics, pp. 758\u2013764."},{"key":"S1351324922000523_ref58","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"S1351324922000523_ref35","doi-asserted-by":"crossref","first-page":"265","DOI":"10.7551\/mitpress\/7287.003.0018","article-title":"Combining local context and WordNet similarity for word sense identification","volume":"49","author":"Leacock","year":"1998","journal-title":"WordNet: An Electronic Lexical Database"},{"key":"S1351324922000523_ref61","unstructured":"Remus, S. and Biemann, C. (2018). Retrofitting word representations for unsupervised sense aware word similarities. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC), Miyazaki, Japan. European Language Resources Association."},{"key":"S1351324922000523_ref3","doi-asserted-by":"crossref","unstructured":"Bengio, Y. , Delalleau, O. and Le Roux, N. (2006). Label propagation and quadratic criterion. In Semi-Supervised Learning.","DOI":"10.7551\/mitpress\/6173.003.0016"},{"key":"S1351324922000523_ref31","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1330"},{"key":"S1351324922000523_ref49","unstructured":"Mikolov, T. , Chen, K. , Corrado, G. and Dean, J. (2013a). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781."},{"key":"S1351324922000523_ref21","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1184"},{"key":"S1351324922000523_ref22","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1145\/503104.503110","article-title":"Placing search in context: The concept revisited","volume":"20","author":"Finkelstein","year":"2002","journal-title":"ACM Transactions on Information Systems"},{"key":"S1351324922000523_ref47","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K17-1012"},{"key":"S1351324922000523_ref30","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0251559"},{"key":"S1351324922000523_ref40","doi-asserted-by":"crossref","unstructured":"Li, J. and Jurafsky, D. (2015). Do multi-sense embeddings improve natural language understanding? In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), Lisbon, Portugal. Association for Computational Linguistics, pp. 1722\u20131732.","DOI":"10.18653\/v1\/D15-1200"},{"volume-title":"WordNet: An Electronic Lexical Database","year":"1998","author":"Miller","key":"S1351324922000523_ref51"},{"key":"S1351324922000523_ref73","unstructured":"Yin, Z. and Shen, Y. (2018). On the dimensionality of word embedding. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS), vol. 31, Montr\u00e9al, Canada. Curran Associates, Inc., pp. 895\u2013906."},{"key":"S1351324922000523_ref71","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1104"},{"key":"S1351324922000523_ref12","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11259"},{"key":"S1351324922000523_ref26","unstructured":"Huang, E.H. , Socher, R. , Manning, C.D. and Ng, A.Y. (2012). Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL), Jeju Island, Korea. Association for Computational Linguistics, pp. 873\u2013882."},{"key":"S1351324922000523_ref53","doi-asserted-by":"crossref","unstructured":"Pavlick, E. , Rastogi, P. , Ganitkevitch, J. , Van Durme, B. and Callison-Burch, C. (2015). PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) (ACL-IJCNLP), Beijing, China. Association for Computational Linguistics, pp. 425\u2013430.","DOI":"10.3115\/v1\/P15-2070"},{"key":"S1351324922000523_ref14","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1285"},{"key":"S1351324922000523_ref10","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193020"},{"volume-title":"Roget\u2019s 21st Century Thesaurus in Dictionary Form: The Essential Reference for Home, School, or Office","year":"1993","author":"Kipfer","key":"S1351324922000523_ref32"},{"key":"S1351324922000523_ref54","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1351324922000523_ref67","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10338"},{"key":"S1351324922000523_ref50","unstructured":"Mikolov, T. , Le, Q.V. and Sutskever, I. (2013b). Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168."},{"key":"S1351324922000523_ref28","doi-asserted-by":"crossref","unstructured":"Jarmasz, M. and Szpakowicz, S. (2004). Roget\u2019s thesaurus and semantic similarity. In Recent Advances in Natural Language Processing III: Selected Papers from RANLP, 2003, 111.","DOI":"10.1075\/cilt.260.12jar"},{"key":"S1351324922000523_ref1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1250"},{"key":"S1351324922000523_ref20","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1163"},{"key":"S1351324922000523_ref65","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1113"},{"key":"S1351324922000523_ref66","unstructured":"Smith, S.L. , Turban, D.H. , Hamblin, S. and Hammerla, N.Y. (2017). Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In 5th International Conference on Learning Representations (ICLR), Toulon, France. OpenReview.net."},{"key":"S1351324922000523_ref5","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"S1351324922000523_ref18","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2717879"},{"key":"S1351324922000523_ref68","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44795-4_42"},{"key":"S1351324922000523_ref72","doi-asserted-by":"publisher","DOI":"10.1145\/3184558.3186906"},{"key":"S1351324922000523_ref24","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1004"},{"key":"S1351324922000523_ref45","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1569"},{"key":"S1351324922000523_ref48","unstructured":"Maneewongvatana, S. and Mount, D.M. (1999). It\u2019s okay to be skinny, if your friends are fat. In Center for Geometric Computing 4th Annual Workshop on Computational Geometry, vol. 2, pp. 1\u20138."},{"key":"S1351324922000523_ref15","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9"},{"key":"S1351324922000523_ref33","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2509"},{"key":"S1351324922000523_ref7","unstructured":"Bolukbasi, T. , Chang, K.-W. , Zou, J.Y. , Saligrama, V. and Kalai, A.T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS), vol. 29, Barcelona, Spain. Curran Associates, Inc."},{"key":"S1351324922000523_ref16","unstructured":"Devlin, J. , Chang, M.-W. , Lee, K. and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (NAACL-HLT), Minneapolis, MN, USA. Association for Computational Linguistics, pp. 4171\u20134186."},{"key":"S1351324922000523_ref39","unstructured":"Lengerich, B.J. , Maas, A.L. and Potts, C. (2017). Retrofitting distributional embeddings to knowledge graphs with functional relations. In Proceedings of the 27th International Conference on Computational Linguistics (COLING), Santa Fe, NM, USA. Association for Computational Linguistics."},{"key":"S1351324922000523_ref62","unstructured":"Sanh, V. , Debut, L. , Chaumond, J. and Wolf, T. (2019). Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108."},{"key":"S1351324922000523_ref59","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963455"},{"key":"S1351324922000523_ref69","unstructured":"Wiedemann, G. , Remus, S. , Chawla, A. and Biemann, C. (2019). Does BERT make any sense? Interpretable word sense disambiguation with contextualized embeddings. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS), Erlangen, Germany. German Society for Computational Linguistics & Language Technology."}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324922000523","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T12:32:32Z","timestamp":1729168352000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324922000523\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":74,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["S1351324922000523"],"URL":"https:\/\/doi.org\/10.1017\/s1351324922000523","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"type":"print","value":"1351-3249"},{"type":"electronic","value":"1469-8110"}],"subject":[],"published":{"date-parts":[[2023,3,31]]},"assertion":[{"value":"\u00a9 The Author(s), 2023. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}