{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:14:19Z","timestamp":1750306459309,"version":"3.41.0"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2015,8,24]],"date-time":"2015-08-24T00:00:00Z","timestamp":1440374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2015,10]]},"abstract":"<jats:p>Neural network techniques are widely applied to obtain high-quality distributed representations of words (i.e., word embeddings) to address text mining, information retrieval, and natural language processing tasks. Most recent efforts have proposed several efficient methods to learn word embeddings from context such that they can encode both semantic and syntactic relationships between words. However, it is quite challenging to handle unseen or rare words with insufficient context. Inspired by the study on the word recognition process in cognitive psychology, in this article, we propose to take advantage of seemingly less obvious but essentially important morphological knowledge to address these challenges. In particular, we introduce a novel neural network architecture called KNET that leverages both words\u2019 contextual information and morphological knowledge to learn word embeddings. Meanwhile, this new learning architecture is also able to benefit from noisy knowledge and balance between contextual information and morphological knowledge. Experiments on an analogical reasoning task and a word similarity task both demonstrate that the proposed KNET framework can greatly enhance the effectiveness of word embeddings.<\/jats:p>","DOI":"10.1145\/2797137","type":"journal-article","created":{"date-parts":[[2015,8,26]],"date-time":"2015-08-26T14:00:30Z","timestamp":1440597630000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["KNET"],"prefix":"10.1145","volume":"34","author":[{"given":"Qing","family":"Cui","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, P. R. China"}]},{"given":"Bin","family":"Gao","sequence":"additional","affiliation":[{"name":"Microsoft Research, Danling St, Beijing, P. R. China"}]},{"given":"Jiang","family":"Bian","sequence":"additional","affiliation":[{"name":"Microsoft Research, Danling St, Beijing, P. R. China"}]},{"given":"Siyu","family":"Qiu","sequence":"additional","affiliation":[{"name":"Nankai University, Tianjin, P. R. China"}]},{"given":"Hanjun","family":"Dai","sequence":"additional","affiliation":[{"name":"Fudan University, Shanghai, P. R. China"}]},{"given":"Tie-Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"Microsoft Research, Danling St, Beijing, P. R. China"}]}],"member":"320","published-online":{"date-parts":[[2015,8,24]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Y. Bengio and J.-S. Senecal and others. 2003. Quick Training of Probabilistic Neural Nets by Importance Sampling. Y. Bengio and J.-S. Senecal and others. 2003. Quick Training of Probabilistic Neural Nets by Importance Sampling."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2007.912312"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/3120260.3120270"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944937"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"A. Bordes J. Weston R. Collobert Y. Bengio and others. 2011. Learning structured embeddings of knowledge bases. In AAAI. A. Bordes J. Weston R. Collobert Y. Bengio and others. 2011. Learning structured embeddings of knowledge bases. In AAAI.","DOI":"10.1609\/aaai.v25i1.7917"},{"key":"e_1_2_1_6_1","unstructured":"J. W. Chapman. 1998. Language prediction skill phonological recoding ability and beginning reading. Reading and Spelling: Development and Disorders 33. J. W. Chapman. 1998. Language prediction skill phonological recoding ability and beginning reading. Reading and Spelling: Development and Disorders 33."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390177"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078186"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1187415.1187418"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"L. Deng X. He and J. Gao. 2013. Deep stacking networks for information retrieval. In ICASSP. 3153--3157. L. Deng X. He and J. Gao. 2013. Deep stacking networks for information retrieval. In ICASSP. 3153--3157.","DOI":"10.1109\/ICASSP.2013.6638239"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1207\/s1532799xssr0902_4"},{"key":"e_1_2_1_12_1","first-page":"383","article-title":"Development of the ability to read words","volume":"2","author":"Ehri L. C.","year":"1991","unstructured":"L. C. Ehri , R. Barr , M. L. Kamil , P. Mosenthal , and P. D. Pearson . 1991 . Development of the ability to read words . Handbook of Reading Research 2 , 383 -- 417 . L. C. Ehri, R. Barr, M. L. Kamil, P. Mosenthal, and P. D. Pearson. 1991. Development of the ability to read words. Handbook of Reading Research 2, 383--417.","journal-title":"Handbook of Reading Research"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372094"},{"volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML\u201911)","author":"Glorot X.","key":"e_1_2_1_14_1","unstructured":"X. Glorot , A. Bordes , and Y. Bengio . 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach . In Proceedings of the 28th International Conference on Machine Learning (ICML\u201911) . 513--520. X. Glorot, A. Bordes, and Y. Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning (ICML\u201911). 513--520."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/0022-0965(86)90016-0"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/2503308.2188396"},{"key":"e_1_2_1_17_1","unstructured":"G. E. Hinton J. L. McClelland and D. E. Rumelhart. 1986. Distributed representations. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press 3:1137--1155. G. E. Hinton J. L. McClelland and D. E. Rumelhart. 1986. Distributed representations. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press 3:1137--1155."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 289--296","author":"Hofmann T.","year":"1999","unstructured":"T. Hofmann . 1999 . Probabilistic latent semantic analysis . In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 289--296 . T. Hofmann. 1999. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 289--296."},{"volume-title":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, 873--882","author":"Huang E. H.","key":"e_1_2_1_19_1","unstructured":"E. H. Huang , R. Socher , C. D. Manning , and A. Y. Ng . 2012. Improving word representations via global context and multiple word prototypes . In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, 873--882 . E. H. Huang, R. Socher, C. D. Manning, and A. Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, 873--882."},{"volume-title":"Computer)","author":"Liang F. M.","key":"e_1_2_1_20_1","unstructured":"F. M. Liang . 1983. Word Hy-phen-a-tion by Com-put-er (Hyphenation , Computer) . Stanford University , Stanford, CA, USA . F. M. Liang. 1983. Word Hy-phen-a-tion by Com-put-er (Hyphenation, Computer). Stanford University, Stanford, CA, USA."},{"key":"e_1_2_1_21_1","unstructured":"M.-T. Luong R. Socher and C. D. Manning. 2013. Better word representations with recursive neural networks for morphology. CoNLL-2013. 104. M.-T. Luong R. Socher and C. D. Manning. 2013. Better word representations with recursive neural networks for morphology. CoNLL-2013. 104."},{"key":"e_1_2_1_23_1","unstructured":"T. Mikolov K. Chen G. Corrado and J. Dean. 2013a. Efficient estimation of word representations in vector space (ICLR\u201913). T. Mikolov K. Chen G. Corrado and J. Dean. 2013a. Efficient estimation of word representations in vector space (ICLR\u201913)."},{"key":"e_1_2_1_24_1","unstructured":"T. Mikolov I. Sutskever K. Chen G. S. Corrado and J. Dean. 2013b. Distributed representations of words and phrases and their compositionality. In NIPS. 3111--3119. T. Mikolov I. Sutskever K. Chen G. S. Corrado and J. Dean. 2013b. Distributed representations of words and phrases and their compositionality. In NIPS. 3111--3119."},{"key":"e_1_2_1_25_1","unstructured":"A. Mnih and G. E. Hinton. 2008. A scalable hierarchical distributed language model. In NIPS. 1081--1088. A. Mnih and G. E. Hinton. 2008. A scalable hierarchical distributed language model. In NIPS. 1081--1088."},{"key":"e_1_2_1_26_1","unstructured":"A. Mnih and K. Kavukcuoglu. 2013. Learning word embeddings efficiently with noise-contrastive estimation. In NIPS. 2265--2273. A. Mnih and K. Kavukcuoglu. 2013. Learning word embeddings efficiently with noise-contrastive estimation. In NIPS. 2265--2273."},{"key":"e_1_2_1_27_1","unstructured":"A Mnih and Y. W. Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. In ICML. Omnipress New York NY 1751--1758. A Mnih and Y. W. Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. In ICML. Omnipress New York NY 1751--1758."},{"key":"e_1_2_1_28_1","unstructured":"F. Morin and Y. Bengio. 2005. Hierarchical probabilistic neural network language model. In AISTATS. 246--252. F. Morin and Y. Bengio. 2005. Hierarchical probabilistic neural network language model. In AISTATS. 246--252."},{"volume-title":"Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8435--8439","author":"El-Desoky Mousa A.","key":"e_1_2_1_29_1","unstructured":"A. El-Desoky Mousa , H.-K. J. Kuo , L. Mangu , and H. Soltau . 2013. Morpheme-based feature-rich language models using deep neural networks for lvcsr of egyptian arabic . In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8435--8439 . A. El-Desoky Mousa, H.-K. J. Kuo, L. Mangu, and H. Soltau. 2013. Morpheme-based feature-rich language models using deep neural networks for lvcsr of egyptian arabic. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8435--8439."},{"key":"e_1_2_1_30_1","volume-title":"Proc. of COLING.","author":"Qiu S.","year":"2014","unstructured":"S. Qiu , Q. Cui , J. Bian , B. Gao , and T.-Y. Liu . 2014 . Co-learning of word representations and morpheme representations . In Proc. of COLING. S. Qiu, Q. Cui, J. Bian, B. Gao, and T.-Y. Liu. 2014. Co-learning of word representations and morpheme representations. In Proc. of COLING."},{"key":"e_1_2_1_31_1","unstructured":"R. Socher D. Chen C. D. Manning and A. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NIPS. 926--934. R. Socher D. Chen C. D. Manning and A. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NIPS. 926--934."},{"volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML\u201911)","author":"Socher R.","key":"e_1_2_1_32_1","unstructured":"R. Socher , C. C. Lin , A. Y. Ng , and C. D. Manning . 2011. Parsing natural scenes and natural language with recursive neural networks . In Proceedings of the 28th International Conference on Machine Learning (ICML\u201911) . 129--136. R. Socher, C. C. Lin, A. Y. Ng, and C. D. Manning. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML\u201911). 129--136."},{"volume-title":"Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality. 30--39","author":"Sperr H.","key":"e_1_2_1_33_1","unstructured":"H. Sperr , J. Niehues , and A. Waibel . 2013. Letter n-gram-based input encoding for continuous space language models . In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality. 30--39 . H. Sperr, J. Niehues, and A. Waibel. 2013. Letter n-gram-based input encoding for continuous space language models. In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality. 30--39."},{"key":"e_1_2_1_34_1","unstructured":"J. P. Turian L.-A. Ratinov and Y. Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In ACL. 384--394. J. P. Turian L.-A. Ratinov and Y. Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In ACL. 384--394."},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","unstructured":"P. D. Turney. 2013. Distributional semantics beyond words: Supervised learning of analogy and paraphrase. TACL 353--366. P. D. Turney. 2013. Distributional semantics beyond words: Supervised learning of analogy and paraphrase. TACL 353--366.","DOI":"10.1162\/tacl_a_00233"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/1861751.1861756"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"J. Weston A. Bordes O. Yakhnenko and N. Usunier. 2013. Connecting language and knowledge bases with embedding models for relation extraction. arXiv preprint arXiv:1307.7973. J. Weston A. Bordes O. Yakhnenko and N. Usunier. 2013. Connecting language and knowledge bases with embedding models for relation extraction. arXiv preprint arXiv:1307.7973.","DOI":"10.18653\/v1\/D13-1136"},{"key":"e_1_2_1_38_1","doi-asserted-by":"crossref","unstructured":"M. Yu and M. Dredze. 2014. Improving lexical embeddings with semantic knowledge. In Association for Computational Linguistics (ACL). 545--550. M. Yu and M. Dredze. 2014. Improving lexical embeddings with semantic knowledge. In Association for Computational Linguistics (ACL). 545--550.","DOI":"10.3115\/v1\/P14-2089"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2797137","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2797137","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:43:29Z","timestamp":1750225409000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2797137"}},"subtitle":["A General Framework for Learning Word Embedding Using Morphological Knowledge"],"short-title":[],"issued":{"date-parts":[[2015,8,24]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,10]]}},"alternative-id":["10.1145\/2797137"],"URL":"https:\/\/doi.org\/10.1145\/2797137","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"type":"print","value":"1046-8188"},{"type":"electronic","value":"1558-2868"}],"subject":[],"published":{"date-parts":[[2015,8,24]]},"assertion":[{"value":"2014-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-08-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}