{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,5]],"date-time":"2025-10-05T19:50:16Z","timestamp":1759693816983},"reference-count":38,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p> We propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are shared across documents, a word is generated by a hidden semantic vector encoding its contextual semantic meaning, and its context words are generated conditional on both the hidden semantic vector and global latent topics. Topics are trained jointly with the word embeddings. The trained model maps words to topic-dependent embeddings, which naturally addresses the issue of word polysemy. Experimental results show that the proposed model outperforms the word-level embedding methods in both word similarity evaluation and word sense disambiguation. Furthermore, the model also extracts more coherent topics compared with existing neural topic models or other models for joint learning of topics and word embeddings. Finally, the model can be easily integrated with existing deep contextualized word embedding learning methods to further improve the performance of downstream tasks such as sentiment classification. <\/jats:p>","DOI":"10.1162\/tacl_a_00326","type":"journal-article","created":{"date-parts":[[2020,9,3]],"date-time":"2020-09-03T15:05:10Z","timestamp":1599145510000},"page":"471-485","source":"Crossref","is-referenced-by-count":6,"title":["A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings"],"prefix":"10.1162","volume":"8","author":[{"given":"Lixing","family":"Zhu","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Warwick, UK."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yulan","family":"He","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Warwick, UK."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University, China."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","first-page":"3135","volume-title":"Proceedings of the 31st AAAI Conference on Artificial Intelligence","author":"Barkan Oren","year":"2017"},{"key":"bib2","first-page":"2095","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"Bouchacourt Diane","year":"2018"},{"key":"bib3","first-page":"1775","volume-title":"Proceedings of COLING 2018, the 27th International Conference on Computational Linguistics: Technical Papers","author":"Bra\u017einskas Arthur","year":"2018"},{"key":"bib4","first-page":"1052","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Briakou Eleftheria","year":"2019"},{"key":"bib5","first-page":"795","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing","author":"Das Rajarshi","year":"2015"},{"key":"bib6","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019"},{"key":"bib7","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1145\/371920.372094","volume-title":"Proceedings of the 10th international conference on World Wide Web","author":"Finkelstein Lev","year":"2001"},{"key":"bib8","first-page":"86","volume-title":"Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics","author":"Foulds James R.","year":"2018"},{"key":"bib9","doi-asserted-by":"crossref","first-page":"328","DOI":"10.18653\/v1\/P18-1031","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Howard Jeremy","year":"2018"},{"key":"bib10","doi-asserted-by":"crossref","first-page":"1685","DOI":"10.18653\/v1\/P19-1165","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Iacobacci Ignacio","year":"2019"},{"key":"bib11","doi-asserted-by":"crossref","first-page":"3319","DOI":"10.18653\/v1\/P19-1321","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jameel Shoaib","year":"2019"},{"issue":"2","key":"bib12","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1007665907178","volume":"37","author":"Jordan Michael I.","year":"1999","journal-title":"Machine Learning"},{"key":"bib13","first-page":"1","volume":"1050","author":"Kingma Diederik P.","year":"2014","journal-title":"stat"},{"issue":"2","key":"bib14","first-page":"11","volume":"36","author":"Li Chenliang","year":"2017","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"bib15","first-page":"2418","volume-title":"Proceedings of the 29th AAAI Conference on Artificial Intelligence","author":"Liu Yang","year":"2015"},{"key":"bib16","doi-asserted-by":"crossref","first-page":"48","DOI":"10.3115\/1621474.1621483","volume-title":"Proceedings of the 4th International Workshop on Semantic Evaluations","author":"McCarthy Diana","year":"2007"},{"key":"bib17","first-page":"1","volume-title":"Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing","author":"Melamud Oren","year":"2015"},{"key":"bib18","first-page":"1727","volume-title":"International Conference on Machine Learning (ICML)","author":"Miao Yishu","year":"2016"},{"key":"bib19","author":"Mikolov Tomas","year":"2013","journal-title":"CoRR 2013"},{"key":"bib20","first-page":"3111","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems","volume":"13","author":"Mikolov Tomas","year":"2013"},{"key":"bib21","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.artint.2012.07.001","volume":"193","author":"Navigli Roberto","year":"2012","journal-title":"Artificial Intelligence"},{"key":"bib22","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.3115\/v1\/D14-1113","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Neelakantan Arvind","year":"2014"},{"key":"bib23","first-page":"299","volume-title":"Transactions of ACL","author":"Nguyen Dat Quoc","year":"2015"},{"key":"bib24","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","volume-title":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)","author":"Pennington Jeffrey","year":"2014"},{"key":"bib25","first-page":"2227","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","volume":"1","author":"Peters Matthew","year":"2018"},{"key":"bib26","doi-asserted-by":"crossref","first-page":"1680","DOI":"10.18653\/v1\/D16-1174","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pilehvar Mohammad Taher","year":"2016"},{"key":"bib27","first-page":"1011","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Rios Miguel","year":"2018"},{"key":"bib28","volume-title":"Proceedings of the Eighth International Conference on Web Search and Data Mining, Shanghai, February 2-6","author":"R\u00f6der Michael","year":"2015"},{"key":"bib29","volume-title":"CoRR","author":"Rosner Frank","year":"2014"},{"key":"bib30","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1145\/3077136.3080806","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Shi Bei","year":"2017"},{"key":"bib31","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR)","author":"Srivastava Akash","year":"2017"},{"key":"bib32","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1007\/978-3-030-32381-3_16","volume-title":"China National Conference on Chinese Computational Linguistics","author":"Sun Chi","year":"2019"},{"key":"bib33","first-page":"1134","volume-title":"Proceedings of 5th International Joint Conference on Natural Language Processing","author":"Thater Stefan","year":"2011"},{"issue":"3","key":"bib34","volume":"3","author":"Tran Nam N.","year":"2017","journal-title":"JMIR Public Health and Surveillance"},{"key":"bib35","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR)","author":"Vilnis Luke","year":"2015"},{"key":"bib36","first-page":"1722","volume-title":"Advances in Neural Information Processing Systems 31","author":"Hongteng Xu","year":"2018"},{"key":"bib37","doi-asserted-by":"crossref","first-page":"207","DOI":"10.3115\/1621474.1621518","volume-title":"Proceedings of the 4th International Workshop on Semantic Evaluations","author":"Yuret Deniz","year":"2007"},{"key":"bib38","first-page":"5887","volume-title":"International Conference on Machine Learning (ICML)","author":"He Zhao","year":"2018"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00326","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:41Z","timestamp":1615585181000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/96487"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":38,"alternative-id":["10.1162\/tacl_a_00326"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00326","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}