{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T22:56:30Z","timestamp":1768344990469,"version":"3.49.0"},"reference-count":34,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T00:00:00Z","timestamp":1743984000000},"content-version":"vor","delay-in-days":96,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,3,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Neural topic modeling is a scalable automated technique for text data mining. In various downstream tasks of topic modeling, it is preferred that the discovered topics well align with labels. However, due to the lack of guidance from labels, unsupervised neural topic models are less powerful in this situation. Existing supervised neural topic models often adopt a label-free prior to generate the latent document-topic distributions and use them to predict the labels and thus achieve label-topic alignment indirectly. Such a mechanism faces the following issues: 1) The label-free prior leads to topics blending the latent patterns of multiple labels; and 2) One is unable to intuitively identify the explicit relationships between labels and the discovered topics. To tackle these problems, we develop a novel supervised neural topic model which utilizes a chain-structured graphical model with a label-conditioned prior. Soft indicators are introduced to explicitly construct the label-topic relationships. To obtain well-organized label-topic relationships, we formalize an entropy-regularized optimal transport problem on the embedding space and model them as the transport plan. Moreover, our proposed method can be flexibly integrated with most existing unsupervised neural topic models. Experimental results on multiple datasets demonstrate that our model can greatly enhance the alignment between labels and topics while maintaining good topic quality.<\/jats:p>","DOI":"10.1162\/tacl_a_00738","type":"journal-article","created":{"date-parts":[[2025,4,7]],"date-time":"2025-04-07T18:49:37Z","timestamp":1744051777000},"page":"249-263","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":4,"title":["Supervised Neural Topic Modeling with Label Alignment"],"prefix":"10.1162","volume":"13","author":[{"given":"Ruihao","family":"Chen","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China. chenrh29@mail2.sysu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hegang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China. chenhg25@mail2.sysu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuyin","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China. luyy37@mail2.sysu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanghui","family":"Rao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China. raoyangh@mail.sysu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chunjiang","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of North Carolina at Greensboro, USA. chunjiang.zhu@uncg.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2025,3,19]]},"reference":[{"key":"2025051914252198900_bib1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/1120.003.0082","article-title":"Latent dirichlet allocation","volume-title":"Advances in Neural Information Processing Systems","author":"Blei","year":"2001"},{"issue":"1","key":"2025051914252198900_bib2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9499","article-title":"A novel neural topic model and its supervised extension","volume":"29","author":"Cao","year":"2015","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2025051914252198900_bib3","doi-asserted-by":"publisher","first-page":"2031","DOI":"10.18653\/v1\/P18-1189","article-title":"Neural models for documents with metadata","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Card","year":"2018"},{"key":"2025051914252198900_bib4","article-title":"Sinkhorn distances: Lightspeed computation of optimal transport","volume-title":"Advances in Neural Information Processing Systems","author":"Cuturi","year":"2013"},{"key":"2025051914252198900_bib5","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1162\/tacl_a_00325","article-title":"Topic modeling in embedding spaces","volume":"8","author":"Dieng","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025051914252198900_bib6","first-page":"511","article-title":"Kernel topic models","volume-title":"Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics","author":"Hennig","year":"2012"},{"key":"2025051914252198900_bib7","first-page":"2018","article-title":"Is automated topic model evaluation broken? The incoherence of coherence","volume-title":"Advances in Neural Information Processing Systems","author":"Hoyle","year":"2021"},{"key":"2025051914252198900_bib8","doi-asserted-by":"publisher","first-page":"5321","DOI":"10.18653\/v1\/2022.findings-emnlp.390","article-title":"Are neural topic models broken?","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2022","author":"Hoyle","year":"2022"},{"key":"2025051914252198900_bib9","doi-asserted-by":"publisher","first-page":"4652","DOI":"10.18653\/v1\/D18-1494","article-title":"Siamese network-based supervised topic modeling","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Huang","year":"2018"},{"key":"2025051914252198900_bib10","article-title":"Adam: A method for stochastic optimization","volume-title":"3rd International Conference on Learning Representations","author":"Kingma","year":"2015"},{"key":"2025051914252198900_bib11","article-title":"Auto-encoding variational bayes","volume-title":"2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14\u201316, 2014, Conference Track Proceedings","author":"Kingma","year":"2014"},{"key":"2025051914252198900_bib12","article-title":"DiscLDA: Discriminative learning for dimensionality reduction and classification","volume-title":"Advances in Neural Information Processing Systems","author":"Lacoste-Julien","year":"2008"},{"key":"2025051914252198900_bib13","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/B978-1-55860-377-6.50048-7","article-title":"Newsweeder: Learning to filter netnews","volume-title":"Proceedings of the Twelfth International Conference on Machine Learning","author":"Lang","year":"1995"},{"key":"2025051914252198900_bib14","doi-asserted-by":"publisher","first-page":"530","DOI":"10.3115\/v1\/E14-1056","article-title":"Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality","volume-title":"Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics","author":"Lau","year":"2014"},{"key":"2025051914252198900_bib15","first-page":"142","article-title":"Learning word vectors for sentiment analysis","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Maas","year":"2011"},{"key":"2025051914252198900_bib16","first-page":"800","article-title":"SSHLDA: A semi-supervised hierarchical topic model","volume-title":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","author":"Mao","year":"2012"},{"key":"2025051914252198900_bib17","article-title":"Supervised topic models","volume-title":"Advances in Neural Information Processing Systems","author":"Mcauliffe","year":"2007"},{"key":"2025051914252198900_bib18","first-page":"2410","article-title":"Discovering discrete latent topics with neural variational inference","volume-title":"Proceedings of the 34th International Conference on Machine Learning","author":"Miao","year":"2017"},{"key":"2025051914252198900_bib19","first-page":"1727","article-title":"Neural variational inference for text processing","volume-title":"Proceedings of The 33rd International Conference on Machine Learning","author":"Miao","year":"2016"},{"key":"2025051914252198900_bib20","article-title":"Hierarchically supervised latent dirichlet allocation","volume-title":"Advances in Neural Information Processing Systems","author":"Perotte","year":"2011"},{"key":"2025051914252198900_bib21","first-page":"670","article-title":"A hierarchical model of web summaries","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Petinot","year":"2011"},{"key":"2025051914252198900_bib22","doi-asserted-by":"publisher","first-page":"248","DOI":"10.3115\/1699510.1699543","article-title":"Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora","volume-title":"Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing","author":"Ramage","year":"2009"},{"key":"2025051914252198900_bib23","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1145\/2684822.2685324","article-title":"Exploring the space of topic coherence measures","volume-title":"Proceedings of the Eighth ACM International Conference on Web Search and Data Mining","author":"R\u00f6der","year":"2015"},{"key":"2025051914252198900_bib24","article-title":"Autoencoding variational inference for topic models","volume-title":"International Conference on Learning Representations","author":"Srivastava","year":"2017"},{"key":"2025051914252198900_bib25","article-title":"Sharing clusters among related groups: Hierarchical dirichlet processes","volume-title":"Advances in Neural Information Processing Systems","author":"Teh","year":"2004"},{"key":"2025051914252198900_bib26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICCCI.2018.8441220","article-title":"Topic model based opinion mining and sentiment analysis","volume-title":"2018 International Conference on Computer Communication and Informatics (ICCCI)","author":"Vamshi","year":"2018"},{"issue":"86","key":"2025051914252198900_bib27","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"van der Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"2025051914252198900_bib28","article-title":"Representing mixtures of word embeddings with mixtures of topic embeddings","volume-title":"International Conference on Learning Representations","author":"Wang","year":"2022"},{"key":"2025051914252198900_bib29","first-page":"37335","article-title":"Effective neural topic modeling with embedding clustering regularization","volume-title":"Proceedings of the 40th International Conference on Machine Learning","author":"Xiaobao","year":"2023"},{"issue":"17","key":"2025051914252198900_bib30","doi-asserted-by":"publisher","first-page":"19261","DOI":"10.1609\/aaai.v38i17.29895","article-title":"On the affinity, rationality, and diversity of hierarchical topic modeling","volume":"38","author":"Xiaobao","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"8","key":"2025051914252198900_bib31","doi-asserted-by":"publisher","first-page":"2098","DOI":"10.1109\/TMM.2019.2951194","article-title":"Knowledge-based topic model for multi-modal social event analysis","volume":"22","author":"Xue","year":"2020","journal-title":"IEEE Transactions on Multimedia"},{"key":"2025051914252198900_bib32","article-title":"Character-level convolutional networks for text classification","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang","year":"2015"},{"key":"2025051914252198900_bib33","article-title":"Neural topic model via optimal transport","volume-title":"International Conference on Learning Representations","author":"He","year":"2021"},{"key":"2025051914252198900_bib34","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1145\/1553374.1553535","article-title":"MedLDA: Maximum margin supervised topic models for regression and classification","volume-title":"Proceedings of the 26th Annual International Conference on Machine Learning","author":"Zhu","year":"2009"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00738\/2511931\/tacl_a_00738.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00738\/2511931\/tacl_a_00738.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,19]],"date-time":"2025-05-19T18:25:37Z","timestamp":1747679137000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00738\/128712\/Supervised-Neural-Topic-Modeling-with-Label"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":34,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00738","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025]]},"published":{"date-parts":[[2025]]}}}