{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T00:20:31Z","timestamp":1767918031310,"version":"3.49.0"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2022,3,9]],"date-time":"2022-03-09T00:00:00Z","timestamp":1646784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2022,10,31]]},"abstract":"<jats:p>In topic models, collections are organized as documents where they arise as mixtures over latent clusters called topics. A topic is a distribution over the vocabulary. In large-scale applications, parametric or finite topic mixture models such as LDA (latent Dirichlet allocation) and its variants are very restrictive in performance due to their reduced hypothesis space. In this article, we address the problem related to model selection and sharing ability of topics across multiple documents in standard parametric topic models. We propose as an alternative a BNP (Bayesian nonparametric) topic model where the HDP (hierarchical Dirichlet process) prior models documents topic mixtures through their multinomials on infinite simplex. We, therefore, propose asymmetric BL (Beta-Liouville) as a diffuse base measure at the corpus level DP (Dirichlet process) over a measurable space. This step illustrates the highly heterogeneous structure in the set of all topics that describes the corpus probability measure. For consistency in posterior inference and predictive distributions, we efficiently characterize random probability measures whose limits are the global and local DPs to approximate the HDP from the stick-breaking formulation with the GEM (Griffiths-Engen-McCloskey) random variables. Due to the diffuse measure with the BL prior as conjugate to the count data distribution, we obtain an improved version of the standard HDP that is usually based on symmetric Dirichlet (Dir). In addition, to improve coordinate ascent framework while taking advantage of its deterministic nature, our model implements an online optimization method based on stochastic, at document level, variational inference to accommodate fast topic learning when processing large collections of text documents with natural gradient. The high value in the predictive likelihood per document obtained when compared to the performance of its competitors is also consistent with the robustness of our fully asymmetric BL-based HDP. While insuring the predictive accuracy of the model using the probability of the held-out documents, we also added a combination of metrics such as the topic coherence and topic diversity to improve the quality and interpretability of the topics discovered. We also compared the performance of our model using these metrics against the standard symmetric LDA. We show that online HDP-LBLA (Latent BL Allocation)\u2019s performance is the asymptote for parametric topic models. The accuracy in the results (improved predictive distributions of the held out) is a product of the model\u2019s ability to efficiently characterize dependency between documents (topic correlation) as now they can easily share topics, resulting in a much robust and realistic compression algorithm for information modeling.<\/jats:p>","DOI":"10.1145\/3502727","type":"journal-article","created":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T14:03:20Z","timestamp":1646921000000},"page":"1-48","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Stochastic Variational Optimization of a Hierarchical Dirichlet Process Latent Beta-Liouville Topic Model"],"prefix":"10.1145","volume":"16","author":[{"given":"Koffi Eddy","family":"Ihou","sequence":"first","affiliation":[{"name":"Concordia University, Montreal, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manar","family":"Amayri","sequence":"additional","affiliation":[{"name":"Grenoble Institute of Technology, Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nizar","family":"Bouguila","sequence":"additional","affiliation":[{"name":"Concordia University, Montreal, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,3,9]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"19","volume-title":"Proceedings of the 11th International Conference on Artificial Intelligence and Statistics","author":"Ahmed Amr","year":"2007","unstructured":"Amr Ahmed and Eric P. Xing. 2007. Seeking the truly correlated topic posterior\u2014on tight approximate inference of logistic-normal admixture model. In Proceedings of the 11th International Conference on Artificial Intelligence and Statistics. Marina Meila and Xiaotong Shen (Eds.), JMLR.org, 19\u201326. Retrieved from http:\/\/proceedings.mlr.press\/v2\/ahmed07a.html."},{"key":"e_1_3_2_3_2","first-page":"27","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Asuncion Arthur U.","year":"2009","unstructured":"Arthur U. Asuncion, Max Welling, Padhraic Smyth, and Yee Whye Teh. 2009. On smoothing and inference for topic models. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Jeff A. Bilmes and Andrew Y. Ng (Eds.), AUAI Press, 27\u201334. Retrieved from https:\/\/dslpitt.org\/uai\/displayArticleDetails.jsp?mmnu=1&smnu=2&article_id=1663&proceeding_id=25."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-55032-4_28"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2015.09.044"},{"key":"e_1_3_2_6_2","volume-title":"Pattern Recognition and Machine Learning","author":"Bishop Christopher M.","year":"2006","unstructured":"Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer Science+ Business Media."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/1123816"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2133806.2133826"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1667053.1667056"},{"key":"e_1_3_2_10_2","first-page":"147","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 18","author":"Blei David M.","year":"2005","unstructured":"David M. Blei and John D. Lafferty. 2005. Correlated topic models. In Proceedings of the Advances in Neural Information Processing Systems 18. 147\u2013154. Retrieved from http:\/\/papers.nips.cc\/paper\/2906-correlated-topic-models."},{"key":"e_1_3_2_11_2","first-page":"993","article-title":"Latent dirichlet allocation","volume":"3","author":"Blei David M.","year":"2003","unstructured":"David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3, Jan (2003), 993\u20131022.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_12_2","unstructured":"Arnim Bleier. 2013. Practical collapsed stochastic variational inference for the HDP. CoRR abs\/1312.0412. http:\/\/arxiv.org\/abs\/1312.0412."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2007.190726"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2010.2091428"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/tkde.2011.162"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2011.09.037"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10044-011-0236-8"},{"key":"e_1_3_2_18_2","first-page":"185","volume-title":"Proceedings of the 22nd Annual Conference on Neural Information Processing Systems","author":"Boyd-Graber Jordan L.","year":"2008","unstructured":"Jordan L. Boyd-Graber and David M. Blei. 2008. Syntactic topic models. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems. Daphne Koller, Dale Schuurmans, Yoshua Bengio, and L\u00e9on Bottou (Eds.), Curran Associates, Inc., 185\u2013192. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2008\/hash\/8d3bba7425e7c98c50f52ca1b52d3735-Abstract.html."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623691"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-71246-8_12"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2396860"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1007\/978-3-642-23780-5_29","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"Chen Changyou","year":"2011","unstructured":"Changyou Chen, Lan Du, and Wray Buntine. 2011. Sampling table configurations for the hierarchical poisson-dirichlet process. In Machine Learning and Knowledge Discovery in Databases. Dimitrios Gunopulos, Thomas Hofmann, Donato Malerba, and Michalis Vazirgiannis (Eds.). Springer, Berlin, 296\u2013311."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2019.01.018"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00325"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143881"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2011.81"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2012.09.047"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2938830"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176342360"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176342752"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487697"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1646396.1646430"},{"key":"e_1_3_2_33_2","first-page":"475","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Ghahramani Zoubin","year":"2006","unstructured":"Zoubin Ghahramani and Thomas L. Griffiths. 2006. Infinite latent feature models and the Indian buffet process. In Proceedings of the Advances in Neural Information Processing Systems. 475\u2013482."},{"key":"e_1_3_2_34_2","first-page":"856","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Hoffman Matthew D.","year":"2010","unstructured":"Matthew D. Hoffman, Francis R. Bach, and David M. Blei. 2010. Online learning for latent dirichlet allocation. In Proceedings of the Advances in Neural Information Processing Systems. 856\u2013864."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.5555\/2567709.2502622"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPTA.2017.8310106"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/MWSCAS.2018.8623978"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2018.12.046"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2019.103364"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2372618"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974348.82"},{"key":"e_1_3_2_42_2","first-page":"688","volume-title":"Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)","author":"Liang Percy","year":"2007","unstructured":"Percy Liang, Slav Petrov, Michael I. Jordan, and Dan Klein. 2007. The infinite PCFG using hierarchical Dirichlet processes. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). 688\u2013697."},{"key":"e_1_3_2_43_2","volume-title":"Proceedings of the Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP\/CoNLL)","author":"Liang Percy","year":"2007","unstructured":"Percy Liang, Slav Petrov, Michael I. Jordan, and Dan Klein. 2007. The infinite PCFG using hierarchical dirichlet processes. In Proceedings of the Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP\/CoNLL)."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551518822455"},{"key":"e_1_3_2_45_2","unstructured":"Andrew Kachites McCallum. 2002. Mallet: A machine learning for language toolkit. Retrieved July 2021 from http:\/\/mallet.cs.umass.edu (2002)."},{"key":"e_1_3_2_46_2","unstructured":"David Mimno Matt Hoffman and David Blei. 2012. Sparse stochastic inference for latent Dirichlet allocation. arXiv:1206.6425. Retrieved from https:\/\/arxiv.org\/abs\/1206.6425."},{"key":"e_1_3_2_47_2","first-page":"100","volume-title":"Proceedings of the Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics","author":"Newman David","year":"2010","unstructured":"David Newman, Jey Han Lau, Karl Grieser, and Timothy Baldwin. 2010. Automatic evaluation of topic coherence. In Proceedings of the Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics. The Association for Computational Linguistics, 100\u2013108. Retrieved from https:\/\/aclanthology.org\/N10-1012\/."},{"key":"e_1_3_2_48_2","first-page":"74","volume-title":"Proceedings of the 14th International Conference on Artificial Intelligence and Statistics","author":"Paisley John","year":"2011","unstructured":"John Paisley, Chong Wang, and David Blei. 2011. The discrete infinite logistic normal distribution for mixed-membership modeling. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 74\u201382."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3153018"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0963548302005163"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1214\/aop\/1024404422"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553481"},{"key":"e_1_3_2_53_2","unstructured":"Jipeng Qiang Zhenyu Qian Yun Li Yunhao Yuan and Xindong Wu. 2019. Short text topic modeling techniques applications and performance: A survey. arXiv:1904.07695. Retrieved from https:\/\/arxiv.org\/abs\/1904.07695."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339550"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835890"},{"key":"e_1_3_2_56_2","first-page":"639","article-title":"A constructive definition of Dirichlet priors","volume":"4","author":"Sethuraman Jayaram","year":"1994","unstructured":"Jayaram Sethuraman. 1994. A constructive definition of Dirichlet priors. Statistica Sinica 4 (1994), 639\u2013650.","journal-title":"Statistica Sinica"},{"key":"e_1_3_2_57_2","first-page":"1585","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Sudderth Erik B.","year":"2009","unstructured":"Erik B. Sudderth and Michael I. Jordan. 2009. Shared segmentation of natural scenes using dependent Pitman-Yor processes. In Proceedings of the Advances in Neural Information Processing Systems. 1585\u20131592."},{"key":"e_1_3_2_58_2","volume-title":"A Bayesian Interpretation of Interpolated Kneser-Ney","author":"Teh Yee Whye","year":"2006","unstructured":"Yee Whye Teh. 2006. A Bayesian Interpretation of Interpolated Kneser-Ney. Technical Report."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220299"},{"key":"e_1_3_2_60_2","unstructured":"Yee Whye Teh Dilan G\u00f6r\u00fcr and Zoubin Ghahramani. 2007. Stick-breaking construction for the indian buffet process. InProceedings of the Machine Learning Research. Marina Meila and Xiaotong Shen (Eds.) PMLR San Juan Puerto Rico 556\u2013563. Retrieved from http:\/\/proceedings.mlr.press\/v2\/teh07a.html."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511802478.006"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000000302"},{"key":"e_1_3_2_63_2","first-page":"1481","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Teh Yee Whye","year":"2008","unstructured":"Yee Whye Teh, Kenichi Kurihara, and Max Welling. 2008. Collapsed variational inference for HDP. In Proceedings of the Advances in Neural Information Processing Systems. 1481\u20131488."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA629956"},{"key":"e_1_3_2_65_2","first-page":"1973","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Wallach Hanna M.","year":"2009","unstructured":"Hanna M. Wallach, David M. Mimno, and Andrew McCallum. 2009. Rethinking LDA: Why priors matter. In Proceedings of the Advances in Neural Information Processing Systems. 1973\u20131981."},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553515"},{"key":"e_1_3_2_67_2","first-page":"1990","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Wang Chong","year":"2009","unstructured":"Chong Wang and David M. Blei. 2009. Variational inference for the nested Chinese restaurant process. In Proceedings of the Advances in Neural Information Processing Systems. 1990\u20131998."},{"key":"e_1_3_2_68_2","first-page":"413","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Wang Chong","year":"2012","unstructured":"Chong Wang and David M. Blei. 2012. Truncation-free online variational inference for Bayesian nonparametric models. In Proceedings of the Advances in Neural Information Processing Systems. 413\u2013421."},{"key":"e_1_3_2_69_2","first-page":"752","volume-title":"Proceedings of the 14th International Conference on Artificial Intelligence and Statistics","author":"Wang Chong","year":"2011","unstructured":"Chong Wang, John Paisley, and David Blei. 2011. Online variational inference for the hierarchical Dirichlet process. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 752\u2013760."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2492565"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.41"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502727","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3502727","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:47Z","timestamp":1750183787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502727"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,9]]},"references-count":70,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,10,31]]}},"alternative-id":["10.1145\/3502727"],"URL":"https:\/\/doi.org\/10.1145\/3502727","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,9]]},"assertion":[{"value":"2020-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}