{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:25:32Z","timestamp":1750220732264,"version":"3.41.0"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,7,13]],"date-time":"2019-07-13T00:00:00Z","timestamp":1562976000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2017YFB1002201"],"award-info":[{"award-number":["2017YFB1002201"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Natural Science Fund for Distinguished Young Scholar","award":["61625204"],"award-info":[{"award-number":["61625204"]}]},{"name":"State Key Program of National Science Foundation of China","award":["61836006"],"award-info":[{"award-number":["61836006"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2020,1,31]]},"abstract":"<jats:p>It has been previously observed that training Variational Recurrent Autoencoders (VRAE) for text generation suffers from serious uninformative latent variables problems. The model would collapse into a plain language model that totally ignores the latent variables and can only generate repeating and dull samples. In this article, we explore the reason behind this issue and propose an effective regularizer-based approach to address it. The proposed method directly injects extra constraints on the posteriors of latent variables into the learning process of VRAE, which can flexibly and stably control the tradeoff between the Kullback-Leibler (KL) term and the reconstruction term, making the model learn dense and meaningful latent representations. The experimental results show that the proposed method outperforms several strong baselines and can make the model learn interpretable latent variables and generate diverse meaningful sentences. Furthermore, the proposed method can perform well without using other strategies, such as KL annealing.<\/jats:p>","DOI":"10.1145\/3341110","type":"journal-article","created":{"date-parts":[[2019,7,15]],"date-time":"2019-07-15T12:15:40Z","timestamp":1563192940000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["\u03bc-Forcing"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8755-8941","authenticated-orcid":false,"given":"Dayiheng","family":"Liu","sequence":"first","affiliation":[{"name":"College of Computer Science, State Key Laboratory of Hydraulics and Mountain River Engineering, Sichuan University, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Xue","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"He","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanyuan","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiancheng","family":"Lv","sequence":"additional","affiliation":[{"name":"College of Computer Science, State Key Laboratory of Hydraulics and Mountain River Engineering, Sichuan University, Chengdu, China;"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,7,13]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Alexander Alemi Ben Poole Ian Fischer Joshua Dillon Rif A. Saurous and Kevin Murphy. 2018. Fixing a broken elbo. In ICML.  Alexander Alemi Ben Poole Ian Fischer Joshua Dillon Rif A. Saurous and Kevin Murphy. 2018. Fixing a broken elbo. In ICML."},{"volume-title":"Jamie Ryan Kiros, and Geoffrey E. Hinton","year":"2016","author":"Ba Jimmy Lei","key":"e_1_2_1_2_1"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Samuel R. Bowman Luke Vilnis Oriol Vinyals Andrew M. Dai Rafal Jozefowicz and Samy Bengio. 2016. Generating sentences from a continuous space. In CONLL.  Samuel R. Bowman Luke Vilnis Oriol Vinyals Andrew M. Dai Rafal Jozefowicz and Samy Bengio. 2016. Generating sentences from a continuous space. In CONLL.","DOI":"10.18653\/v1\/K16-1002"},{"key":"e_1_2_1_4_1","unstructured":"Xi Chen Diederik P. Kingma Tim Salimans Yan Duan Prafulla Dhariwal John Schulman Ilya Sutskever and Pieter Abbeel. 2016. Variational lossy autoencoder. In ICML.  Xi Chen Diederik P. Kingma Tim Salimans Yan Duan Prafulla Dhariwal John Schulman Ilya Sutskever and Pieter Abbeel. 2016. Variational lossy autoencoder. In ICML."},{"volume-title":"Le","year":"2015","author":"Dai Andrew M.","key":"e_1_2_1_5_1"},{"volume-title":"Imagenet: A large-scale hierarchical image database. In CVPR.","year":"2009","author":"Deng Jia","key":"e_1_2_1_6_1"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Li Dong Shaohan Huang Furu Wei Mirella Lapata Ming Zhou and Ke Xu. 2017. Learning to generate product reviews from attributes. In EACL.  Li Dong Shaohan Huang Furu Wei Mirella Lapata Ming Zhou and Ke Xu. 2017. Learning to generate product reviews from attributes. In EACL.","DOI":"10.18653\/v1\/E17-1059"},{"volume-title":"van Amersfoort","year":"2014","author":"Fabius Otto","key":"e_1_2_1_8_1"},{"key":"e_1_2_1_9_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS.   Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS."},{"volume-title":"Nan Rosemary Ke, and Yoshua Bengio","year":"2017","author":"Alias Parth Goyal Anirudh Goyal","key":"e_1_2_1_10_1"},{"volume-title":"Danilo Jimenez Rezende, and Daan Wierstra","year":"2015","author":"Gregor Karol","key":"e_1_2_1_11_1"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"volume-title":"Workshop in Advances in Approximate Bayesian Inference, NIPS.","author":"Matthew","key":"e_1_2_1_13_1"},{"volume-title":"Rush","year":"2018","author":"Kim Yoon","key":"e_1_2_1_14_1"},{"volume-title":"Kingma and Jimmy Ba","year":"2014","author":"Diederik","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","unstructured":"Diederik P. Kingma Tim Salimans Rafal Jozefowicz Xi Chen Ilya Sutskever and Max Welling. 2016. Improved variational inference with inverse autoregressive flow. In NIPS.   Diederik P. Kingma Tim Salimans Rafal Jozefowicz Xi Chen Ilya Sutskever and Max Welling. 2016. Improved variational inference with inverse autoregressive flow. In NIPS."},{"volume-title":"Kingma and Max Welling","year":"2013","author":"Diederik","key":"e_1_2_1_17_1"},{"key":"e_1_2_1_18_1","unstructured":"Alireza Makhzani Jonathon Shlens Navdeep Jaitly Ian Goodfellow and Brendan Frey. 2015. Adversarial autoencoders. In ICML.  Alireza Makhzani Jonathon Shlens Navdeep Jaitly Ian Goodfellow and Brendan Frey. 2015. Adversarial autoencoders. In ICML."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783381"},{"key":"e_1_2_1_20_1","unstructured":"Tomas Mikolov Wen-tau Yih and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In NAACL.  Tomas Mikolov Wen-tau Yih and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In NAACL."},{"key":"e_1_2_1_21_1","unstructured":"Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In ICML.   Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In ICML."},{"key":"e_1_2_1_22_1","unstructured":"Danilo Jimenez Rezende Shakir Mohamed and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML.   Danilo Jimenez Rezende Shakir Mohamed and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML."},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Stanislau Semeniuta Aliaksei Severyn and Erhardt Barth. 2017. A hybrid convolutional variational autoencoder for text generation. In EMNLP.  Stanislau Semeniuta Aliaksei Severyn and Erhardt Barth. 2017. A hybrid convolutional variational autoencoder for text generation. In EMNLP.","DOI":"10.18653\/v1\/D17-1066"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Xiaoyu Shen Hui Su Shuzi Niu and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. In AAAI.  Xiaoyu Shen Hui Su Shuzi Niu and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. In AAAI.","DOI":"10.1609\/aaai.v32i1.11960"},{"volume-title":"S\u00f8ren Kaae S\u00f8nderby, and Ole Winther","year":"2016","author":"S\u00f8nderby Casper Kaae","key":"e_1_2_1_25_1"},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Martin Sundermeyer Ralf Schl\u00fcter and Hermann Ney. 2012. LSTM neural networks for language modeling. In INTERSPEECH.  Martin Sundermeyer Ralf Schl\u00fcter and Hermann Ney. 2012. LSTM neural networks for language modeling. In INTERSPEECH.","DOI":"10.21437\/Interspeech.2012-65"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1953039"},{"key":"e_1_2_1_28_1","unstructured":"Xinchen Yan Jimei Yang Kihyuk Sohn and Honglak Lee. 2016. Attribute2image: Conditional image generation from visual attributes. In ECCV.  Xinchen Yan Jimei Yang Kihyuk Sohn and Honglak Lee. 2016. Attribute2image: Conditional image generation from visual attributes. In ECCV."},{"key":"e_1_2_1_29_1","unstructured":"Zichao Yang Zhiting Hu Ruslan Salakhutdinov and Taylor Berg-Kirkpatrick. 2017. Improved variational autoencoders for text modeling using dilated convolutions. In ICML.   Zichao Yang Zhiting Hu Ruslan Salakhutdinov and Taylor Berg-Kirkpatrick. 2017. Improved variational autoencoders for text modeling using dilated convolutions. In ICML."},{"volume-title":"Tackling over-pruning in variational autoencoders. arXiv preprint arXiv:1706.03643","year":"2017","author":"Yeung Serena","key":"e_1_2_1_30_1"},{"key":"e_1_2_1_31_1","unstructured":"Lantao Yu Weinan Zhang Jun Wang and Yong Yu. 2017. SeqGAN: Sequence generative adversarial nets with policy gradient. In AAAI.   Lantao Yu Weinan Zhang Jun Wang and Yong Yu. 2017. SeqGAN: Sequence generative adversarial nets with policy gradient. In AAAI."},{"volume-title":"Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.02262","year":"2017","author":"Zhao Shengjia","key":"e_1_2_1_32_1"},{"key":"e_1_2_1_33_1","unstructured":"Tiancheng Zhao Ran Zhao and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In ACL.  Tiancheng Zhao Ran Zhao and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In ACL."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210080"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341110","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3341110","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:38:24Z","timestamp":1750199904000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341110"}},"subtitle":["Training Variational Recurrent Autoencoders for Text Generation"],"short-title":[],"issued":{"date-parts":[[2019,7,13]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,1,31]]}},"alternative-id":["10.1145\/3341110"],"URL":"https:\/\/doi.org\/10.1145\/3341110","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2019,7,13]]},"assertion":[{"value":"2018-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}