{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T08:32:22Z","timestamp":1768638742405,"version":"3.49.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,1,22]],"date-time":"2016-01-22T00:00:00Z","timestamp":1453420800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100013286","name":"Specialized Research Fund for the Doctoral Program of Higher Education of China","doi-asserted-by":"crossref","award":["20110091110003"],"award-info":[{"award-number":["20110091110003"]}],"id":[{"id":"10.13039\/501100013286","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61300158, 61170181"],"award-info":[{"award-number":["61300158, 61170181"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Graduate Research and Innovation Projects in Jiangsu Province","award":["CXZZ12_0058"],"award-info":[{"award-number":["CXZZ12_0058"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2016,3,8]]},"abstract":"<jats:p>Neural network language models (LMs) are shown to be effective in improving the performance of statistical machine translation (SMT) systems. However, state-of-the-art neural network LMs usually use words before the current position as context and neglect global topic information, which can help machine translation (MT) systems to select better translation candidates from a higher perspective. In this work, we propose improvement of the state-of-the-art feedforward neural language model with topic information. Two main issues need to be tackled when adding topics into neural network LMs for SMT: one is how to incorporate topics to the neural network; the other is how to get target-side topic distribution before translation. We incorporate topics by appending topic distribution to the input layer of a feedforward LM. We adopt a multinomial logistic-regression (MLR) model to predict the target-side topic distribution based on source side information. Moreover, we propose a feedforward neural network model to learn joint representations on the source side for topic prediction. LM experiments demonstrate that the perplexity on validation set can be greatly reduced by the topic-enhanced feedforward LM, and the prediction of target-side topics can be improved dramatically with the MLR model equipped with the joint source representations. A final MT experiment, conducted on a large-scale Chinese--English dataset, shows that our feedforward LM with predicted topics improves the translation performance against a strong baseline.<\/jats:p>","DOI":"10.1145\/2816816","type":"journal-article","created":{"date-parts":[[2016,1,26]],"date-time":"2016-01-26T13:25:01Z","timestamp":1453814701000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Adaptation of Language Models for SMT Using Neural Networks with Topic Information"],"prefix":"10.1145","volume":"15","author":[{"given":"Yinggong","family":"Zhao","sequence":"first","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shujian","family":"Huang","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin-Yu","family":"Dai","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiajun","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,1,22]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics","author":"Auli Michael","year":"2013"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1083"},{"key":"e_1_2_1_3_1","article-title":"A neural probabilistic language model","author":"Bengio Yoshua","year":"2003","journal-title":"Journal of Machine Learning Research. 1137--1155."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/146680.146685"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2007.33.2.201"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 49th ACL-HLT. Association for Computational Linguistics","author":"Clark Jonathan H.","year":"2031"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1129"},{"key":"e_1_2_1_9_1","volume-title":"IEEE International Conference on Neural Networks, 1996","volume":"1","author":"Goller C.","year":"1996"},{"key":"e_1_2_1_10_1","unstructured":"Joshua Goodman. 2001. Classes for fast maximum entropy training. CoRR cs.CL\/0108006 (2001). http:\/\/arxiv.org\/abs\/cs.CL\/0108006.  Joshua Goodman. 2001. Classes for fast maximum entropy training. CoRR cs.CL\/0108006 (2001). http:\/\/arxiv.org\/abs\/cs.CL\/0108006."},{"key":"e_1_2_1_11_1","unstructured":"\u00c7aglar G\u00fcl\u00e7ehre Orhan Firat Kelvin Xu Kyunghyun Cho Lo\u00efc Barrault Huei-Chi Lin Fethi Bougares Holger Schwenk and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. CoRR abs\/1503.03535 (2015). http:\/\/arxiv.org\/abs\/1503.03535.  \u00c7aglar G\u00fcl\u00e7ehre Orhan Firat Kelvin Xu Kyunghyun Cho Lo\u00efc Barrault Huei-Chi Lin Fethi Bougares Holger Schwenk and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. CoRR abs\/1503.03535 (2015). http:\/\/arxiv.org\/abs\/1503.03535."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of AISTATS.","author":"Gutmann Michael","year":"2010"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1110"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1062"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of EMNLP","author":"Koehn Philipp","year":"2004"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073445.1073462"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics","author":"Le Hai-Son","year":"2012"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning","author":"Le Quoc","year":"2014"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220252"},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Thang Luong Ilya Sutskever Quoc V. Le Oriol Vinyals and Wojciech Zaremba. 2014. Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206.  Thang Luong Ilya Sutskever Quoc V. Le Oriol Vinyals and Wojciech Zaremba. 2014. Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206.","DOI":"10.3115\/v1\/P15-1002"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/1610075.1610083"},{"key":"e_1_2_1_22_1","unstructured":"Tom\u00e1\u0161 Mikolov. 2012. Statistical Language Models Based on Neural Networks. Ph.D. Dissertation. Brno University of Technology Brno Czech Republic.  Tom\u00e1\u0161 Mikolov. 2012. Statistical Language Models Based on Neural Networks. Ph.D. Dissertation. Brno University of Technology Brno Czech Republic."},{"key":"e_1_2_1_23_1","unstructured":"Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR abs\/1301.3781 (2013). http:\/\/arxiv.org\/abs\/1301.3781.  Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR abs\/1301.3781 (2013). http:\/\/arxiv.org\/abs\/1301.3781."},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of INTERSPEECH.","author":"Mikolov Tom\u00e1\u0161"},{"key":"e_1_2_1_25_1","volume-title":"Spoken Language Technologies","author":"Mikolov Tomas"},{"key":"e_1_2_1_26_1","unstructured":"Andriy Mnih and Geoffrey E. Hinton. 2009. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems 21. 1081--1088.  Andriy Mnih and Geoffrey E. Hinton. 2009. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems 21. 1081--1088."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of ICML. 807--814","author":"Nair Vinod"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1162\/089120103321337421"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 6th Workshop on Statistical Machine Translation. Association for Computational Linguistics","author":"Ruiz Nick","year":"2011"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of ACL-08: HLT. Association for Computational Linguistics","author":"Shen Libin","year":"2008"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of EMNLP. Association for Computational Linguistics","author":"Socher Richard","year":"2013"},{"key":"e_1_2_1_32_1","unstructured":"Ilya Sutskever Oriol Vinyals and Quoc V. V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. 3104--3112.  Ilya Sutskever Oriol Vinyals and Quoc V. V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. 3104--3112."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of ACL. Association for Computational Linguistics, Prague, Czech Republic, 520--527","author":"Tam Yik-Cheung","year":"2007"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of EMNLP. Association for Computational Linguistics","author":"Vaswani Ashish","year":"2013"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1151"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of ACL. Association for Computational Linguistics","author":"Xiao Xinyan","year":"2012"},{"key":"e_1_2_1_37_1","volume-title":"Retrieved","author":"Xiong Deyi","year":"2013"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the 6th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing","author":"Yu Heng","year":"2013"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.5555\/2693068.2693069"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-12277-9_16"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2816816","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2816816","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:48:19Z","timestamp":1750225699000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2816816"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,1,22]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,3,8]]}},"alternative-id":["10.1145\/2816816"],"URL":"https:\/\/doi.org\/10.1145\/2816816","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,1,22]]},"assertion":[{"value":"2015-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-01-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}