{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T22:13:18Z","timestamp":1769724798054,"version":"3.49.0"},"reference-count":46,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T00:00:00Z","timestamp":1603411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The task of dialogue generation has attracted increasing attention due to its diverse downstream applications, such as question-answering systems and chatbots. Recently, the deep neural network (DNN)-based dialogue generation models have achieved superior performance against conventional models utilizing statistical machine learning methods. However, despite that an enormous number of state-of-the-art DNN-based models have been proposed, there lacks detailed empirical comparative analysis for them on the open Chinese corpus. As a result, relevant researchers and engineers might find it hard to get an intuitive understanding of the current research progress. To address this challenge, we conducted an empirical study for state-of-the-art DNN-based dialogue generation models in various Chinese corpora. Specifically, extensive experiments were performed on several well-known single-turn and multi-turn dialogue corpora, including KdConv, Weibo, and Douban, to evaluate a wide range of dialogue generation models that are based on the symmetrical architecture of Seq2Seq, RNNSearch, transformer, generative adversarial nets, and reinforcement learning respectively. Moreover, we paid special attention to the prevalent pre-trained model for the quality of dialogue generation. Their performances were evaluated by four widely-used metrics in this area: BLEU, pseudo, distinct, and rouge. Finally, we report a case study to show example responses generated by these models separately.<\/jats:p>","DOI":"10.3390\/sym12111756","type":"journal-article","created":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T02:44:12Z","timestamp":1603421052000},"page":"1756","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0519-7434","authenticated-orcid":false,"given":"Zhe","family":"Li","sequence":"first","affiliation":[{"name":"Xinjiang Laboratory of Multi-Language Information Technology, Xinjiang Multilingual Information Technology Research Center, College of Software, Xinjiang University, Urumqi 830046, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4231-6519","authenticated-orcid":false,"given":"Mieradilijiang","family":"Maimaiti","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7731-8688","authenticated-orcid":false,"given":"Jiabao","family":"Sheng","sequence":"additional","affiliation":[{"name":"College of Software, Xinjiang University, Urumqi 830046, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2589-8377","authenticated-orcid":false,"given":"Zunwang","family":"Ke","sequence":"additional","affiliation":[{"name":"College of Software, Xinjiang University, Urumqi 830046, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4592-7806","authenticated-orcid":false,"given":"Wushour","family":"Silamu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China"}]},{"given":"Qinyong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD 4702, Australia"}]},{"given":"Xiuhong","family":"Li","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,23]]},"reference":[{"key":"ref_1","unstructured":"Wang, H., Lu, Z., Li, H., and Chen, E. (2013, January 18\u201321). A dataset for research on short-text conversations. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Washington, DC, USA."},{"key":"ref_2","unstructured":"Hu, B., Lu, Z., Li, H., and Chen, Q. (2014, January 8\u201313). Convolutional neural network architectures for matching natural language sentences. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_3","unstructured":"Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., and Pineau, J. (2015). Hierarchical neural network generative models for movie dialogues. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., and Gao, J. (2016). Deep Reinforcement Learning for Dialogue Generation. arXiv.","DOI":"10.18653\/v1\/D16-1127"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Olabiyi, O., and Mueller, E.T. (2019). Multi-turn dialogue response generation with autoregressive transformer models. arXiv.","DOI":"10.18653\/v1\/W19-4114"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, X., Li, P., Bi, W., Liu, X., and Lam, W. (2019). Relevance-Promoting Language Model for Short-Text Conversation. arXiv.","DOI":"10.1609\/aaai.v34i05.6340"},{"key":"ref_7","unstructured":"Li, P. (2020). An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, Q., Yin, H., Wang, H., Hung, N.Q.V., Huang, Z., and Cui, L. (2019). Enhancing Collaborative Filtering with Generative Augmentation. KDD \u201919: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery.","DOI":"10.1145\/3292500.3330873"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, Q., Yin, H., Chen, T., Huang, Z., Wang, H., Zhao, Y., and Viet Hung, N.Q. (2020, January 20\u201324). Next Point-of-Interest Recommendation on Resource-Constrained Mobile Devices. Proceedings of the Web Conference, Taipei, Taiwan.","DOI":"10.1145\/3366423.3380170"},{"key":"ref_10","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, Q., Yin, H., Hu, Z., Lian, D., Wang, H., and Huang, Z. (2018). Neural Memory Streaming Recommender Networks with Adversarial Training. KDD \u201918: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery.","DOI":"10.1145\/3219819.3220004"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dai, Y., Li, H., Tang, C., Li, Y., Sun, J., and Zhu, X. (2020, January 9\u201310). Learning Low-Resource End-To-End Goal-Oriented Dialog for Fast and Reliable System Deployment. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Stroudsburg, PA, USA.","DOI":"10.18653\/v1\/2020.acl-main.57"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cui, L., Wu, Y., Liu, S., Zhang, Y., and Zhou, M. (2020). MuTual: A Dataset for Multi-Turn Dialogue Reasoning. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.130"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Qin, L., Xu, X., Che, W., Zhang, Y., and Liu, T. (2020). Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.565"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cho, H., and May, J. (2020). Grounding Conversations with Improvised Dialogues. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.218"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ma, W., Cui, Y., Liu, T., Wang, D., Wang, S., and Hu, G. (2020). Conversational Word Embedding for Retrieval-Based Dialog System. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.127"},{"key":"ref_17","unstructured":"Sutskever, I., and Vinyals, Q.V. (2014, January 8\u201313). Sequence to sequence learning with neural networks. Proceedings of the Advances in neural information processing systems, Montreal, QC, Canada."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., and Pineau, J. (2016, January 12\u201317). Building end-to-end dialogue systems using generative hierarchical neural network models. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.9883"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ghazvininejad, M., Brockett, C., Chang, M.W., Dolan, B., Gao, J., Yih, W.t., and Galley, M. (2018, January 2\u20137). A knowledge-grounded neural conversation model. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11977"},{"key":"ref_20","unstructured":"Vinyals, O., and Le, Q. (2015). A neural conversational model. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shang, L., Lu, Z., and Li, H. (2015, January 16\u201321). Neural Responding Machine for Short-Text Conversation. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, Beijing, China.","DOI":"10.3115\/v1\/P15-1152"},{"key":"ref_22","unstructured":"Zhang, H., Lan, Y., and Cheng, X. (August, January 28). ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-Turn Dialogue Generation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_23","unstructured":"Tian, Z., Yan, R., Mou, L., Song, Y., Feng, Y., and Zhao, D. (August, January 30). How to make context more useful? An empirical study on context-aware neural conversational models. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xing, C., Wu, W., Wu, Y., Zhou, M., Huang, Y., and Ma, W.Y. (2018, January 2\u20137). Hierarchical recurrent attention network for response generation. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11965"},{"key":"ref_25","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_26","unstructured":"Li, Z., Niu, C., Meng, F., Feng, Y., Li, Q., and Zhou, J. (August, January 28). Incremental Transformer with Deliberation Decoder for Document Grounded Conversations. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_27","unstructured":"Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., and Weston, J. (3, January 30). Wizard of Wikipedia: Knowledge-Powered Conversational Agents. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yu, L., Zhang, W., Wang, J., and Yu, Y. (2017, January 4\u20139). Seqgan: Sequence generative adversarial nets with policy gradient. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10804"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xu, Z., Liu, B., Wang, B., Sun, C.J., Wang, X., Wang, Z., and Qi, C. (2017, January 9\u201311). Neural response generation via gan with an approximate embedding layer. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1065"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Boyd, A., Puri, R., Shoeybi, M., Patwary, M., and Catanzaro, B. (2020). Large Scale Multi-Actor Generative Dialog Modeling. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.8"},{"key":"ref_31","unstructured":"Zhang, Z., Li, X., Gao, J., and Chen, E. (August, January 28). Budgeted Policy Learning for Task-Oriented Dialogue Systems. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Su, S.Y., Li, X., Gao, J., Liu, J., and Chen, Y.N. (November, January 31). Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1416"},{"key":"ref_33","unstructured":"Wu, Y., Li, X., Liu, J., Gao, J., and Yang, Y. (February, January 27). Switch-based active deep dyna-q: Efficient adaptive planning for task-completion dialogue policy learning. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_34","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"ref_35","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, MN, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ham, D., Lee, J.G., Jang, Y., and Kim, K.E. (2020). End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.54"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bao, S., He, H., Wang, F., Wu, H., and Wang, H. (2020). PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.9"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Shang, L., Lu, Z., and Li, H. (2015). Neural Responding Machine for Short-Text Conversation. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics.","DOI":"10.3115\/v1\/P15-1152"},{"key":"ref_39","unstructured":"Gao, J., Bi, W., Liu, X., Li, J., and Shi, S. (February, January 27). Generating multiple diverse responses for short-text conversation. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_40","unstructured":"Wu, Y., Wu, W., Xing, C., Zhou, M., and Li, Z. (August, January 30). Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pan, Z., Bai, K., Wang, Y., Zhou, L., and Liu, X. (2019, January 3\u20137). Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1191"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7\u201312). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B. (2016, January 12\u201317). A Diversity-Promoting Objective Function for Neural Conversation Models. Proceedings of the NAACL-HLT 2016, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-1014"},{"key":"ref_44","unstructured":"Lin, C.Y. (2004). ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out, Association for Computational Linguistics."},{"key":"ref_45","unstructured":"Kingma, D.P., and Welling, M. (2014). Auto-Encoding Variational Bayes. arXiv."},{"key":"ref_46","unstructured":"Zhao, T., Zhao, R., and Eskenazi, M. (August, January 30). Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/11\/1756\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:26:35Z","timestamp":1760178395000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/11\/1756"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,23]]},"references-count":46,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["sym12111756"],"URL":"https:\/\/doi.org\/10.3390\/sym12111756","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,23]]}}}