{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T22:49:46Z","timestamp":1762642186611,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,15]],"date-time":"2019-02-15T00:00:00Z","timestamp":1550188800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Human agents in technical customer support provide users with instructional answers to solve a task that would otherwise require a lot of time, money, energy, physical costs. Developing a dialogue system in this domain is challenging due to the broad variety of user questions. Moreover, user questions are noisy (for example, spelling mistakes), redundant and have various natural language expressions. In this work, we introduce a conversational system, MOLI (the name of our dialogue system), to solve customer questions by providing instructional answers from a knowledge base. Our approach combines models for question type and intent category classification with slot filling and a back-end knowledge base for filtering and ranking answers, and uses a dialog framework to actively query the user for missing information. For answer-ranking we find that sequential matching networks and neural multi-perspective sentence similarity networks clearly outperform baseline models, achieving a 43% error reduction. The end-to-end P@1(Precision at top 1) of MOLI was 0.69 and the customers\u2019 satisfaction was 0.73.<\/jats:p>","DOI":"10.3390\/info10020063","type":"journal-article","created":{"date-parts":[[2019,2,17]],"date-time":"2019-02-17T22:11:50Z","timestamp":1550441510000},"page":"63","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["MOLI: Smart Conversation Agent for Mobile Customer Service"],"prefix":"10.3390","volume":"10","author":[{"given":"Guoguang","family":"Zhao","sequence":"first","affiliation":[{"name":"No. 10, East Xibeiwang Rd., Haidian District, Beijing 100094, China"}]},{"given":"Jianyu","family":"Zhao","sequence":"additional","affiliation":[{"name":"No. 10, East Xibeiwang Rd., Haidian District, Beijing 100094, China"}]},{"given":"Yang","family":"Li","sequence":"additional","affiliation":[{"name":"No. 10, East Xibeiwang Rd., Haidian District, Beijing 100094, China"}]},{"given":"Christoph","family":"Alt","sequence":"additional","affiliation":[{"name":"Alt-Moabit 91c, 10559 Berlin, Germany"}]},{"given":"Robert","family":"Schwarzenberg","sequence":"additional","affiliation":[{"name":"Alt-Moabit 91c, 10559 Berlin, Germany"}]},{"given":"Leonhard","family":"Hennig","sequence":"additional","affiliation":[{"name":"Alt-Moabit 91c, 10559 Berlin, Germany"}]},{"given":"Stefan","family":"Schaffer","sequence":"additional","affiliation":[{"name":"Alt-Moabit 91c, 10559 Berlin, Germany"}]},{"given":"Sven","family":"Schmeier","sequence":"additional","affiliation":[{"name":"Alt-Moabit 91c, 10559 Berlin, Germany"}]},{"given":"Changjian","family":"Hu","sequence":"additional","affiliation":[{"name":"No. 10, East Xibeiwang Rd., Haidian District, Beijing 100094, China"}]},{"given":"Feiyu","family":"Xu","sequence":"additional","affiliation":[{"name":"No. 10, East Xibeiwang Rd., Haidian District, Beijing 100094, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, Y., Miao, Q., and Geng, J. (2018). Question Answering for Technical Customer Support. 7th CCF International Conference, Springer.","DOI":"10.1007\/978-3-319-99495-6_1"},{"key":"ref_2","unstructured":"Wen, T.H., Vandyke, D., and Mrksic, N. (arXiv, 2008). A Network-based End-to-End Trainable Task-oriented Dialogue System, arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kahaduwa, H., Pathirana, D., and Arachchi, P.L. (2017, January 29\u201331). Question Answering system for the travel domain. Proceedings of the Moratuwa Engineering Research Conference (MERCon), Moratuwa, Sri Lanka.","DOI":"10.1109\/MERCon.2017.7980526"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"31","DOI":"10.5087\/dad.2017.102","article-title":"Training end-to-end dialogue systems with the ubuntu dialogue corpus","volume":"8","author":"Lowe","year":"2017","journal-title":"Dialogue Discourse"},{"key":"ref_5","unstructured":"Feng, M., Xiang, B., and Glass, M.R. Applying deep learning to answer selection: A study and an open task. Proceedings of the 3rd International Proceedings on Automatic Speech Recognition and Understanding (ASRU)."},{"key":"ref_6","unstructured":"Wan, S., Dras, M., Dale, R., and Paris, C. (December, January 30). Using Dependency-based Features to Take the Para-farce out of Paraphrase. Proceedings of the Australasian Language Technology Workshop, Sydney, Australia."},{"key":"ref_7","unstructured":"Madnani, N., Tetreault, J., and Chodorow, M. (2012, January 3\u20138). Re-examining machine translation metrics for paraphrase identification. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, QC, Canada."},{"key":"ref_8","unstructured":"Fernando, S., and Stevenson, M. (2019, February 14). A Semantic Similarity Approach to Paraphrase Detection. Available online: https:\/\/pdfs.semanticscholar.org\/d020\/eb83f03a9f9c97e728355c4a9010fa65d8ef.pdf."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Das, D., and Smith, N.A. (2009, January 2\u20137). Paraphrase identification as probabilistic quasi-synchronous recognition. Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore.","DOI":"10.3115\/1687878.1687944"},{"key":"ref_10","unstructured":"Guo, W.W., and Diab, M. (2012, January 8\u201314). Modeling sentences in the latent space. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju Island, Korea."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Collobert, R., and Weston, J. (2008, January 5\u20139). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.","DOI":"10.1145\/1390156.1390177"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kalchbrenner, N., Grefenstette, E., and Blunsom, P. (arXiv, 2014). A convolutional neural network for modelling sentences, arXiv.","DOI":"10.3115\/v1\/P14-1062"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference on Empirical Methods for Natural Language Processing, Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_14","unstructured":"Hu, B., Lu, Z., Li, H., and Chen, Q. (arXiv, 2015). Convolutional neural network architectures for matching natural language sentences, arXiv."},{"key":"ref_15","unstructured":"Weston, J., Bengio, S., and Usunier, N. (2011, January 16\u201322). Wsabie: Scaling up to large vocabulary image annotation. Proceedings of the International Joint Conference on Artificial Intelligence, Barcelona, Spain."},{"key":"ref_16","unstructured":"Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., and Heck, L. (November, January 27). Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM International Conference on Information Knowledge Management, San Francisco, CA, USA."},{"key":"ref_17","unstructured":"Andrew, G., Arora, R., Bilmes, J., and Livescu, K. (2013, January 16\u201321). Deep canonical correlation analysis. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tai, K.S., Socher, R., and Manning, C.D. (arXiv, 2015). Improved semantic representations from tree-structured long short-term memory networks, arXiv.","DOI":"10.3115\/v1\/P15-1150"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"He, H., Gimpel, K., and Lin, J. (2015, January 17\u201321). Multi-perspective sentence similarity modeling with convolutional neural networks. Proceedings of the 20th International Proceedings on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1181"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wu, Y., Wu, W., and Xing, C. (2017). Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. 55th Annual Meeting of the Association for Computational Linguistics, ACL.","DOI":"10.18653\/v1\/P17-1046"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rajpurkar, P., Zhang, J., and Lopyrev, K. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. 21st International Proceedings on Empirical Methods in Natural Language Processing, ACL.","DOI":"10.18653\/v1\/D16-1264"},{"key":"ref_22","unstructured":"Yang, Y., Yih, S.W., and Meek, C. (2017). WikiQA: A Challenge Dataset for Open-Domain Question Answering. 20th International Proceedings on Empirical Methods in Natural Language Processing, ACL."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Manning, C., Surdeanu, M., and Bauer, J. (2014). The Stanford CoreNLP natural language processing toolkit. 52th Annual Meeting of the Association for Computational Linguistics, ACL.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, R., Lee, H., and Radev, D. (arXiv, 2016). Dependency sensitive convolutional neural networks for modeling sentences and documents, arXiv.","DOI":"10.18653\/v1\/N16-1177"},{"key":"ref_25","first-page":"39","article-title":"A C-LSTM Neural Network for Text Classification","volume":"1","author":"Zhou","year":"2015","journal-title":"Comput. Sci."},{"key":"ref_26","unstructured":"Mikolov, T., Sutskever, I., and Chen, K. (2014). Distributed representations of words and phrases and their compositionality. 9th International Proceedings on Advances in Neural Information Processing System, MIT Press."},{"key":"ref_27","unstructured":"Kusner, M., Sun, Y., and Kolkin, N. (2015). From word embeddings to document distances. 32nd International Proceedings on International Conference on Machine Learning, ACM."},{"key":"ref_28","unstructured":"Chung, J., Gulcehre, C., and Cho, K.H. (arXiv, 2014). Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word representation. 19th International Proceedings on Empirical Methods in Natural Language Processing, ACL.","DOI":"10.3115\/v1\/D14-1162"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/2\/63\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:32:29Z","timestamp":1760185949000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/2\/63"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,15]]},"references-count":29,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["info10020063"],"URL":"https:\/\/doi.org\/10.3390\/info10020063","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2019,2,15]]}}}