{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T13:41:18Z","timestamp":1770903678529,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,2,26]],"date-time":"2019-02-26T00:00:00Z","timestamp":1551139200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Recent advances in deep neural networks, language modeling and language generation have introduced new ideas to the field of conversational agents. As a result, deep neural models such as sequence-to-sequence, memory networks, and the Transformer have become key ingredients of state-of-the-art dialog systems. While those models are able to generate meaningful responses even in unseen situations, they need a lot of training data to build a reliable model. Thus, most real-world systems have used traditional approaches based on information retrieval (IR) and even hand-crafted rules, due to their robustness and effectiveness, especially for narrow-focused conversations. Here, we present a method that adapts a deep neural architecture from the domain of machine reading comprehension to re-rank the suggested answers from different models using the question as a context. We train our model using negative sampling based on question\u2013answer pairs from the Twitter Customer Support Dataset. The experimental results show that our re-ranking framework can improve the performance in terms of word overlap and semantics both for individual models as well as for model combinations.<\/jats:p>","DOI":"10.3390\/info10030082","type":"journal-article","created":{"date-parts":[[2019,2,26]],"date-time":"2019-02-26T11:00:44Z","timestamp":1551178844000},"page":"82","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Machine Reading Comprehension for Answer Re-Ranking in Customer Support Chatbots"],"prefix":"10.3390","volume":"10","author":[{"given":"Momchil","family":"Hardalov","sequence":"first","affiliation":[{"name":"Faculty of Mathematics and Informatics, Sofia University, 1164 Sofia, Bulgaria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivan","family":"Koychev","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Informatics, Sofia University, 1164 Sofia, Bulgaria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Preslav","family":"Nakov","sequence":"additional","affiliation":[{"name":"Qatar Computing Research Institute, Hamad Bin Khalifa University, 34110 Doha, Qatar"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,26]]},"reference":[{"key":"ref_1","unstructured":"Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., and Le, Q.V. (May, January 30). QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension. Proceedings of the 2018 International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_2","unstructured":"Seo, M., Kembhavi, A., Farhadi, A., and Hajishirzi, H. (2017, January 24\u201326). Bi-directional attention flow for machine comprehension. Proceedings of the 2017 International Conference on Learning Representations, Toulon, France."},{"key":"ref_3","unstructured":"Chen, D., Fisch, A., Weston, J., and Bordes, A. (August, January 30). Reading Wikipedia to Answer Open-Domain Questions. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7\u201312). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_5","unstructured":"Lin, C.Y. (2004, January 25\u201326). ROUGE: A Package for Automatic Evaluation of Summaries. Proceedings of the ACL Workshop on Text Summarization Branches Out, Barcelona, Spain."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lowe, R., Pow, N., Serban, I., and Pineau, J. (2015, January 2\u20134). The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems. Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Prague, Czech Republic.","DOI":"10.18653\/v1\/W15-4640"},{"key":"ref_7","unstructured":"Lison, P., and Tiedemann, J. (2016, January 23\u201328). OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. Proceedings of the Tenth International Conference on Language Resources and Evaluation, Portoro\u017e, Slovenia."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Reddy, S., Chen, D., and Manning, C.D. (arXiv, 2018). CoQA: A conversational question answering challenge, arXiv.","DOI":"10.1162\/tacl_a_00266"},{"key":"ref_9","unstructured":"(2019, February 20). Microsoft Research Social Media Conversation Corpus. Available online: http:\/\/research.microsoft.com\/convo\/."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Luong, T., Pham, H., and Manning, C.D. (2015, January 17\u201321). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_11","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to Sequence Learning with Neural Networks. Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_12","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (arXiv, 2014). Neural machine translation by jointly learning to align and translate, arXiv."},{"key":"ref_13","unstructured":"Vinyals, O., and Le, Q.V. (arXiv, 2015). A Neural Conversational Model, arXiv."},{"key":"ref_14","unstructured":"Serban, I.V., Sordoni, A., Bengio, Y., Courville, A.C., and Pineau, J. (arXiv, 2015). Hierarchical neural network generative models for movie dialogues, arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J.Y., Gao, J., and Dolan, B. (June, January 31). A Neural Network Approach to Context-Sensitive Generation of Conversational Responses. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, USA.","DOI":"10.3115\/v1\/N15-1020"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., and Nie, J.Y. (2015, January 18\u201323). A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia.","DOI":"10.1145\/2806416.2806493"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Boyanov, M., Nakov, P., Moschitti, A., Da San Martino, G., and Koychev, I. (2017, January 2\u20138). Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics. Proceedings of the International Conference Recent Advances in Natural Language Processing, Varna, Bulgaria.","DOI":"10.26615\/978-954-452-049-6_018"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nakov, P., Hoogeveen, D., M\u00e0rquez, L., Moschitti, A., Mubarak, H., Baldwin, T., and Verspoor, K. (2017, January 3\u20134). SemEval-2017 Task 3: Community Question Answering. Proceedings of the 11th International Workshop on Semantic Evaluation, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/S17-2003"},{"key":"ref_19","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., and Polosukhin, I. (2017, January 4\u20139). Attention is All you Need. Proceedings of the 30th Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hardalov, M., Koychev, I., and Nakov, P. (2018, January 12\u201314). Towards Automated Customer Support. Proceedings of the 18th International Conference on Artificial Intelligence: Methodology, Systems, and Applications, Varna, Bulgaria.","DOI":"10.1007\/978-3-319-99344-7_5"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Y., Miao, Q., Geng, J., Alt, C., Schwarzenberg, R., Hennig, L., Hu, C., and Xu, F. (2018, January 26\u201330). Question Answering for Technical Customer Support. Proceedings of the 7th CCF International Conference, NLPCC 2018, Hohhot, China.","DOI":"10.1007\/978-3-319-99495-6_1"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Feng, M., Xiang, B., Glass, M.R., Wang, L., and Zhou, B. (2015, January 13\u201317). Applying deep learning to answer selection: A study and an open task. Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding, Scottsdale, AZ, USA.","DOI":"10.1109\/ASRU.2015.7404872"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Back-propagation Applied to Handwritten Zip Code Recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Comput."},{"key":"ref_24","unstructured":"Li, X., Li, L., Gao, J., He, X., Chen, J., Deng, L., and He, J. (2016, January 2\u20134). Recurrent reinforcement learning: A hybrid approach. Proceedings of the 2016 International Conference on Learning Representations, San Juan, Puerto Rico."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_26","unstructured":"Qiu, M., Li, F.L., Wang, S., Gao, X., Chen, Y., Zhao, W., Chen, H., Huang, J., and Chu, W. (August, January 30). AliMe Chat: A Sequence to Sequence and Rerank based Chatbot Engine. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Cui, L., Huang, S., Wei, F., Tan, C., Duan, C., and Zhou, M. (August, January 30). SuperAgent: A Customer Service Chatbot for E-commerce Websites. Proceedings of the Association for Computational Linguistics 2017, System Demonstrations, Vancouver, BC, Cananda.","DOI":"10.18653\/v1\/P17-4017"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Pang, L., Lan, Y., Guo, J., Xu, J., Xu, J., and Cheng, X. (2017, January 6\u201310). DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, Singapore.","DOI":"10.1145\/3132847.3132914"},{"key":"ref_29","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5\u20138). Distributed Representations of Words and Phrases and Their Compositionality. Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_30","unstructured":"Srivastava, R.K., Greff, K., and Schmidhuber, J. (arXiv, 2015). Highway networks, arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C. (2014, January 25\u201329). GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018, January 1\u20136). Deep Contextualized Word Representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA. Association for Computational Linguistics.","DOI":"10.18653\/v1\/N18-1202"},{"key":"ref_33","unstructured":"Ba, J.L., Kiros, J.R., and Hinton, G.E. (arXiv, 2016). Layer normalization, arXiv."},{"key":"ref_34","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5087\/dad.2018.101","article-title":"A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version","volume":"9","author":"Serban","year":"2018","journal-title":"Dialog. Discourse"},{"key":"ref_36","unstructured":"(2019, February 18). Customer Support on Twitter. Available online: http:\/\/www.kaggle.com\/thoughtvector\/customer-support-on-twitter."},{"key":"ref_37","unstructured":"(2019, February 12). Codebase of Towards Automated Customer Support. Available online: https:\/\/github.com\/mhardalov\/customer-support-chatbot."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., and McClosky, D. (2014, January 22\u201327). The Stanford CoreNLP Natural Language Processing Toolkit. Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MA, USA.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 2015 International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_40","first-page":"265","article-title":"Tensorflow: A system for large-scale machine learning","volume":"16","author":"Abadi","year":"2016","journal-title":"OSDI"},{"key":"ref_41","unstructured":"(2019, February 20). ElasticSearch. Available online: http:\/\/www.elastic.co\/products\/elasticsearch."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1561\/1500000019","article-title":"The Probabilistic Relevance Framework: BM25 and Beyond","volume":"3","author":"Robertson","year":"2009","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Liu, C.W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., and Pineau, J. (2016, January 1\u20135). How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.","DOI":"10.18653\/v1\/D16-1230"},{"key":"ref_44","unstructured":"Lowe, R., Noseworthy, M., Serban, I.V., Angelard-Gontier, N., Bengio, Y., and Pineau, J. (August, January 30). Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rus, V., and Lintean, M. (2012, January 7). A Comparison of Greedy and Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics. Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, Montreal, QC, Canada.","DOI":"10.1007\/978-3-642-30950-2_116"},{"key":"ref_46","unstructured":"Forgues, G., Pineau, J., Larchev\u00eaque, J.M., and Tremblay, R. (2014, January 12). Bootstrapping dialog systems with word embeddings. Proceedings of the NIPS Workshop on Modern Machine Learning and Natural Language Processing, Montreal, QC, Canada."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lin, C.Y., and Och, F.J. (2004, January 21\u201326). Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. Proceedings of the 42nd Annual Conference of the Association for Computational Linguistics, Barcelona, Spain.","DOI":"10.3115\/1218955.1219032"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Introduction to Reinforcement Learning, MIT Press.","DOI":"10.1109\/TNN.1998.712192"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/3\/82\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:34:41Z","timestamp":1760186081000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/3\/82"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,26]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,3]]}},"alternative-id":["info10030082"],"URL":"https:\/\/doi.org\/10.3390\/info10030082","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,26]]}}}