{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T17:29:11Z","timestamp":1779211751281,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,6]],"date-time":"2023-01-06T00:00:00Z","timestamp":1672963200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Basic Science Research Program through the NRF (National Research Foundation of Korea)","award":["2022R1H1A20925671112982076870101"],"award-info":[{"award-number":["2022R1H1A20925671112982076870101"]}]},{"name":"Basic Science Research Program through the NRF (National Research Foundation of Korea)","award":["GCU-202103390001"],"award-info":[{"award-number":["GCU-202103390001"]}]},{"name":"MSIT (Ministry of Science and Author ICT)","award":["2022R1H1A20925671112982076870101"],"award-info":[{"award-number":["2022R1H1A20925671112982076870101"]}]},{"name":"MSIT (Ministry of Science and Author ICT)","award":["GCU-202103390001"],"award-info":[{"award-number":["GCU-202103390001"]}]},{"name":"Gachon University research fund","award":["2022R1H1A20925671112982076870101"],"award-info":[{"award-number":["2022R1H1A20925671112982076870101"]}]},{"name":"Gachon University research fund","award":["GCU-202103390001"],"award-info":[{"award-number":["GCU-202103390001"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>For a task-oriented dialogue system to provide appropriate answers to and services for users\u2019 questions, it is necessary for it to be able to utilize knowledge related to the topic of the conversation. Therefore, the system should be able to select the most appropriate knowledge snippet from the knowledge base, where external unstructured knowledge is used to respond to user requests that cannot be solved by the internal knowledge addressed by the database or application programming interface. Therefore, this paper constructs a three-step knowledge-grounded task-oriented dialogue system with knowledge-seeking-turn detection, knowledge selection, and knowledge-grounded generation. In particular, we propose a hierarchical structure of domain-classification, entity-extraction, and snippet-ranking tasks by subdividing the knowledge selection step. Each task is performed through the pre-trained language model with advanced techniques to finally determine the knowledge snippet to be used to generate a response. Furthermore, the domain and entity information obtained because of the previous task is used as knowledge to reduce the search range of candidates, thereby improving the performance and efficiency of knowledge selection and proving it through experiments.<\/jats:p>","DOI":"10.3390\/s23020685","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T06:38:27Z","timestamp":1673246307000},"page":"685","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Knowledge-Grounded Task-Oriented Dialogue System with Hierarchical Structure for Enhancing Knowledge Selection"],"prefix":"10.3390","volume":"23","author":[{"given":"Hayoung","family":"Lee","sequence":"first","affiliation":[{"name":"School of Computing, Gachon University, 1342 Sujeong-gu, Seongnam-si 13120, Gyeonggi-do, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Okran","family":"Jeong","sequence":"additional","affiliation":[{"name":"School of Computing, Gachon University, 1342 Sujeong-gu, Seongnam-si 13120, Gyeonggi-do, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3151","DOI":"10.1007\/s10115-022-01744-y","article-title":"Conversational question answering: A survey","volume":"64","author":"Zaib","year":"2022","journal-title":"Knowl. Inf. Syst."},{"key":"ref_2","first-page":"1","article-title":"Conversational AI: Dialogue systems, conversational agents, and chatbots","volume":"13","author":"McTear","year":"2020","journal-title":"Synth. Lect. Hum. Lang. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ponnusamy, P., Ghias, A.R., Guo, C., and Sarikaya, R. (2020, January 7\u201312). Feedback-based self-learning in large-scale conversational ai agents. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i08.7022"},{"key":"ref_4","unstructured":"Ram, A., Prasad, R., Khatri, C., Venkatesh, A., Gabriel, R., Liu, Q., Nunn, J., Hedayatnia, B., Cheng, M., and Nagar, A. (2018). Conversational ai: The science behind the alexa prize. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Ou, Z., and Yu, Z. (2020, January 7\u201312). Task-oriented dialog systems that consider multiple appropriate responses under the same context. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6507"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Henderson, M., Vuli\u0107, I., Gerz, D., Casanueva, I., Budzianowski, P., Coope, S., Spithourakis, G., Wen, T.H., Mrk\u0161i\u0107, N., and Su, P.H. (2019). Training neural response selection for task-oriented dialogue systems. arXiv.","DOI":"10.18653\/v1\/P19-1536"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kim, S., Eric, M., Gopalakrishnan, K., Hedayatnia, B., Liu, Y., and Hakkani-Tur, D. (2020). Beyond domain APIs: Task-oriented conversational modeling with unstructured knowledge access. arXiv.","DOI":"10.18653\/v1\/2020.sigdial-1.35"},{"key":"ref_8","unstructured":"Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., and Weston, J. (2018). Wizard of wikipedia: Knowledge-powered conversational agents. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhou, H., Young, T., Huang, M., Zhao, H., Xu, J., and Zhu, X. (2018, January 13\u201319). Commonsense knowledge aware conversation generation with graph attention. Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/643"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ma, X., Xu, P., Wang, Z., Nallapati, R., and Xiang, B. (2019, January 3). Domain adaptation with BERT-based domain classification and data selection. Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP, Hong kong, China.","DOI":"10.18653\/v1\/D19-6109"},{"key":"ref_11","unstructured":"He, H., Lu, H., Bao, S., Wang, F., Wu, H., Niu, Z., and Wang, H. (2021). Learning to select external knowledge with multi-scale negative sampling. arXiv."},{"key":"ref_12","unstructured":"Fu, B., Qiu, Y., Tang, C., Li, Y., Yu, H., and Sun, J. (2020). A survey on complex question answering over knowledge base: Recent advances and challenges. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gopalakrishnan, K., Hedayatnia, B., Chen, Q., Gottardi, A., Kwatra, S., Venkatesh, A., Gabriel, R., Hakkani-T\u00fcr, D., and Amazon Alexa, A.I. (2019). Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations. Proc. Interspeech, 1891\u20131895.","DOI":"10.21437\/Interspeech.2019-3079"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Budzianowski, P., Wen, T.H., Tseng, B.H., Casanueva, I., Ultes, S., Ramadan, O., and Ga\u0161i\u0107, M. (2018). MultiWOZ\u2014a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv.","DOI":"10.18653\/v1\/D18-1547"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, H., Liu, Z., Xiong, C., and Liu, Z. (2019). Grounded conversation generation as guided traverses in commonsense knowledge graphs. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.184"},{"key":"ref_16","first-page":"20179","article-title":"A simple language model for task-oriented dialogue","volume":"33","author":"McCann","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wu, C.S., Hoi, S., Socher, R., and Xiong, C. (2020). TOD-BERT: Pre-trained natural language understanding for task-oriented dialogue. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-main.66"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ghazvininejad, M., Brockett, C., Chang, M.W., Dolan, B., Gao, J., Yih, W.T., and Galley, M. (2018, January 2\u20137). A knowledge-grounded neural conversation model. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11977"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhao, X., Wu, W., Xu, C., Tao, C., Zhao, D., and Yan, R. (2020). Knowledge-grounded dialogue generation with pre-trained language models. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-main.272"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Song, Y., Yan, R., Li, C.T., Nie, J.Y., Zhang, M., and Zhao, D. (2018, January 13\u201319). An Ensemble of Retrieval-Based and Generation-Based Human-Computer Conversation Systems. Proceedings of the 27th International Joint Conference on Artificial Intelligence(IJCAI), Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/609"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yan, R., and Zhao, D. (2018, January 19\u201323). Coupled context modeling for deep chit-chat: Towards conversations between human and computer. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3220045"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, X., Tao, C., Wu, W., Xu, C., Zhao, D., and Yan, R. (2019). A document-grounded matching network for response selection in retrieval-based chatbots. arXiv.","DOI":"10.24963\/ijcai.2019\/756"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gu, J.C., Ling, Z.H., Liu, Q., Chen, Z., and Zhu, X. (2020). Filtering before iteratively referring for knowledge-grounded response selection in retrieval-based chatbots. arXiv.","DOI":"10.18653\/v1\/2020.findings-emnlp.127"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/TKDE.2020.2981314","article-title":"A Survey on Deep Learning for Named Entity Recognition","volume":"34","author":"Li","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1016\/j.jbi.2013.08.004","article-title":"Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts","volume":"46","author":"Zhang","year":"2013","journal-title":"J. Biomed. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ji, Z., Sun, A., Cong, G., and Han, J. (2016, January 11\u201315). Joint recognition and linking of fine-grained locations from tweets. Proceedings of the 25th International Conference on World Wide Web, Montreal, QC, Canada.","DOI":"10.1145\/2872427.2883067"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, S., Sun, Y., Li, B., Wang, W., and Zhao, X. (2020, January 7\u201312). HAMNER: Headword amplified multi-span distantly supervised method for domain specific named entity recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6358"},{"key":"ref_28","unstructured":"Ritter, A., Clark, S., and Etzioni, O. (2011, January 27\u201331). Named entity recognition in tweets: An experimental study. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Einburgh, UK."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shen, Y., Yun, H., Lipton, Z.C., Kronrod, Y., and Anandkumar, A. (2017). Deep active learning for named entity recognition. arXiv.","DOI":"10.18653\/v1\/W17-2630"},{"key":"ref_30","unstructured":"Liu, L., Ren, X., Shang, J., Peng, J., and Han, J. (November, January 31). Efficient contextualized representation: Language model pruning for sequence labeling. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, L., Shang, J., Ren, X., Xu, F., Gui, H., Peng, J., and Han, J. (,  2018). Empower sequence labeling with task-aware neural language model. Proceedings of the AAAI Conference on Artificial Intelligence.","DOI":"10.1609\/aaai.v32i1.12006"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hakala, K., and Pyysalo, S. (2019, January 4). Biomedical named entity recognition with multilingual BERT. Proceedings of the 5th Workshop on BioNLP Open Shared Tasks, Hong Kong, China.","DOI":"10.18653\/v1\/D19-5709"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv.","DOI":"10.18653\/v1\/N16-1030"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Pappas, D., and Androutsopoulos, I. (2021). A neural model for joint document and snippet ranking in question answering for large document collections. arXiv.","DOI":"10.18653\/v1\/2021.acl-long.301"},{"key":"ref_35","unstructured":"Han, J., Shin, J., Song, H., Jo, H., Kim, G., Kim, Y., and Choi, S.J. (2022). External Knowledge Selection with Weighted Negative Sampling in Knowledge-grounded Task-oriented Dialogue Systems. arXiv."},{"key":"ref_36","unstructured":"Eric, M., Goel, R., Paul, S., Sethi, A., Agarwal, S., Gao, S., Kumar, A., Goyal, A.K., Ku, P., and Hakkani-Tur, D. (2020, January 11\u201316). MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. Proceedings of the 12th Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_37","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_38","unstructured":"Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv."},{"key":"ref_39","unstructured":"Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., and Le, Q.V. (2019). XLNet: Generalized autoregressive pretraining for language understanding. Adv. Neural Inf. Process. Syst., 5574\u20135764."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1002\/grl.50288","article-title":"GPT2: Empirical slant delay model for radio space geodetic techniques","volume":"40","author":"Lagler","year":"2013","journal-title":"Geophys. Res. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Silalahi, S., Ahmad, T., and Studiawan, H. (2022, January 6\u20137). Named entity recognition for drone forensic using BERT and distilbert. Proceedings of the 2022 International Conference on Data Science and Its Applications (ICoDSA), Bandung, Indonesia.","DOI":"10.1109\/ICoDSA55874.2022.9862916"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sharma, A., and Pandey, H. (2020, January 16\u201320). LRG at TREC 2020: Document Ranking with XLNet-Based Models. Proceedings of the Twenty-Ninth Text Retrieval Conference (TREC), Gaithersburg, MD, USA.","DOI":"10.6028\/NIST.SP.1266.podcast-LRG_REtrievers"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Arabadzhieva-Kalcheva, N., and Kovachev, I. (2022, January 2\u20134). Comparison of BERT and XLNet accuracy with classical methods and algorithms in text classification. Proceedings of the 2021 International Conference on Biomedical Innovations and Applications (BIA), Varna, Bulgaria.","DOI":"10.1109\/BIA52594.2022.9831281"},{"key":"ref_44","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv."},{"key":"ref_45","unstructured":"Mi, H., Ren, Q., Dai, Y., He, Y., Sun, J., Li, Y., Zheng, J., and Xu, P. (2021, January 8\u20139). Towards generalized models for beyond domain api task-oriented dialogue. Proceedings of the AAAI-21 DSTC9 Workshop, virtual event."},{"key":"ref_46","unstructured":"Clark, K., Luong, M.T., Le, Q.V., and Manning, C.D. (2020). Electra: Pre-training text encoders as discriminators rather than generators. arXiv."},{"key":"ref_47","unstructured":"Tang, L., Shang, Q., Lv, K., Fu, Z., Zhang, S., Huang, C., and Zhang, Z. (2021, January 8\u20139). RADGE: Relevance Learning and Generation Evaluating Method for Task-Oriented Conversational Systems. Proceedings of the AAAI 2021 Workshop DSTC9, Virtual Event."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/685\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:02:12Z","timestamp":1760119332000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/685"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,6]]},"references-count":47,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020685"],"URL":"https:\/\/doi.org\/10.3390\/s23020685","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,6]]}}}