{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T07:15:31Z","timestamp":1761808531370,"version":"build-2065373602"},"reference-count":24,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,3,18]],"date-time":"2022-03-18T00:00:00Z","timestamp":1647561600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003329","name":"Ministry of Economy, Industry and Competitiveness","doi-asserted-by":"publisher","award":["TIN2016-76770-R and BES-2017-082017"],"award-info":[{"award-number":["TIN2016-76770-R and BES-2017-082017"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["TIN2016-76770-R and BES-2017-082017"],"award-info":[{"award-number":["TIN2016-76770-R and BES-2017-082017"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010067","name":"Gobierno de Arag\u00f3n","doi-asserted-by":"publisher","award":["Reference Group T31_20R"],"award-info":[{"award-number":["Reference Group T31_20R"]}],"id":[{"id":"10.13039\/501100010067","id-type":"DOI","asserted-by":"publisher"}]},{"name":"FEDER 2014-2020 \u201cConstruyendo 285 Europa desde Arag\u00f3n\u201d","award":["FEDER 2014-2020"],"award-info":[{"award-number":["FEDER 2014-2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study evaluates the impacts of slot tagging and training data length on joint natural language understanding (NLU) models for medication management scenarios using chatbots in Spanish. In this study, we define the intents (purposes of the sentences) for medication management scenarios and two types of slot tags. For training the model, we generated four datasets, combining long\/short sentences with long\/short slots, while for testing, we collect the data from real interactions of users with a chatbot. For the comparative analysis, we chose six joint NLU models (SlotRefine, stack-propagation framework, SF-ID network, capsule-NLU, slot-gated modeling, and a joint SLU-LM model) from the literature. The results show that the best performance (with a sentence-level semantic accuracy of 68.6%, an F1-score of 76.4% for slot filling, and an accuracy of 79.3% for intent detection) is achieved using short sentences and short slots. Our results suggest that joint NLU models trained with short slots yield better results than those trained with long slots for the slot filling task. The results also indicate that short slots could be a better choice for the dialog system because of their simplicity. Importantly, the work demonstrates that the performance of the joint NLU models can be improved by selecting the correct slot configuration according to the usage scenario.<\/jats:p>","DOI":"10.3390\/s22062364","type":"journal-article","created":{"date-parts":[[2022,3,20]],"date-time":"2022-03-20T21:37:17Z","timestamp":1647812237000},"page":"2364","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2608-6431","authenticated-orcid":false,"given":"Surya","family":"Roca","sequence":"first","affiliation":[{"name":"Arag\u00f3n Institute of Engineering Research (I3A), University of Zaragoza, 50018 Zaragoza, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6865-4989","authenticated-orcid":false,"given":"Sophie","family":"Rosset","sequence":"additional","affiliation":[{"name":"Laboratoire Interdisciplinaire des Sciences du Num\u00e9rique, CNRS, Universit\u00e9 Paris-Saclay, 91405 Orsay, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9485-7678","authenticated-orcid":false,"given":"Jos\u00e9","family":"Garc\u00eda","sequence":"additional","affiliation":[{"name":"Arag\u00f3n Institute of Engineering Research (I3A), University of Zaragoza, 50018 Zaragoza, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5254-1402","authenticated-orcid":false,"given":"\u00c1lvaro","family":"Alesanco","sequence":"additional","affiliation":[{"name":"Arag\u00f3n Institute of Engineering Research (I3A), University of Zaragoza, 50018 Zaragoza, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MC.2018.3191249","article-title":"Social media\u2014Based conversational agents for health management and interventions","volume":"51","author":"Wang","year":"2018","journal-title":"Computer"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1720","DOI":"10.1016\/j.pec.2017.04.015","article-title":"Engaging women with an embodied conversational agent to deliver mindfulness and lifestyle recommendations: A feasibility randomized control trial","volume":"100","author":"Gardiner","year":"2017","journal-title":"Patient Educ. Couns."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Stoica, A., Kadar, T., Lemnaru, C., Potolea, R., and D\u00een\u015foreanu, M. (2019, January 5\u20137). The impact of data challenges on intent detection and slot filling for the home assistant scenario. Proceedings of the 2019 IEEE 15th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.","DOI":"10.1109\/ICCP48234.2019.8959642"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, Y., Patel, A., Shen, Y., and Jin, H. (2018, January 2\u20136). A Deep Reinforcement Learning Based Multimodal Coaching Model (DCM) for Slot Filling in Spoken Language Understanding (SLU). Proceedings of the INTERSPEECH, Hyderabad, India.","DOI":"10.21437\/Interspeech.2018-1379"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kobayashi, Y., Yoshida, T., Iwata, K., and Fujimura, H. (2019, January 15\u201319). Slot Filling with Weighted Multi-Encoders for Out-of-Domain Values. Proceedings of the INTERSPEECH, Graz, Austria.","DOI":"10.21437\/Interspeech.2019-1226"},{"key":"ref_6","unstructured":"Adel, H., and Sch\u00fctze, H. (2017). Impact of Coreference Resolution on Slot Filling. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kim, J.K., Tur, G., Celikyilmaz, A., Cao, B., and Wang, Y.Y. (2016, January 13\u201316). Intent detection using semantically enriched word embeddings. Proceedings of the 2016 IEEE Spoken Language Technology Workshop (SLT), San Diego, CA, USA.","DOI":"10.1109\/SLT.2016.7846297"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, W., Hosseini, S., Awadallah, A.H., Bennett, P.N., and Quirk, C. (2019, January 21\u201325). Context-aware intent identification in email conversations. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.","DOI":"10.1145\/3331184.3331260"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Goo, C.W., Gao, G., Hsu, Y.K., Huo, C.L., Chen, T.C., Hsu, K.W., and Chen, Y.N. (2018, January 1\u20136). Slot-gated modeling for joint slot filling and intent prediction. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana.","DOI":"10.18653\/v1\/N18-2118"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, B., and Lane, I. (2016). Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv.","DOI":"10.21437\/Interspeech.2016-1352"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, C., Huang, Z., and Hu, M. (2020, January 4\u20136). SASGBC: Improving sequence labeling performance for joint learning of slot filling and intent detection. Proceedings of the 2020 the 6th International Conference on Computing and Data Engineering (ICCDE), Sanya, China.","DOI":"10.1145\/3379247.3379266"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jbi.2018.09.008","article-title":"A comparison of word embeddings for the biomedical natural language processing","volume":"87","author":"Wang","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ghannay, S., Neuraz, A., and Rosset, S. (2020, January 4\u20138). What is best for spoken language understanding: Small but task-dependant embeddings or huge but out-of-domain embeddings?. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053278"},{"key":"ref_14","unstructured":"Roca, S., Rosset, S., Garc\u00eda, J., and Alesanco, \u00c1. (December, January 29). Evaluation of Embeddings in Medication Domain for Spanish Language Using Joint Natural Language Understanding. Proceedings of the European Medical and Biological Engineering Conference, Portoro\u017e, Slovenia."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sushil, M., Suster, S., and Daelemans, W. (2021, January 11). Are we there yet? Exploring clinical domain knowledge of BERT models. Proceedings of the 20th Workshop on Biomedical Language Processing, Online.","DOI":"10.18653\/v1\/2021.bionlp-1.5"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dimovski, M., Musat, C., Ilievski, V., Hossmann, A., and Baeriswyl, M. (2018). Submodularity-inspired data selection for goal-oriented chatbot training based on sentence embeddings. arXiv.","DOI":"10.24963\/ijcai.2018\/559"},{"key":"ref_17","unstructured":"Boulanger, H. (2020, January 8\u201319). \u00c9valuation syst\u00e9matique d\u2019une m\u00e9thode commune de g\u00e9n\u00e9ration. Proceedings of the 6e Conf\u00e9rence Conjointe Journ\u00e9es D\u2019\u00c9tudes sur la Parole (JEP, 33e \u00e9dition), Traitement Automatique des Langues Naturelles (TALN, 27e \u00e9dition), Rencontre des \u00c9tudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (R\u00c9CITAL, 22e \u00e9dition), Volume 3: Rencontre des \u00c9tudiants Chercheurs en Informatique pour le TAL, Nancy, France. Available online: https:\/\/hal.archives-ouvertes.fr\/hal-02786185\/."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.chb.2015.02.026","article-title":"Real conversations with artificial intelligence: A comparison between human\u2014Human online conversations and human\u2014Chatbot conversations","volume":"49","author":"Hill","year":"2015","journal-title":"Comput. Hum. Behav."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wu, D., Ding, L., Lu, F., and Xie, J. (2020). SlotRefine: A fast non-autoregressive model for joint intent detection and slot filling. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-main.152"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Qin, L., Che, W., Li, Y., Wen, H., and Liu, T. (2019). A stack-propagation framework with token-level intent detection for spoken language understanding. arXiv.","DOI":"10.18653\/v1\/D19-1214"},{"key":"ref_21","unstructured":"Niu, P., Chen, Z., and Song, M. (2019). A novel bi-directional interrelated model for joint intent detection and slot filling. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, C., Li, Y., Du, N., Fan, W., and Yu, P.S. (2018). Joint slot filling and intent detection via capsule neural networks. arXiv.","DOI":"10.18653\/v1\/P19-1519"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, B., and Lane, I. (2016). Joint online spoken language understanding and language modeling with recurrent neural networks. arXiv.","DOI":"10.18653\/v1\/W16-3603"},{"key":"ref_24","unstructured":"(2021, December 15). R: The R Project for Statistical Computing. Available online: https:\/\/www.r-project.org\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2364\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:39:09Z","timestamp":1760135949000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2364"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,18]]},"references-count":24,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22062364"],"URL":"https:\/\/doi.org\/10.3390\/s22062364","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,3,18]]}}}