{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T21:26:42Z","timestamp":1771104402386,"version":"3.50.1"},"reference-count":71,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,6,3]],"date-time":"2023-06-03T00:00:00Z","timestamp":1685750400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,3]],"date-time":"2023-06-03T00:00:00Z","timestamp":1685750400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["766287"],"award-info":[{"award-number":["766287"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005278","name":"Universidad de Antioquia","doi-asserted-by":"publisher","award":["PRG2020-34068"],"award-info":[{"award-number":["PRG2020-34068"]}],"id":[{"id":"10.13039\/501100005278","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Pratech","award":["PI2019-24110"],"award-info":[{"award-number":["PI2019-24110"]}]},{"DOI":"10.13039\/501100005278","name":"University of Antioquia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005278","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Chatbots enable the automation of several components in customer service and allow the support of multiple users. Despite their multiple advantages, due to the large amount of conversations generated by a chatbot, it is difficult to determine whether customer requests are well-addressed. For practical reasons, chatbot\u2019s effectiveness is evaluated manually based upon a small sample (randomly chosen) of conversations or through self-reported user satisfaction. This procedure does not guarantee the correct evaluation of the service because the sample is generally not large enough and self-reports might be influenced by different external factors not directly associated to the chatbot\u2019s functioning. This study proposes a methodology for automatic evaluation of chatbot effectiveness in real production environments. The analysis considers convolutional neural networks adapted for natural language processing, using two parallel convolutional layers to evaluate questions and answers independently. The proposed model also incorporates filters to extract features with multiple temporal resolution. This methodology is tested upon real conversations of chatbots that provide service to two different companies. The results are compared to baseline models based on classical techniques with different pre-trained word embedding models. According to our results, the proposed approach provides accuracies between 78.95% and 80.18%, which outperforms the best result of the baseline models by 2.9%.<\/jats:p>","DOI":"10.1007\/s11042-023-14896-y","type":"journal-article","created":{"date-parts":[[2023,6,3]],"date-time":"2023-06-03T06:01:46Z","timestamp":1685772106000},"page":"5473-5492","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Evaluation of effectiveness in conversations between humans and chatbots using parallel convolutional neural networks with multiple temporal resolutions"],"prefix":"10.1007","volume":"83","author":[{"given":"Daniel","family":"Escobar-Grisales","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4946-9232","authenticated-orcid":false,"given":"Juan Camilo","family":"V\u00e1squez-Correa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Juan Rafael","family":"Orozco-Arroyave","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,6,3]]},"reference":[{"key":"14896_CR1","unstructured":"Aksu H (2013) Customer Service: the new proactive marketing. https:\/\/www.huffpost.com\/entry\/customer-service-the-new-_b_2827889?guccounter=1. Accessed 2021 Feb 17"},{"key":"14896_CR2","unstructured":"Bakarov A (2018) A survey of word embeddings evaluation methods. arXiv:1801.09536"},{"issue":"1","key":"14896_CR3","first-page":"1","volume":"12","author":"H Basak","year":"2022","unstructured":"Basak H et al (2022) A union of deep learning and swarm-based optimization for 3D human action recognition. Sci Reports 12(1):1\u201317","journal-title":"Sci Reports"},{"key":"14896_CR4","unstructured":"Cahn J (2017) Chatbot: architecture, design, & development. University of Pennsylvania School of Engineering and Applied Science Department of Computer and Information Science"},{"key":"14896_CR5","unstructured":"Canete J et al (2020) Spanish pre-trained bert model and evaluation data. PML4DC at ICLR 2020"},{"key":"14896_CR6","doi-asserted-by":"crossref","unstructured":"Caselles-Dupr\u00e9 H, Lesaint F, Royo-Letelier J (2018) Word2vec applied to recommendation: hyperparameters matter. In: Proceedings of the 12th ACM conference on recommender systems (RecSys), pp 352\u2013356","DOI":"10.1145\/3240323.3240377"},{"key":"14896_CR7","unstructured":"Chakrabarti C, Luger GF (2013) A framework for simulating and evaluating artificial chatter bot conversations. In: The 26th international Florida artificial intelligence research society conference (FLAIRS)"},{"key":"14896_CR8","doi-asserted-by":"crossref","unstructured":"Cui L et al (2017) Superagent: a customer service chatbot for e-commerce websites. In: Proceedings of ACL 2017, system demonstrations, pp 97\u2013102","DOI":"10.18653\/v1\/P17-4017"},{"key":"14896_CR9","unstructured":"Devlin J et al (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805"},{"key":"14896_CR10","unstructured":"Feine J, Morana S, Gnewuch U (2019) Measuring service encounter satisfaction with customer service chatbots using sentiment analysis. In: Proceedings of the 14th international conference on wirtschaftsinformatik (WI2019)"},{"key":"14896_CR11","doi-asserted-by":"crossref","unstructured":"Gan Z et al (2016) Learning generic sentence representations using convolutional neural networks. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 2390\u20132400","DOI":"10.18653\/v1\/D17-1254"},{"key":"14896_CR12","doi-asserted-by":"crossref","unstructured":"Ghazvininejad M et al (2018) A knowledge-grounded neural conversation model. In: Proceedings of the AAAI conference on artificial intelligence, vol 32","DOI":"10.1609\/aaai.v32i1.11977"},{"key":"14896_CR13","volume-title":"Deep Learning, 2, vol 1","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I et al (2016) Deep Learning, 2, vol 1. MIT Press, Cambridge"},{"key":"14896_CR14","doi-asserted-by":"crossref","unstructured":"Gridach M, Haddad H, Mulki H (2017) Churn identification in microblogs using convolutional neural networks with structured logical knowledge. In: Proceedings of the 3rd workshop on noisy user-generated text (WNUT), pp 21\u201330","DOI":"10.18653\/v1\/W17-4403"},{"key":"14896_CR15","unstructured":"Gu X et al (2018) Dialogwae: multimodal response generation with conditional wasserstein auto-encoder. In: International conference on learning representations (ICLR)"},{"key":"14896_CR16","unstructured":"Heller B et al (2005) Freudbot: an investigation of chatbot technology in distance education. In: EdMedia + innovate learning, association for the Advancement of Computing in Education (AACE), pp 3913\u20133918"},{"key":"14896_CR17","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.chb.2015.02.026","volume":"49","author":"J Hill","year":"2015","unstructured":"Hill J, Ford WR, Farreras IG (2015) Real conversations with artificial intelligence: A comparison between human\u2013human online conversations and human\u2013chatbot conversations. Comput Human Behav 49:245\u2013250","journal-title":"Comput Human Behav"},{"key":"14896_CR18","unstructured":"Hoang M, Bihorac OA, Rouces J (2019) Aspect-based sentiment analysis using Bert. In: Proceedings of the 22nd nordic conference on computational linguistics (NoDaLiDa), pp 187\u2013196"},{"key":"14896_CR19","doi-asserted-by":"publisher","unstructured":"Huang S-C et al (2020) Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. npj Digit Med 3. https:\/\/doi.org\/10.1038\/s41746-020-00341-z","DOI":"10.1038\/s41746-020-00341-z"},{"key":"14896_CR20","doi-asserted-by":"crossref","unstructured":"Hung V et al (2009) Towards a method for evaluating naturalness in conversational dialog systems. In: 2009 IEEE International conference on systems, man and cybernetics (SMC). IEEE, pp 1236\u20131241","DOI":"10.1109\/ICSMC.2009.5345904"},{"issue":"23","key":"14896_CR21","doi-asserted-by":"publisher","first-page":"4941","DOI":"10.3390\/rs13234941","volume":"13","author":"R Hussain","year":"2021","unstructured":"Hussain R et al (2021) Revise-net: exploiting reverse attention mechanism for salient object detection. Remote Sens 13(23):4941","journal-title":"Remote Sens"},{"key":"14896_CR22","doi-asserted-by":"crossref","unstructured":"Jenkins M-C et al (2007) Analysis of user interaction with service oriented chatbot systems. In: International conference on human-computer interaction (HCI). Springer, pp 76\u201383","DOI":"10.1007\/978-3-540-73110-8_9"},{"issue":"4","key":"14896_CR23","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/j.knosys.2008.09.001","volume":"22","author":"J Jia","year":"2009","unstructured":"Jia J (2009) CSIEC: a computer assisted English learning chatbot based on textual knowledge and reasoning. Knowl-Based Syst 22(4):249\u2013255","journal-title":"Knowl-Based Syst"},{"key":"14896_CR24","doi-asserted-by":"crossref","unstructured":"Jwalapuram P (2017) Evaluating dialogs based on grice\u2019s maxims. In: Proceedings of the student research workshop associated with recent advances in natural language processing (RANLP), pp 17\u201324","DOI":"10.26615\/issn.1314-9156.2017_003"},{"key":"14896_CR25","unstructured":"Kannan A, Vinyals O (2017) Adversarial evaluation of dialogue models. arXiv:1701.08198"},{"key":"14896_CR26","doi-asserted-by":"crossref","unstructured":"Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1746\u20131751","DOI":"10.3115\/v1\/D14-1181"},{"issue":"11","key":"14896_CR27","doi-asserted-by":"publisher","first-page":"2347","DOI":"10.3390\/app9112347","volume":"9","author":"H Kim","year":"2019","unstructured":"Kim H, Jeong Y-S (2019) Sentiment classification using convolutional neural networks. Appl Sci 9(11):2347","journal-title":"Appl Sci"},{"issue":"6","key":"14896_CR28","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","volume":"60","author":"A Krizhevsky","year":"2017","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84\u201390","journal-title":"Commun ACM"},{"key":"14896_CR29","doi-asserted-by":"crossref","unstructured":"Li J et al (2015) A diversity-promoting objective function for neural conversation models. In: 2015 Annual conference of the North American chapter of the association for computational linguistics (NAACL-HLT 2015)","DOI":"10.18653\/v1\/N16-1014"},{"key":"14896_CR30","doi-asserted-by":"crossref","unstructured":"Li J et al (2016) A persona-based neural conversation model. In: Proceedings of the 54th annual meeting of the association for computational linguistics (ACL), vol 1, pp 994\u20131003","DOI":"10.18653\/v1\/P16-1094"},{"issue":"2","key":"14896_CR31","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1007\/s41019-019-0096-6","volume":"4","author":"B Li","year":"2019","unstructured":"Li B et al (2019) Scaling word2vec on big corpus. Data Sci Eng 4 (2):157\u2013175","journal-title":"Data Sci Eng"},{"key":"14896_CR32","unstructured":"Lin C-Y (2004) Rouge: a package for automatic evaluation of summaries. In: Text summarization branches out association for computational linguistics (ACL), pp 74\u201381"},{"key":"14896_CR33","doi-asserted-by":"crossref","unstructured":"Liu C-W et al (2016) How not to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 conference on empirical methods in natural language processing (EMNLP)","DOI":"10.18653\/v1\/D16-1230"},{"key":"14896_CR34","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1007\/s00779-018-1183-9","volume":"23","author":"L Luo","year":"2019","unstructured":"Luo L (2019) Network text sentiment analysis method combining LDA text representation and GRU-CNN. Personal and Ubiquit Comput 23:405\u2013412","journal-title":"Personal and Ubiquit Comput"},{"key":"14896_CR35","doi-asserted-by":"crossref","unstructured":"Miaschi A, Dell\u2019Orletta F (2020) Contextual and non-contextual word embeddings: an in-depth linguistic investigation. In: Proceedings of the 5th workshop on representation learning for NLP (RepL4NLP), pp 110\u2013119","DOI":"10.18653\/v1\/2020.repl4nlp-1.15"},{"key":"14896_CR36","unstructured":"Mikolov T et al (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems (ANPIS), pp 3111\u20133119"},{"issue":"6","key":"14896_CR37","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1016\/j.jretconser.2012.07.006","volume":"19","author":"MSB Mimoun","year":"2012","unstructured":"Mimoun MSB, Poncin I, Garnier M (2012) Case study\u2014embodied virtual agents: an analysis on reasons for failure. J Retail Consum Serv 19(6):605\u2013612","journal-title":"J Retail Consum Serv"},{"key":"14896_CR38","unstructured":"Minaee S, Azimi E, Abdolrashidi A (2019) Deep-sentiment: sentiment analysis using ensemble of cnn and bi-lstm models. arXiv:1904.04206"},{"key":"14896_CR39","doi-asserted-by":"crossref","unstructured":"Nuruzzaman M, Hussain OK (2018) A survey on chatbot implementation in customer service industry through deep neural networks. In: 2018 IEEE 15th International conference on e-business engineering (ICEBE). IEEE, pp 54\u201361","DOI":"10.1109\/ICEBE.2018.00019"},{"key":"14896_CR40","doi-asserted-by":"publisher","DOI":"10.4324\/9781315700892","volume-title":"Satisfaction: A Behavioral Perspective on the Consumer: A Behavioral Perspective on the Consumer","author":"RL Oliver","year":"2014","unstructured":"Oliver RL (2014) Satisfaction: A Behavioral Perspective on the Consumer: A Behavioral Perspective on the Consumer. Routledge, Evanston"},{"key":"14896_CR41","doi-asserted-by":"crossref","unstructured":"Papineni K et al (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for computational linguistics (ACL), pp 311\u2013318","DOI":"10.3115\/1073083.1073135"},{"key":"14896_CR42","doi-asserted-by":"crossref","unstructured":"Park Y, Gates SC (2009) Towards real-time measurement of customer satisfaction using automatically generated call transcripts. In: Proceedings of the 18th ACM conference on information and knowledge management (CIKM), pp 1387\u20131396","DOI":"10.1145\/1645953.1646128"},{"key":"14896_CR43","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532\u20131543","DOI":"10.3115\/v1\/D14-1162"},{"key":"14896_CR44","unstructured":"Peras D (2018) Chatbot evaluation metrics. Economic and Social Development: Book of Proceedings:89\u201397"},{"key":"14896_CR45","doi-asserted-by":"crossref","unstructured":"Pereira J, D\u0131az O (2018) A quality analysis of facebook messenger\u2019s most popular chatbots. In: Proceedings of the 33rd annual ACM symposium on applied computing (SAC), pp 2144\u20132150","DOI":"10.1145\/3167132.3167362"},{"key":"14896_CR46","doi-asserted-by":"publisher","unstructured":"Perez-Toro PA (2020) PauPerezT\/WEBERT: word embeddings using BERT, version V0.0.1, Jul. 2020. https:\/\/doi.org\/10.5281\/zenodo.3964244","DOI":"10.5281\/zenodo.3964244"},{"key":"14896_CR47","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Toro PA et al (2021) Acoustic and linguistic analyses to assess early-onset and genetic Alzheimer\u2019s disease. IEEE International conference on acoustics, speech and signal processing, in press","DOI":"10.1109\/ICASSP39728.2021.9414009"},{"key":"14896_CR48","doi-asserted-by":"crossref","unstructured":"Peters ME et al (2018) Deep contextualized word representations. arXiv:1802.05365","DOI":"10.18653\/v1\/N18-1202"},{"key":"14896_CR49","doi-asserted-by":"crossref","unstructured":"Ranoliya BR, Raghuwanshi N, Singh S (2017) Chatbot for university related faqs. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI). IEEE, pp 1525\u20131530","DOI":"10.1109\/ICACCI.2017.8126057"},{"key":"14896_CR50","unstructured":"Reese S et al (2010) Word-sense disambiguated multilingual Wikipedia corpus. In: 7th International conference on language resources and evaluation (LREC)"},{"issue":"3","key":"14896_CR51","doi-asserted-by":"publisher","first-page":"e7004","DOI":"10.2196\/publichealth.7004","volume":"3","author":"M Roccetti","year":"2017","unstructured":"Roccetti M et al (2017) Attitudes of Crohn\u2019s disease patients: infodemiology case study and sentiment analysis of Facebook and Twitter posts. JMIR Public Health Surveill 3(3):e7004","journal-title":"JMIR Public Health Surveill"},{"key":"14896_CR52","doi-asserted-by":"crossref","unstructured":"Rodrigues Makiuchi M et al (2019) Multimodal fusion of BERT-CNN and gated CNN representations for depression detection. In: Proceedings of the 9th international on audio\/visual emotion challenge and workshop (AVEC), pp 55\u201363","DOI":"10.1145\/3347320.3357694"},{"key":"14896_CR53","unstructured":"Rong X (2014) Word2vec parameter learning explained. arXiv:1411.2738"},{"issue":"4","key":"14896_CR54","doi-asserted-by":"publisher","first-page":"2236","DOI":"10.1016\/j.dss.2006.07.001","volume":"42","author":"RP Schumaker","year":"2007","unstructured":"Schumaker RP et al (2007) An evaluation of the chat and knowledge delivery components of a low-level dialog system: The az-alice experiment. Decision Support Syst 42(4):2236\u20132246","journal-title":"Decision Support Syst"},{"key":"14896_CR55","unstructured":"Sedoc J et al (2019) Chateval: a tool for chatbot evaluation. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (NAACL), pp 60\u201365"},{"key":"14896_CR56","doi-asserted-by":"crossref","unstructured":"Shang L, Lu Z, Li H (2015) Neural responding machine for short-text conversation. arXiv:1503.02364","DOI":"10.3115\/v1\/P15-1152"},{"issue":"3-4","key":"14896_CR57","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1057\/jt.2010.17","volume":"18","author":"MH Siddiqui","year":"2010","unstructured":"Siddiqui MH, Sharma TG (2010) Analyzing customer satisfaction with service quality in life insurance services. J Target Meas Anal Market 18 (3-4):221\u2013238","journal-title":"J Target Meas Anal Market"},{"key":"14896_CR58","unstructured":"Song Y et al (2016) Two are better than one: an ensemble of retrieval-and generation-based dialog systems. arXiv:1610.07149"},{"key":"14896_CR59","doi-asserted-by":"crossref","unstructured":"Sordoni A et al (2015) A neural network approach to context-sensitive generation of conversational responses. arXiv:506.06714","DOI":"10.3115\/v1\/N15-1020"},{"key":"14896_CR60","doi-asserted-by":"publisher","first-page":"115461","DOI":"10.1016\/j.eswa.2021.115461","volume":"184","author":"SM Suhaili","year":"2021","unstructured":"Suhaili SM, Salim N, Jambli MN (2021) Service chatbots: a systematic review. Expert Syst Appl 184:115461","journal-title":"Expert Syst Appl"},{"key":"14896_CR61","unstructured":"Tang S et al (2018) Exploring asymmetric encoder-decoder structure for context-based sentence representation learning"},{"key":"14896_CR62","unstructured":"Tiedemann J (2012) Parallel data, tools and interfaces in opus. In: Lrec, vol 2012, pp 2214\u20132218"},{"key":"14896_CR63","unstructured":"Vaswani A et al (2017) Attention is all you need. In: Advances in neural information processing systems (NIPS), pp 5998\u20136008"},{"key":"14896_CR64","unstructured":"Venkatesh A et al (2018) On evaluating and comparing conversational agents. arXiv:1801.03625 4:60\u201368"},{"key":"14896_CR65","doi-asserted-by":"crossref","unstructured":"Wang G et al (2018) Joint embedding of words and labels for text classification. arXiv:1805.04174","DOI":"10.18653\/v1\/P18-1216"},{"key":"14896_CR66","doi-asserted-by":"crossref","unstructured":"Xiang Y et al (2014) Problematic situation analysis and automatic recognition for chinese online conversational system. In: Proceedings of the 3rd CIPS-SIGHAN joint conference on chinese language processing, pp 43\u201351","DOI":"10.3115\/v1\/W14-6808"},{"key":"14896_CR67","doi-asserted-by":"crossref","unstructured":"Xu Z et al (2017) Neural response generation via gan with an approximate embedding layer. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 617\u2013626","DOI":"10.18653\/v1\/D17-1065"},{"key":"14896_CR68","doi-asserted-by":"crossref","unstructured":"Yan J (2009) Text representation","DOI":"10.1007\/978-0-387-39940-9_420"},{"key":"14896_CR69","unstructured":"Yao K et al (2016) An attentional neural conversation model with improved specificity. In: Proceedings of the 53th annual meeting of the association for computational linguistics (ACL)"},{"key":"14896_CR70","doi-asserted-by":"crossref","unstructured":"Zhao R, Romero OJ, Rudnicky A (2018) SOGO: a social intelligent negotiation dialogue system. In: Proceedings of the 18th international conference on intelligent virtual agents (IVA), pp 239\u2013246","DOI":"10.1145\/3267851.3267880"},{"key":"14896_CR71","doi-asserted-by":"crossref","unstructured":"Zhong J, Li W (2019) Predicting customer churn in the telecommunication industry by analyzing phone call transcripts with convolutional neural networks. In: Proceedings of the 2019 3rd International Conference on Innovation in Artificial Intelligence (ICAI), pp 55\u201359","DOI":"10.1145\/3319921.3319937"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14896-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-023-14896-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14896-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,6]],"date-time":"2024-01-06T05:50:08Z","timestamp":1704520208000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-023-14896-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,3]]},"references-count":71,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["14896"],"URL":"https:\/\/doi.org\/10.1007\/s11042-023-14896-y","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,3]]},"assertion":[{"value":"15 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 December 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 February 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 June 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}]}}