{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T03:59:26Z","timestamp":1763092766132,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,12,13]],"date-time":"2021-12-13T00:00:00Z","timestamp":1639353600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Currently, most chatbots are unable to detect the emotional state of the interlocutor and respond according to the interlocutor\u2019s emotional state. Over the last few years, there has been growing interest in empathic chatbots. In other disciplines aside from artificial intelligence, e.g., in medicine, there is growing interest in the study and simulation of human emotions. However, there is a fundamental issue that is not commonly addressed, and it is the design of protocols for quantitatively evaluating an empathic chatbot by utilizing the analysis of the conversation between the bot and an interlocutor. This study is motivated by the aforementioned scenarios and by the lack of methods for assessing the performance of an empathic bot; thus, a chatbot with the ability to recognize the emotions of its interlocutor is needed. The main novelty of this study is the protocol with which it is possible to analyze the conversations between a chatbot and an interlocutor, regardless of whether the latter is a person or another chatbot. For this purpose, we have designed a minimally viable prototype of an empathic chatbot, named LENNA, for evaluating the usefulness of the proposed protocol. The proposed approach uses Shannon entropy to measure the changes in the emotional state experienced by the chatbot during a conversation, applying sentiment analysis techniques to the analysis of the conversation. Once the simulation experiments were performed, the conversations were analyzed by applying multivariate statistical methods and Fourier analysis. We show the usefulness of the proposed methodology for evaluating the emotional state of LENNA during conversations, which could be useful in the evaluation of other empathic chatbots.<\/jats:p>","DOI":"10.3390\/computers10120170","type":"journal-article","created":{"date-parts":[[2021,12,14]],"date-time":"2021-12-14T01:20:41Z","timestamp":1639444841000},"page":"170","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["LENNA (Learning Emotions Neural Network Assisted): An Empathic Chatbot Designed to Study the Simulation of Emotions in a Bot and Their Analysis in a Conversation"],"prefix":"10.3390","volume":"10","author":[{"given":"Rafael","family":"Lahoz-Beltra","sequence":"first","affiliation":[{"name":"Department of Biodiversity, Ecology and Evolution (Biomathematics), Faculty of Biological Sciences, Complutense University of Madrid, 28040 Madrid, Spain"},{"name":"Modeling, Data Analysis and Computational Tools for Biology Research Group, Complutense University of Madrid, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claudia Corona","family":"L\u00f3pez","sequence":"additional","affiliation":[{"name":"Department of Biodiversity, Ecology and Evolution (Biomathematics), Faculty of Biological Sciences, Complutense University of Madrid, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA\u2014A computer program for the study of natural language communication between man and machine","volume":"9","author":"Weizenbaum","year":"1966","journal-title":"Commun. ACM"},{"unstructured":"Koch, S. (2010). A theory of therapy, personality and interpersonal relationships as developed in the client-centered framework. Psychology: A Study of a Science. Volume 3: Formulations of the Person and the Social Context, McGraw Hill.","key":"ref_2"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/0004-3702(72)90049-5","article-title":"Turing-like indistinguishability tests for the validation of a computer simulation of paranoid processes","volume":"3","author":"Colby","year":"1972","journal-title":"Artif. Intell."},{"unstructured":"Pamungkas, E.W. (2019). Emotionally-aware chatbots: A survey. arXiv.","key":"ref_4"},{"doi-asserted-by":"crossref","unstructured":"Darwin, C. (1872). The Expression of the Emotions in Man and Animals, John Murray.","key":"ref_5","DOI":"10.1037\/10001-000"},{"doi-asserted-by":"crossref","unstructured":"Klusch, M., and Kerschberg, L. (2000). Affective computing for future agents. Cooperative Information Agents IV\u2014The Future of Information Agents in Cyberspace. CIA 2000, Springer. Lecture Notes in Computer Science.","key":"ref_6","DOI":"10.1007\/b75220"},{"doi-asserted-by":"crossref","unstructured":"Ghandeharioun, A., McDuff, D., Czerwinski, M., and Rowan, K. (2019, January 3\u20136). EMMA: An emotion-aware wellbeing chatbot. Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, UK.","key":"ref_7","DOI":"10.1109\/ACII.2019.8925455"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1017\/S0954579405050340","article-title":"The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology","volume":"17","author":"Posner","year":"2005","journal-title":"Dev. Psychopathol."},{"key":"ref_9","first-page":"1483","article-title":"An e-commerce website based chatbot","volume":"6","author":"Gupta","year":"2015","journal-title":"Int. J. Comput. Sci. Inf. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1093\/joc\/jqy026","article-title":"Psychological, relational, and emotional effects of self-disclorsure after conversations with a chatbot","volume":"68","author":"Ho","year":"2018","journal-title":"J. Commun."},{"key":"ref_11","first-page":"27","article-title":"How to create a psychologist-chatbot","volume":"41","author":"Romero","year":"2020","journal-title":"Psychol. Pap."},{"key":"ref_12","first-page":"1164","article-title":"Towards emotion-sensitive conversational user interfaces in healthcare applications","volume":"264","author":"Denecke","year":"2019","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1007\/7854_2012_211","article-title":"Inflammation and depression","volume":"14","author":"Zunszain","year":"2013","journal-title":"Curr. Top. Behav. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1186\/s12199-018-0740-1","article-title":"Inflammation and cancer","volume":"23","author":"Murata","year":"2018","journal-title":"Environ. Health Prev. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"240128","DOI":"10.1155\/2013\/240128","article-title":"The clinical obesity maintenance model: An integration of psychological constructs including mood, emotional regulation, disordered overeating, habitual cluster behaviours, health literacy and cognitive function","volume":"2013","author":"Raman","year":"2013","journal-title":"J. Obes."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104261","DOI":"10.1016\/j.biosystems.2020.104261","article-title":"Modeling a cancerous tumor development in a virtual patient suffering from a depressed state of mind: Simulation of somatic evolution with a customized genetic algorithm","volume":"198","author":"Rodriguez","year":"2020","journal-title":"Biosystems"},{"unstructured":"Spring, T., Casas, J., Daher, K., Mugellini, E., and Abou Khaled, O. (2019, January 18\u201319). Empathic response generation in chatbots. Proceedings of the 4th Swiss Text Anlytics Conference (SwissText 2019), Winterthur, Switzerland.","key":"ref_17"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000011","article-title":"Opinion mining and sentiment analysis","volume":"2","author":"Pang","year":"2008","journal-title":"Found. Trends Inf. Retr."},{"doi-asserted-by":"crossref","unstructured":"Wilson, T., Wiebe, J., and Hoffmann, P. (2005, January 6\u20138). Recognizing contextual polarity in phrase-level sentiment analysis. Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT \u201805), Vancouver, BC, Canada.","key":"ref_19","DOI":"10.3115\/1220575.1220619"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1615","DOI":"10.1002\/1097-0142(20010415)91:8+<1615::AID-CNCR1175>3.0.CO;2-L","article-title":"Artificial neural networks: Opening the black box","volume":"91","author":"Dayhoff","year":"2001","journal-title":"Cancer"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1301","DOI":"10.1109\/JSTSP.2017.2764438","article-title":"End-to-end multimodal emotion recognition using deep neural networks","volume":"11","author":"Tzirakis","year":"2017","journal-title":"IEEE J. Sel. Top. Signal. Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1631\/jzus.C1000044","article-title":"Recent advances in the artificial endocrine system","volume":"12","author":"Xu","year":"2011","journal-title":"J. Zhejiang Univ.-Sci. Comput. Electron."},{"unstructured":"Strout, J., and Epler, J. (2021, October 08). Eliza.py, ELIZA in Python. Available online: https:\/\/github.com\/jezhiggins\/eliza.py.","key":"ref_23"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1016\/0013-4694(52)90073-4","article-title":"Some psychiatric implications of physiological studies on frontotemporal portion of limbic system (visceral brain)","volume":"4","author":"MacLean","year":"1952","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1126\/science.164.3875.86","article-title":"Pancultural elements in facial displays of emotions","volume":"164","author":"Ekman","year":"1969","journal-title":"Science"},{"unstructured":"Harold, L.M. (2016). Ekman\u2019s theory of basic emotions. The Sage Encyclopedia of Theory in Psychology, Sage Publications.","key":"ref_26"},{"doi-asserted-by":"crossref","unstructured":"Yu, L.-C., Lee, L.-H., Hao, S., Wang, J., He, Y., Hu, J., Lai, K.R., and Zhang, X. (2016, January 12\u201317). Building Chinese affective resources in valence-arousal dimensions. Proceedings of the NAACL-HLT 2016, San Diego, CA, USA.","key":"ref_27","DOI":"10.18653\/v1\/N16-1066"},{"unstructured":"Lahoz-Beltra, R. (2004). Bioinform\u00e1tica: Simulaci\u00f3n, Vida Artificial e Inteligencia Artificial, Ediciones D\u00edaz de Santos.","key":"ref_28"},{"unstructured":"Loria, S. (2021, October 08). TextBlob Documentation. Available online: https:\/\/textblob.readthedocs.io\/en\/dev\/.","key":"ref_29"},{"unstructured":"BOT LIBRE (2021, October 08). The Open Source Chatbot and Artificial Intelligence Platform. Available online: https:\/\/www.botlibre.com\/.","key":"ref_30"},{"unstructured":"Kozlowski, L. (2021, October 08). Shannon Entropy Calculator. Available online: https:\/\/www.shannonentropy.netmark.pl\/.","key":"ref_31"},{"unstructured":"Jockers, M. (2021, October 08). Syuzhet. Available online: https:\/\/cran.r-project.org\/web\/packages\/syuzhet\/syuzhet.pdf.","key":"ref_32"},{"unstructured":"Mohammad, S.M., and Turney, P.D. (2021, October 08). NRC Emotion Lexicon. Available online: https:\/\/saifmohammad.com\/WebPages\/AccessResource.htm.","key":"ref_33"},{"unstructured":"Jockers, M.L. (2021, October 08). Revealing Sentiment and Plot Arcs with the Syuzhet Package. Available online: https:\/\/www.matthewjockers.net\/2015\/02\/02\/syuzhet\/.","key":"ref_34"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1063\/1.166191","article-title":"Entropy estimation of symbol sequences","volume":"6","author":"Grassberger","year":"1996","journal-title":"Chaos"},{"doi-asserted-by":"crossref","unstructured":"Helmi Setyawan, M.Y., Awangga, R.M., and Efendi, S. (2018, January 3\u20134). Comparison of multinomial naive Bayes algorithm and logistic regression for intent classification in chatbot. Proceedings of the 2018 International Conference on Applied Engineering (ICAE), Batam, Indonesia.","key":"ref_36","DOI":"10.1109\/INCAE.2018.8579372"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"40","DOI":"10.36548\/jismac.2021.1.004","article-title":"Na\u00efve Bayes and entropy based analysis and classification of humans and chat bots","volume":"3","author":"Smys","year":"2021","journal-title":"J. ISMAC"},{"key":"ref_38","first-page":"354","article-title":"Chatbot disease prediction and treatment recommendation using machine learning","volume":"27","author":"Anapuma","year":"2021","journal-title":"High Technol. Lett."},{"doi-asserted-by":"crossref","unstructured":"Zygad\u0142o, A., Koz\u0142owski, M., and Janicki, A. (2021). Text-Based emotion recognition in English and Polish for therapeutic chatbot. Appl. Sci., 11.","key":"ref_39","DOI":"10.3390\/app112110146"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"33","DOI":"10.3991\/ijet.v13i02.7173","article-title":"Classification technique of interviewer-bot result using na\u00efve Bayes and phrase reinforcement algorithms","volume":"13","author":"Sarosa","year":"2018","journal-title":"Int. J. Emerg. Technol. Learn. IJET"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"012059","DOI":"10.1088\/1742-6596\/1716\/1\/012059","article-title":"Building a medical chatbot using support vector machine learning algorithm","volume":"1716","author":"Tamizharasi","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1016\/S0196-9781(01)00399-0","article-title":"Corticotropin-releasing hormone in depression and post-traumatic stress disorder","volume":"22","author":"Kasckow","year":"2001","journal-title":"Peptides"},{"doi-asserted-by":"crossref","unstructured":"Thenius, R., Zahadat, P., and Schmickl, T. (2013, January 2\u20136). EMANN\u2014A model of emotions in an artificial neural network. Proceedings of the ECAL 2013: The Twelfth European Conference on Artificial Life, Sicily, Italy.","key":"ref_43","DOI":"10.1162\/978-0-262-31709-2-ch122"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1973","DOI":"10.1109\/TNNLS.2013.2271094","article-title":"Artificial endocrine controller for power management in robotic systems","volume":"24","author":"Neal","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1016\/j.ijmst.2020.05.020","article-title":"A novel artificial intelligent model for predicting air overpressure using brain inspired emotional neural network","volume":"30","author":"Temeng","year":"2020","journal-title":"Int. J. Min. Sci. Technol."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1109\/TAFFC.2016.2518162","article-title":"Microexpression identification and categorization using a facial dynamics map","volume":"8","author":"Xu","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"doi-asserted-by":"crossref","unstructured":"Yan, F., Iliyasu, A.M., Jiao, S., and Yang, H. (2019). Quantum Structure for Modelling Emotion Space of Robots. Appl. Sci., 9.","key":"ref_47","DOI":"10.3390\/app9163351"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1016\/j.tics.2004.10.004","article-title":"Emotions: From brain to robot","volume":"8","author":"Arbib","year":"2004","journal-title":"TREND Cogn. Sci."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"6","DOI":"10.5772\/45662","article-title":"A multidisciplinary artificial intelligence model of an affective robot","volume":"9","author":"Samani","year":"2012","journal-title":"Int. J. Adv. Robot. Syst."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/12\/170\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:46:38Z","timestamp":1760168798000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/12\/170"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,13]]},"references-count":49,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["computers10120170"],"URL":"https:\/\/doi.org\/10.3390\/computers10120170","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2021,12,13]]}}}