{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:50:33Z","timestamp":1760230233656,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,7,17]],"date-time":"2022-07-17T00:00:00Z","timestamp":1658016000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>The evolution of computers in recent years has given a strong boost to research techniques aimed at improving human\u2013machine interaction. These techniques tend to simulate the dynamics of the human\u2013human interaction process, which is based on our innate ability to understand the emotions of other humans. In this work, we present the design of a classifier to recognize the emotions expressed by human beings, and we discuss the results of its testing in a culture-specific case study. The classifier relies exclusively on the gestures people perform, without the need to access additional information, such as facial expressions, the tone of a voice, or the words spoken. The specific purpose is to test whether a computer can correctly recognize emotions starting only from gestures. More generally, it is intended to allow interactive systems to be able to automatically change their behaviour based on the recognized mood, such as adapting the information contents proposed or the flow of interaction, in analogy to what normally happens in the interaction between humans. The document first introduces the operating context, giving an overview of the recognition of emotions and the approach used. Subsequently, the relevant bibliography is described and analysed, highlighting the strengths of the proposed solution. The document continues with a description of the design and implementation of the classifier and of the study we carried out to validate it. The paper ends with a discussion of the results and a short overview of possible implications.<\/jats:p>","DOI":"10.3390\/mti6070056","type":"journal-article","created":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T01:53:22Z","timestamp":1658109202000},"page":"56","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Detecting Emotions from Illustrator Gestures\u2014The Italian Case"],"prefix":"10.3390","volume":"6","author":[{"given":"Daniele","family":"Fundar\u00f2","sequence":"first","affiliation":[{"name":"synbrAIn srl, 90128 Palermo, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vito","family":"Gentile","sequence":"additional","affiliation":[{"name":"synbrAIn srl, 90128 Palermo, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabrizio","family":"Milazzo","sequence":"additional","affiliation":[{"name":"Dipartimento di Ingegneria, Universit\u00e0 degli Studi di Palermo, 90128 Palermo, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1976-031X","authenticated-orcid":false,"given":"Salvatore","family":"Sorce","sequence":"additional","affiliation":[{"name":"Facolt\u00e0 di Ingegneria e Architettura, Universit\u00e0 degli Studi di Enna \u201cKore\u201d, 94100 Enna, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1016\/j.neunet.2005.03.006","article-title":"Emotion recognition in human-computer interaction","volume":"18","author":"Fragopanagos","year":"2005","journal-title":"Neural Netw."},{"key":"ref_2","unstructured":"Daras, P., Achilleopoulos, N., Panebarco, M., Mayora, O., Stollenmayer, P., Williams, D., Pennick, T., Magnenat-Thalmann, N., Guerrero, C., and Pelt, M. (2008, January 16\u201319). User Centric Media of the Future Internet. Proceedings of the 2008 The Second International Conference on Next Generation Mobile Applications, Services, and Technologies, Cardiff, UK."},{"key":"ref_3","unstructured":"Andreou, P., Georgiadis, D., Pamboris, A., Christophorou, C., and Samaras, G. (2015, January 11\u201313). Towards a backend framework for supporting affective avatar-based interaction systems. Proceedings of the European Conference on Ambient Intelligence 2015, Athens, Greece."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1016\/j.neuroimage.2005.08.009","article-title":"Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection","volume":"28","author":"Davatzikos","year":"2005","journal-title":"Neuroimage"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Picard, R.W. (1997). Affective Computing, The MIT Press.","DOI":"10.7551\/mitpress\/1140.001.0001"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/T-AFFC.2012.16","article-title":"Affective Body Expression Perception and Recognition: A Survey","volume":"4","author":"Kleinsmith","year":"2013","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1037\/a0025737","article-title":"Emotion expression in body action and posture","volume":"12","author":"Dael","year":"2012","journal-title":"Emotion"},{"key":"ref_9","unstructured":"Patwardhan, A., and Knapp, G. (2016). Multimodal affect recognition using kinect. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Peter, C., and Beale, R. (2008). Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech. Affect and Emotion in Human-Computer Interaction, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-540-85099-1"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1334","DOI":"10.1016\/j.jnca.2006.09.007","article-title":"Bimodal emotion recognition from expressive face and body gestures","volume":"30","author":"Gunes","year":"2007","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kipp, M., and Martin, J. (2009, January 10\u201312). Gesture and emotion: Can basic gestural form features discriminate emotions?. Proceedings of the 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, The Netherlands.","DOI":"10.1109\/ACII.2009.5349544"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/T-AFFC.2011.7","article-title":"Toward a Minimal Representation of Affective Gestures","volume":"2","author":"Glowinski","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1002\/cav.1714","article-title":"Continuous body emotion recognition system during theater performances","volume":"27","author":"Senecal","year":"2016","journal-title":"Comp. Anim. Virtual Worlds"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1038\/nrn1872","article-title":"Towards the neurobiology of emotional body language","volume":"7","year":"2006","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Larrazabal, J.M., and Miranda, L.A.P. (2004). Emotional and Conversational Nonverbal Signals. Language, Knowledge, and Representation, Springer.","DOI":"10.1007\/978-1-4020-2783-3"},{"key":"ref_17","unstructured":"Krishnan, H., and Raj, U. (2022, July 15). Cross Cultural Communication. Available online: https:\/\/www.slideshare.net\/harikrishnann2\/crosscultural-communication-42061337."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Escalera, S., Gonz\u00e0lez, J., Bar\u00f3, X., Reyes, M., Lopes, O., Guyon, I., Athitsos, V., and Escalante, H. (2013, January 9\u201313). Multi-modal gesture recognition challenge 2013: Dataset and results. Proceedings of the 15th ACM on International Conference on Multimodal Interaction, (ICMI \u201813), Sydney, Australia.","DOI":"10.1145\/2522848.2532595"},{"key":"ref_19","unstructured":"Poyatos, F. (1988). How gestures can become like words. Cross-Cultural Perspectives in Nonverbal Communication, Hogrefe & Huber Publishers."},{"key":"ref_20","unstructured":"Karam, M., and Schraefel, M.C. (2022, July 15). A Taxonomy of Gestures in Human Computer Interactions. Available online: http:\/\/eprints.soton.ac.uk\/id\/eprint\/261149."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1037\/h0077714","article-title":"A circumplex model of affect","volume":"39","author":"Russell","year":"1980","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Dalgleish, T., and Power, M.J. (1999). Basic Emotions. Handbook of Cognition and Emotion, John Wiley & Sons.","DOI":"10.1002\/0470013494"},{"key":"ref_23","unstructured":"Parrott, W.G. (2001). Emotions in Social Psychology: Essential Readings, Psychology Press."},{"key":"ref_24","unstructured":"Cornelius, R.R. (2000, January 5\u20137). Theoretical approaches to emotion. Proceedings of the ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion, Newcastle, UK."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1037\/1528-3542.2.1.75","article-title":"Cross-cultural patterns in emotion recognition: Highlighting design and analytical techniques","volume":"2","author":"Elfenbein","year":"2002","journal-title":"Emotion"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1037\/h0027576","article-title":"Encoding of attitude by a seated communicator via posture and position cues","volume":"33","author":"Mehrabian","year":"1969","journal-title":"J. Consult. Clin. Psychol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1037\/0033-2909.128.2.203","article-title":"On the universality and cultural specificity of emotion recognition: A meta-analysis","volume":"128","author":"Elfenbein","year":"2002","journal-title":"Psychol Bull."},{"key":"ref_28","first-page":"18486","article-title":"Recognizing emotional facial expressions: Does perceived sociolinguistic group make a difference?","volume":"31","author":"Hess","year":"1996","journal-title":"Int. J. Psychol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1111\/cgf.12598","article-title":"Emotion Analysis and Classification: Understanding the Performers\u2019 Emotions Using the LMA Entities","volume":"34","author":"Aristidou","year":"2015","journal-title":"Comput. Graph. Forum"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Fourati, N., and Pelachaud, C. (2015, January 21\u201324). Relevant body cues for the classification of emotional body expression in daily actions. Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), Xi\u2019an, China.","DOI":"10.1109\/ACII.2015.7344582"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Carreno-Medrano, P., Gibet, S., and Marteau, P. (2015, January 21\u201324). End-effectors trajectories: An efficient low-dimensional characterization of affective-expressive body motions. Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), Xi\u2019an, China.","DOI":"10.1109\/ACII.2015.7344607"},{"key":"ref_32","unstructured":"Cutler, R., and Turk, M. (1998, January 14\u201316). View-based interpretation of real-time optical flow for gesture recognition. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1145\/2398356.2398381","article-title":"Real-time human pose recognition in parts from single depth images","volume":"56","author":"Shotton","year":"2013","journal-title":"Commun. ACM"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Gentile, V., Sorce, S., and Gentile, A. (2014, January 2\u20134). Continuous Hand Openness Detection Using a Kinect-Like Device. Proceedings of the 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems, Birmingham, UK.","DOI":"10.1109\/CISIS.2014.80"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1109\/TMM.2013.2246148","article-title":"Robust Part-Based Hand Gesture Recognition Using Kinect Sensor","volume":"15","author":"Ren","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Song, Y., Gu, Y., Wang, P., Liu, Y., and Li, A. (2013, January 16\u201318). A Kinect based gesture recognition algorithm using GMM and HMM. Proceedings of the 2013 6th International Conference on Biomedical Engineering and Informatics, Hangzhou, China.","DOI":"10.1109\/BMEI.2013.6747040"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Alvarez, L., Mejail, M., Gomez, L., and Jacobo, J. (2012, January 3\u20136). A Performance Evaluation of HMM and DTW for Gesture Recognition. Proceedings of the CIARP 2012: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Buenos Aires, Argentina. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-642-33275-3"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yang, M.H., and Ahuja, N. (2001). Recognizing Hand Gestures Using Motion Trajectories. Face Detection and Gesture Recognition for Human-Computer Interaction, Springer.","DOI":"10.1007\/978-1-4615-1423-7_3"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Coan, J.A., and Allen, J.J.B. (2007). Social psychological methods of emotion elicitation. Handbook of Emotion Elicitation and Assessment, Oxford University Press.","DOI":"10.1093\/oso\/9780195169157.001.0001"},{"key":"ref_40","unstructured":"Heaton, J. (2008). Introduction to Neural Networks for Java, Heaton Research, Inc.. [2nd]."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/7\/56\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:52:24Z","timestamp":1760140344000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/7\/56"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,17]]},"references-count":40,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["mti6070056"],"URL":"https:\/\/doi.org\/10.3390\/mti6070056","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2022,7,17]]}}}