{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T02:45:53Z","timestamp":1785552353226,"version":"3.56.0"},"reference-count":215,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T00:00:00Z","timestamp":1677110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In recent years, the rapid development of sensors and information technology has made it possible for machines to recognize and analyze human emotions. Emotion recognition is an important research direction in various fields. Human emotions have many manifestations. Therefore, emotion recognition can be realized by analyzing facial expressions, speech, behavior, or physiological signals. These signals are collected by different sensors. Correct recognition of human emotions can promote the development of affective computing. Most existing emotion recognition surveys only focus on a single sensor. Therefore, it is more important to compare different sensors or unimodality and multimodality. In this survey, we collect and review more than 200 papers on emotion recognition by literature research methods. We categorize these papers according to different innovations. These articles mainly focus on the methods and datasets used for emotion recognition with different sensors. This survey also provides application examples and developments in emotion recognition. Furthermore, this survey compares the advantages and disadvantages of different sensors for emotion recognition. The proposed survey can help researchers gain a better understanding of existing emotion recognition systems, thus facilitating the selection of suitable sensors, algorithms, and datasets.<\/jats:p>","DOI":"10.3390\/s23052455","type":"journal-article","created":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T02:28:57Z","timestamp":1677119337000},"page":"2455","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":119,"title":["Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0775-5735","authenticated-orcid":false,"given":"Yujian","family":"Cai","sequence":"first","affiliation":[{"name":"School of Electronic Information Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingguang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronic Information Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinsong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronic Information Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Picard, R.W. (2000). Affective Computing, MIT Press.","DOI":"10.7551\/mitpress\/1140.001.0001"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107280","DOI":"10.1016\/j.compeleceng.2021.107280","article-title":"A Human-Computer Interaction Framework for Emotion Recognition through Time-Series Thermal Video Sequences","volume":"93","author":"Nayak","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1111\/medu.13760","article-title":"Emotion Recognition in Medical Students: Effects of Facial Appearance and Care Schema Activation","volume":"53","author":"Colonnello","year":"2019","journal-title":"Med. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Feng, X., Wei, Y.J., Pan, X.L., Qiu, L.H., and Ma, Y.M. (2020). Academic Emotion Classification and Recognition Method for Large-Scale Online Learning Environment-Based on a-Cnn and Lstm-Att Deep Learning Pipeline Method. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17061941"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1186\/s13634-021-00789-5","article-title":"Personnel Emotion Recognition Model for Internet of Vehicles Security Monitoring in Community Public Space","volume":"2021","author":"Fu","year":"2021","journal-title":"Eurasip J. Adv. Signal Process."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Oh, G., Ryu, J., Jeong, E., Yang, J.H., Hwang, S., Lee, S., and Lim, S. (2021). DRER: Deep Learning-Based Driver\u2019s Real Emotion Recognizer. Sensors, 21.","DOI":"10.3390\/s21062166"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MMUL.2020.3025161","article-title":"Toward Sensing Emotions With Deep Visual Analysis: A Long-Term Psychological Modeling Approach","volume":"27","author":"Sun","year":"2020","journal-title":"IEEE Multimed."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mandryk, R.L., Atkins, M.S., and Inkpen, K.M. (2006, January 22\u201327). A continuous and objective evaluation of emotional experience with interactive play environments. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montr\u00e9al, QC, Canada.","DOI":"10.1145\/1124772.1124926"},{"key":"ref_9","unstructured":"Ogata, T., and Sugano, S. (1999, January 10\u201315). Emotional communication between humans and the autonomous robot which has the emotion model. Proceedings of the 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), Detroit, MI, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1016\/S1474-6670(17)32079-7","article-title":"A new architecture for autonomous robots based on emotions","volume":"37","author":"Malfaz","year":"2004","journal-title":"IFAC Proc. Vol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rattanyu, K., Ohkura, M., and Mizukawa, M. (2010, January 27\u201330). Emotion monitoring from physiological signals for service robots in the living space. Proceedings of the ICCAS 2010, Gyeonggi-do, Republic of Korea.","DOI":"10.1109\/ICCAS.2010.5669914"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hasnul, M.A., Aziz, N.A.A., Alelyani, S., Mohana, M., and Aziz, A.A.J.S. (2021). Electrocardiogram-based emotion recognition systems and their applications in healthcare\u2014A review. Sensors, 21.","DOI":"10.3390\/s21155015"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Feidakis, M., Daradoumis, T., and Caball\u00e9, S. (December, January 30). Emotion measurement in intelligent tutoring systems: What, when and how to measure. Proceedings of the 2011 Third International Conference on Intelligent Networking and Collaborative Systems, Fukuoka, Japan.","DOI":"10.1109\/INCoS.2011.82"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Saste, S.T., and Jagdale, S. (2017, January 20\u201322). Emotion recognition from speech using MFCC and DWT for security system. Proceedings of the 2017 international Conference of Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India.","DOI":"10.1109\/ICECA.2017.8203631"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3388790","article-title":"Driver emotion recognition for intelligent vehicles: A survey","volume":"53","author":"Zepf","year":"2020","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1037\/a0038822","article-title":"The relation between short-term emotion dynamics and psychological well-being: A meta-analysis","volume":"141","author":"Houben","year":"2015","journal-title":"Psychol. Bull."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1007\/s10803-009-0884-3","article-title":"Emotion recognition in children with autism spectrum disorders: Relations to eye gaze and autonomic state","volume":"40","author":"Bal","year":"2010","journal-title":"J. Autism Dev. Disord."},{"key":"ref_18","unstructured":"Mart\u00ednez, R., Ipi\u00f1a, K., Irigoyen, E., Asla, N., Garay, N., Ezeiza, A., and Fajardo, I. (2010). Trends in Practical Applications of Agents and Multiagent Systems: 8th International Conference on Practical Applications of Agents and Multiagent Systems, Springer."},{"key":"ref_19","unstructured":"Ekman, P. (1971). Nebraska Symposium on Motivation, University of Nebraska Press."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Darwin, C., and Prodger, P. (1998). The Expression of the Emotions in Man and Animals, Oxford University Press.","DOI":"10.1093\/oso\/9780195112719.002.0002"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1126\/science.164.3875.86","article-title":"Pan-cultural elements in facial displays of emotion","volume":"164","author":"Ekman","year":"1969","journal-title":"Science"},{"key":"ref_22","unstructured":"Plutchik, R. (2003). Emotions and Life: Perspectives from Psychology, Biology, and Evolution, American Psychological Association."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1007\/s12144-014-9219-4","article-title":"Pleasure, arousal, dominance: Mehrabian and Russell revisited","volume":"33","author":"Bakker","year":"2014","journal-title":"Curr. Psychol."},{"key":"ref_24","unstructured":"Mehrabian, A., and Russell, J.A. (1974). An Approach to Environmental Psychology, the MIT Press."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bain, A. (1864). The senses and the intellect, Longman, Green, Longman, Roberts, and Green.","DOI":"10.1037\/12243-000"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.jpdc.2020.06.003","article-title":"Differential privacy in blockchain technology: A futuristic approach","volume":"145","author":"Hassan","year":"2020","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5461","DOI":"10.1021\/acs.chemrev.8b00573","article-title":"Bio-integrated wearable systems: A comprehensive review","volume":"119","author":"Ray","year":"2019","journal-title":"Chem. Rev."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1109\/TAFFC.2020.2981446","article-title":"Deep facial expression recognition: A survey","volume":"13","author":"Li","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Schmid, P.C., Mast, M.S., Bombari, D., Mast, F.W., and Lobmaier, J. (2011). How mood states affect information processing during facial emotion recognition: An eye tracking study. Swiss J. Psychol.","DOI":"10.1024\/1421-0185\/a000060"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1016\/j.imavis.2012.06.005","article-title":"Static and dynamic 3D facial expression recognition: A comprehensive survey","volume":"30","author":"Sandbach","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TBME.2016.2609282","article-title":"Algorithmic principles of remote PPG","volume":"64","author":"Wang","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xie, K., Fu, C.-H., Liang, H., Hong, H., and Zhu, X. (2019, January 28\u201330). Non-contact heart rate monitoring for intensive exercise based on singular spectrum analysis. Proceedings of the 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, USA.","DOI":"10.1109\/MIPR.2019.00048"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"7714","DOI":"10.3390\/s130607714","article-title":"Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines","volume":"13","author":"Ghimire","year":"2013","journal-title":"Sensors"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhao, M., Adib, F., and Katabi, D. (2016, January 3\u20137). Emotion recognition using wireless signals. Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, New York, NY, USA.","DOI":"10.1145\/2973750.2973762"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.inffus.2020.01.011","article-title":"Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review","volume":"59","author":"Zhang","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1016\/j.patcog.2016.07.026","article-title":"Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order","volume":"61","author":"Lopes","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1109\/TAFFC.2017.2695999","article-title":"Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition","volume":"10","author":"Kim","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/TCYB.2014.2354351","article-title":"Learning multiscale active facial patches for expression analysis","volume":"45","author":"Zhong","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 8\u201310). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"529","DOI":"10.3181\/00379727-37-9630","article-title":"Photoelectric plethysmography of the fingers and toes in man","volume":"37","author":"Hertzman","year":"1937","journal-title":"Proc. Soc. Exp. Biol. Med."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1445","DOI":"10.1109\/TIM.2011.2175832","article-title":"A novel approach for motion artifact reduction in PPG signals based on AS-LMS adaptive filter","volume":"61","author":"Ram","year":"2011","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2016","DOI":"10.1109\/TBME.2017.2676243","article-title":"Accurate heart rate monitoring during physical exercises using PPG","volume":"64","author":"Temko","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1109\/TBME.2010.2086456","article-title":"Advancements in noncontact, multiparameter physiological measurements using a webcam","volume":"58","author":"Poh","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Li, X., Chen, J., Zhao, G., and Pietikainen, M. (2014, January 24\u201327). Remote heart rate measurement from face videos under realistic situations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.543"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1088\/0967-3334\/35\/5\/807","article-title":"Non-contact video-based vital sign monitoring using ambient light and auto-regressive models","volume":"35","author":"Tarassenko","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10916-016-0439-z","article-title":"Introducing contactless blood pressure assessment using a high speed video camera","volume":"40","author":"Jeong","year":"2016","journal-title":"J. Med. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"23198","DOI":"10.1109\/JSEN.2021.3107429","article-title":"Non-contact Dual-modality emotion recognition system by CW radar and RGB camera","volume":"21","author":"Zhang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep Neural Networks for Acoustic Modeling in Speech Recognition","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_49","unstructured":"Martin, O., Kotsia, I., Macq, B., and Pitas, I. (2016, January 3\u20137). The eNTERFACE\u201905 audio-visual emotion database. Proceedings of the 22nd International Conference on Data Engineering Workshops (ICDEW\u201906), Atlanta, GA, USA."},{"key":"ref_50","unstructured":"Li, J., Deng, L., Haeb-Umbach, R., and Gong, Y. (2015). Robust Automatic Speech Recognition: A Bridge to Practical Applications, Academic Press."},{"key":"ref_51","first-page":"221","article-title":"Vocal correlates of emotional states","volume":"52","author":"Williams","year":"1981","journal-title":"Speech Eval. Psychiatry"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1145\/3129340","article-title":"Speech emotion recognition: Two decades in a nutshell, benchmarks, and ongoing trends","volume":"61","author":"Schuller","year":"2018","journal-title":"Commun. ACM"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1109\/10.846676","article-title":"Acoustical properties of speech as indicators of depression and suicidal risk","volume":"47","author":"France","year":"2000","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1016\/0167-6393(95)00007-B","article-title":"Icarus: Source generator based real-time recognition of speech in noisy stressful and lombard effect environments","volume":"16","author":"Hansen","year":"1995","journal-title":"Speech Commun."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Ang, J., Dhillon, R., Krupski, A., Shriberg, E., and Stolcke, A. (2002, January 16\u201320). Prosody-based automatic detection of annoyance and frustration in human-computer dialog. Proceedings of the INTERSPEECH, Denver, CO, USA.","DOI":"10.21437\/ICSLP.2002-559"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Cohen, R. (1984, January 2\u20136). A computational theory of the function of clue words in argument understanding. Proceedings of the 10th International Conference on Computational Linguistics and 22nd Annual Meeting of the Association for Computational Linguistics, Stanford University, Stanford, CA, USA.","DOI":"10.3115\/980491.980546"},{"key":"ref_57","first-page":"5235","article-title":"Recognizing emotions from whispered speech based on acoustic feature transfer learning","volume":"5","author":"Deng","year":"2017","journal-title":"IEEE Access"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.neucom.2019.07.049","article-title":"Multi-view laplacian least squares for human emotion recognition","volume":"370","author":"Guo","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_59","first-page":"175","article-title":"Attention, intentions, and the structure of discourse","volume":"12","author":"Grosz","year":"1986","journal-title":"Comput. Linguist."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Dellaert, F., Polzin, T., and Waibel, A. (1996, January 3\u20136). Recognizing emotion in speech. Proceedings of the Fourth International Conference on Spoken Language Processing. ICSLP\u201996, Philadelphia, PA, USA.","DOI":"10.21437\/ICSLP.1996-462"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Burmania, A., and Busso, C. (2017, January 20\u201324). A Stepwise Analysis of Aggregated Crowdsourced Labels Describing Multimodal Emotional Behaviors. Proceedings of the INTERSPEECH, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-1278"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Lee, S.-W. (2019, January 12\u201317). The generalization effect for multilingual speech emotion recognition across heterogeneous languages. Proceedings of the ICASSP 2019\u20142019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683046"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1146\/annurev-neuro-090121-014424","article-title":"Breathing rhythm and pattern and their influence on emotion","volume":"45","author":"Ashhad","year":"2022","journal-title":"Annu. Rev. Neurosci."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"11896","DOI":"10.1109\/ACCESS.2020.2964794","article-title":"Non-contact emotion recognition combining heart rate and facial expression for interactive gaming environments","volume":"8","author":"Du","year":"2020","journal-title":"IEEE Access"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"21434","DOI":"10.1364\/OE.16.021434","article-title":"Remote plethysmographic imaging using ambient light","volume":"16","author":"Verkruysse","year":"2008","journal-title":"Opt. Express"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"94160","DOI":"10.1109\/ACCESS.2019.2928691","article-title":"Interpretable emotion recognition using EEG signals","volume":"7","author":"Qing","year":"2019","journal-title":"IEEE Access"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"313","DOI":"10.5271\/sjweh.1469","article-title":"Influence of job strain and emotion on blood pressure in female hospital personnel during workhours","volume":"19","author":"Theorell","year":"1993","journal-title":"Scand. J. Work Environ. Health"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1109\/JIOT.2021.3097801","article-title":"Recent Advances in Contactless Sensing Technologies for Mental Health Monitoring","volume":"9","author":"Nouman","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/S0301-0511(98)00025-8","article-title":"The effects of emotional behaviour on components of the respiratory cycle","volume":"49","author":"Boiten","year":"1998","journal-title":"Biol. Psychol."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1378\/chest.125.2.683","article-title":"Respiratory sinus arrhythmia: Why does the heartbeat synchronize with respiratory rhythm?","volume":"125","author":"Yasuma","year":"2004","journal-title":"Chest"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/MMM.2008.930675","article-title":"Radar remote monitoring of vital signs","volume":"10","author":"Li","year":"2009","journal-title":"IEEE Microw. Mag."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1109\/JSEN.2019.2946095","article-title":"Bi-LSTM network for multimodal continuous human activity recognition and fall detection","volume":"20","author":"Li","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"3519","DOI":"10.1109\/TMTT.2017.2658567","article-title":"Comparison study of noncontact vital signs detection using a Doppler stepped-frequency continuous-wave radar and camera-based imaging photoplethysmography","volume":"65","author":"Ren","year":"2017","journal-title":"IEEE Trans. Microw. Theory Technol."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"4678","DOI":"10.1109\/TMTT.2013.2288226","article-title":"A hybrid radar-camera sensing system with phase compensation for random body movement cancellation in Doppler vital sign detection","volume":"61","author":"Gu","year":"2013","journal-title":"IEEE Trans. Microw. Theory Technol."},{"key":"ref_75","unstructured":"James, W. (2007). The Principles of Psychology, Cosimo, Inc."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/T-AFFC.2010.7","article-title":"Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis","volume":"1","author":"Petrantonakis","year":"2010","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/JBHI.2017.2688239","article-title":"DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices","volume":"22","author":"Katsigiannis","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/TAFFC.2014.2327617","article-title":"Emotion recognition based on multi-variant correlation of physiological signals","volume":"5","author":"Wen","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Jerritta, S., Murugappan, M., Nagarajan, R., and Wan, K. (2011, January 4\u20136). Physiological signals based human emotion recognition: A review. Proceedings of the 2011 IEEE 7th International Colloquium on Signal Processing and Its Applications, Penang, Malaysia.","DOI":"10.1109\/CSPA.2011.5759912"},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1088\/0967-3334\/32\/10\/002","article-title":"Comparison of blood volume pulse and skin conductance responses to mental and affective stimuli at different anatomical sites","volume":"32","author":"Kushki","year":"2011","journal-title":"Physiol. Meas."},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Lim, J.Z., Mountstephens, J., and Teo, J. (2020). Emotion recognition using eye-tracking: Taxonomy, review and current challenges. Sensors, 20.","DOI":"10.3390\/s20082384"},{"key":"ref_82","first-page":"164","article-title":"The argument and evidence about universals in facial expressions","volume":"Volume 143","author":"Ekman","year":"1989","journal-title":"Handbook of Social Psychophysiology"},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.05.004","article-title":"Pixel-level image fusion: A survey of the state of the art","volume":"33","author":"Li","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.inffus.2020.01.008","article-title":"Feature-level fusion approaches based on multimodal EEG data for depression recognition","volume":"59","author":"Cai","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/34.273716","article-title":"Decision combination in multiple classifier systems","volume":"16","author":"Ho","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.ins.2013.09.031","article-title":"A new adaptive decentralized soft decision combining rule for distributed sensor systems with data fusion","volume":"256","author":"Aziz","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.ins.2021.10.005","article-title":"A survey on facial emotion recognition techniques: A state-of-the-art literature review","volume":"582","author":"Canal","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Kartali, A., Rogli\u0107, M., Barjaktarovi\u0107, M., \u0110uri\u0107-Jovi\u010di\u0107, M., and Jankovi\u0107, M.M. (2018, January 20\u201321). Real-time algorithms for facial emotion recognition: A comparison of different approaches. Proceedings of the 2018 14th Symposium on Neural Networks and Applications (NEUREL), Belgrade, Serbia.","DOI":"10.1109\/NEUREL.2018.8587011"},{"key":"ref_89","first-page":"157","article-title":"Preprocessing signal for speech emotion recognition","volume":"28","author":"Nema","year":"2018","journal-title":"J. Sci."},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Beigi, H. (2011). Fundamentals of Speaker Recognition, Springer Science & Business Media.","DOI":"10.1007\/978-0-387-77592-0"},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1080\/02533839.2013.799946","article-title":"Emotion recognition from facial EMG signals using higher order statistics and principal component analysis","volume":"37","author":"Jerritta","year":"2014","journal-title":"J. Chin. Inst. Eng."},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1146\/annurev.psych.60.110707.163539","article-title":"Emotion theory and research: Highlights, unanswered questions, and emerging issues","volume":"60","author":"Izard","year":"2009","journal-title":"Annu. Rev. Psychol."},{"key":"ref_93","doi-asserted-by":"crossref","first-page":"1084","DOI":"10.1016\/j.eswa.2006.02.005","article-title":"EEG signal classification using wavelet feature extraction and a mixture of expert model","volume":"32","author":"Subasi","year":"2007","journal-title":"Expert Syst. Appl."},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1073\/pnas.88.6.2297","article-title":"Approximate entropy as a measure of system complexity","volume":"88","author":"Pincus","year":"1991","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_95","doi-asserted-by":"crossref","first-page":"H2039","DOI":"10.1152\/ajpheart.2000.278.6.H2039","article-title":"Physiological time-series analysis using approximate entropy and sample entropy","volume":"278","author":"Richman","year":"2000","journal-title":"Am. J. Physiol.-Heart Circ. Physiol."},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1103\/PhysRevLett.85.461","article-title":"Measuring information transfer","volume":"85","author":"Schreiber","year":"2000","journal-title":"Phys. Rev. Lett."},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1109\/TITS.2013.2275192","article-title":"Automated detection of driver fatigue based on entropy and complexity measures","volume":"15","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","article-title":"A global geometric framework for nonlinear dimensionality reduction","volume":"290","author":"Tenenbaum","year":"2000","journal-title":"Science"},{"key":"ref_99","doi-asserted-by":"crossref","unstructured":"Abdulrahman, M., Gwadabe, T.R., Abdu, F.J., and Eleyan, A. (2014, January 23\u201325). Gabor wavelet transform based facial expression recognition using PCA and LBP. Proceedings of the 2014 22nd Signal Processing and Communications Applications Conference (SIU), Trabzon, Turkey.","DOI":"10.1109\/SIU.2014.6830717"},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"3039","DOI":"10.1007\/s11042-020-09726-4","article-title":"AutoFER: PCA and PSO based automatic facial emotion recognition","volume":"80","author":"Arora","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1109\/TPAMI.2004.1261097","article-title":"Two-dimensional PCA: A new approach to appearance-based face representation and recognition","volume":"26","author":"Yang","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TAFFC.2016.2588488","article-title":"A combined rule-based & machine learning audio-visual emotion recognition approach","volume":"9","author":"Seng","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_103","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Dahmane, M., and Meunier, J. (2011, January 21\u201325). Emotion recognition using dynamic grid-based HoG features. Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771368"},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Kumar, P., Happy, S., and Routray, A. (2016, January 19\u201321). A real-time robust facial expression recognition system using HOG features. Proceedings of the 2016 International Conference on Computing, Analytics and Security Trends (CAST), Pune, India.","DOI":"10.1109\/CAST.2016.7914982"},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"11563","DOI":"10.1007\/s11042-022-12438-6","article-title":"Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithm","volume":"81","author":"Hussein","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_107","doi-asserted-by":"crossref","unstructured":"Ahonen, T., Hadid, A., and Pietik\u00e4inen, M. (2004, January 11\u201314). Face recognition with local binary patterns. Proceedings of the European conference on Computer Vision, Prague, Czech Republic.","DOI":"10.1007\/978-3-540-24670-1_36"},{"key":"ref_108","doi-asserted-by":"crossref","unstructured":"Chintalapati, S., and Raghunadh, M. (2013, January 26\u201328). Automated attendance management system based on face recognition algorithms. Proceedings of the 2013 IEEE International Conference on Computational Intelligence and Computing Research, Enathi, India.","DOI":"10.1109\/ICCIC.2013.6724266"},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s10772-018-9491-z","article-title":"Databases, features and classifiers for speech emotion recognition: A review","volume":"21","author":"Swain","year":"2018","journal-title":"Int. J. Speech Technol."},{"key":"ref_110","unstructured":"Molau, S., Pitz, M., Schluter, R., and Ney, H. (2001, January 7\u201311). Computing mel-frequency cepstral coefficients on the power spectrum. Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (cat. No. 01CH37221), Salt Lake City, UT, USA."},{"key":"ref_111","unstructured":"Wong, E., and Sridharan, S. (2001, January 4). Comparison of linear prediction cepstrum coefficients and mel-frequency cepstrum coefficients for language identification. Proceedings of the 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No. 01EX489), Hong Kong, China."},{"key":"ref_112","doi-asserted-by":"crossref","unstructured":"Bandela, S.R., and Kumar, T.K. (2018, January 9\u201313). Emotion recognition of stressed speech using teager energy and linear prediction features. Proceedings of the 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT), Mumbai, India.","DOI":"10.1109\/ICALT.2018.00107"},{"key":"ref_113","unstructured":"Idris, I., and Salam, M.S. (2015, January 15\u201317). Improved speech emotion classification from spectral coefficient optimization. Proceedings of the Advances in Machine Learning and Signal Processing: Proceedings of MALSIP 2015, Ho Chi Minh City, Vietnam."},{"key":"ref_114","doi-asserted-by":"crossref","unstructured":"Feraru, S.M., and Zbancioc, M.D. (2013, January 21\u201323). Emotion recognition in Romanian language using LPC features. Proceedings of the 2013 E-Health and Bioengineering Conference (EHB), Iasi, Romania.","DOI":"10.1109\/EHB.2013.6707314"},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"200953","DOI":"10.1109\/ACCESS.2020.3035531","article-title":"A hybrid meta-heuristic feature selection method using golden ratio and equilibrium optimization algorithms for speech emotion recognition","volume":"8","author":"Dey","year":"2020","journal-title":"IEEE Access"},{"key":"ref_116","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/97.889636","article-title":"Wavelet speech enhancement based on the teager energy operator","volume":"8","author":"Bahoura","year":"2001","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_117","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.procs.2020.08.027","article-title":"Speech emotion recognition with deep learning","volume":"176","author":"Aouani","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_118","doi-asserted-by":"crossref","unstructured":"Li, X., Li, X., Zheng, X., and Zhang, D. (2010, January 17\u201320). EMD-TEO based speech emotion recognition. Proceedings of the Life System Modeling and Intelligent Computing: International Conference on Life System Modeling and Simulation, LSMS 2010, and International Conference on Intelligent Computing for Sustainable Energy and Environment, ICSEE 2010, Wuxi, China.","DOI":"10.1007\/978-3-642-15853-7"},{"key":"ref_119","doi-asserted-by":"crossref","unstructured":"Bandela, S.R., and Kumar, T.K. (2017, January 3\u20135). Stressed speech emotion recognition using feature fusion of teager energy operator and MFCC. Proceedings of the 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Delhi, India.","DOI":"10.1109\/ICCCNT.2017.8204149"},{"key":"ref_120","doi-asserted-by":"crossref","unstructured":"You, M., Chen, C., Bu, J., Liu, J., and Tao, J. (2006, January 9\u201312). Emotion recognition from noisy speech. Proceedings of the 2006 IEEE International Conference on Multimedia and Expo, Toronto, ON, Canada.","DOI":"10.1109\/ICME.2006.262865"},{"key":"ref_121","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1109\/PROC.1975.9799","article-title":"Digital representations of speech signals","volume":"63","author":"Schafer","year":"1975","journal-title":"Proc. IEEE"},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"1664","DOI":"10.1109\/PROC.1967.5957","article-title":"What is the fast Fourier transform?","volume":"55","author":"Cochran","year":"1967","journal-title":"Proc. IEEE"},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Murugappan, M., and Murugappan, S. (2013, January 8\u201310). Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT). Proceedings of the 2013 IEEE 9th International Colloquium on Signal Processing and Its Applications, Kuala Lumpur, Malaysia.","DOI":"10.1109\/CSPA.2013.6530058"},{"key":"ref_124","doi-asserted-by":"crossref","first-page":"107260","DOI":"10.1016\/j.apacoust.2020.107260","article-title":"Emotion recognition using fourier transform and genetic programming","volume":"164","author":"Acharya","year":"2020","journal-title":"Appl. Acoust."},{"key":"ref_125","doi-asserted-by":"crossref","first-page":"2901","DOI":"10.1109\/TNNLS.2020.3008938","article-title":"Time\u2013frequency representation and convolutional neural network-based emotion recognition","volume":"32","author":"Khare","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_126","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1109\/TAFFC.2017.2712143","article-title":"Identifying stable patterns over time for emotion recognition from EEG","volume":"10","author":"Zheng","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_128","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.eswa.2015.10.049","article-title":"Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers","volume":"47","author":"Atkinson","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_129","doi-asserted-by":"crossref","unstructured":"Alazrai, R., Homoud, R., Alwanni, H., and Daoud, M. (2018). EEG-based emotion recognition using quadratic time-frequency distribution. Sensors, 18.","DOI":"10.3390\/s18082739"},{"key":"ref_130","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.neucom.2018.05.005","article-title":"Speech emotion recognition based on an improved brain emotion learning model","volume":"309","author":"Liu","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_131","first-page":"2","article-title":"Locality preserving projections","volume":"16","author":"He","year":"2003","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_132","unstructured":"Kira, K., and Rendell, L.A. (1992). Machine Learning Proceedings 1992, Elsevier."},{"key":"ref_133","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_134","unstructured":"Soentpiet, R. (1999). Advances in Kernel Methods: Support Vector Learning, MIT Press."},{"key":"ref_135","first-page":"1143","article-title":"Support vector machine soft margin classifiers: Error analysis","volume":"5","author":"Chen","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_136","first-page":"101","article-title":"Speech Emotion Recognition Using Support Vector Machine","volume":"6","author":"Pan","year":"2012","journal-title":"Int. J. Smart Home"},{"key":"ref_137","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1016\/j.specom.2010.02.010","article-title":"Class-level spectral features for emotion recognition","volume":"52","author":"Bitouk","year":"2010","journal-title":"Speech Commun."},{"key":"ref_138","doi-asserted-by":"crossref","first-page":"7803","DOI":"10.1007\/s11042-016-3418-y","article-title":"Facial expression recognition based on local region specific features and support vector machines","volume":"76","author":"Ghimire","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_139","doi-asserted-by":"crossref","first-page":"6351","DOI":"10.1016\/j.eswa.2013.05.050","article-title":"Emotion detection in suicide notes","volume":"40","author":"Desmet","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_140","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_141","doi-asserted-by":"crossref","unstructured":"Hu, H., Xu, M.-X., and Wu, W. (2007, January 15\u201320). GMM supervector based SVM with spectral features for speech emotion recognition. Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP\u201907, Honolulu, HI, USA.","DOI":"10.1109\/ICASSP.2007.366937"},{"key":"ref_142","doi-asserted-by":"crossref","first-page":"26777","DOI":"10.1109\/ACCESS.2019.2901352","article-title":"Emotion recognition using hybrid Gaussian mixture model and deep neural network","volume":"7","author":"Shahin","year":"2019","journal-title":"IEEE Access"},{"key":"ref_143","doi-asserted-by":"crossref","unstructured":"Zhang, C., Li, M., and Wu, D. (IEEE Trans. Intell. Transp. Syst., 2022). Federated Multidomain Learning With Graph Ensemble Autoencoder GMM for Emotion Recognition, IEEE Trans. Intell. Transp. Syst., Early Access.","DOI":"10.1109\/TITS.2022.3203800"},{"key":"ref_144","unstructured":"Cohen, I., Garg, A., and Huang, T.S. (2000, January 1\u20132). Emotion recognition from facial expressions using multilevel HMM. Proceedings of the Neural Information PROCESSING systems, Breckenridge, CO, USA."},{"key":"ref_145","doi-asserted-by":"crossref","unstructured":"Schuller, B., Rigoll, G., and Lang, M. (2003, January 6\u201310). Hidden Markov model-based speech emotion recognition. Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP \u201803), Hong Kong, China.","DOI":"10.1109\/ICME.2003.1220939"},{"key":"ref_146","doi-asserted-by":"crossref","first-page":"1880","DOI":"10.1109\/TMM.2013.2269314","article-title":"Two-level hierarchical alignment for semi-coupled HMM-based audiovisual emotion recognition with temporal course","volume":"15","author":"Wu","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_147","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1016\/j.knosys.2018.11.014","article-title":"Hidden topic\u2013emotion transition model for multi-level social emotion detection","volume":"164","author":"Tang","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_148","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_149","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.ins.2019.09.005","article-title":"Two-layer fuzzy multiple random forest for speech emotion recognition in human-robot interaction","volume":"509","author":"Chen","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_150","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.bspc.2010.12.001","article-title":"An integrated system based on physiological signals for the assessment of affective states in patients with anxiety disorders","volume":"6","author":"Katsis","year":"2011","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_151","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_152","first-page":"107101","article-title":"Att-Net: Enhanced emotion recognition system using lightweight self-attention module","volume":"102","author":"Kwon","year":"2021","journal-title":"Proc. IEEE Haffner"},{"key":"ref_153","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1109\/TAFFC.2020.3014171","article-title":"Exploiting multi-cnn features in cnn-rnn based dimensional emotion recognition on the omg in-the-wild dataset","volume":"12","author":"Kollias","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_154","doi-asserted-by":"crossref","first-page":"1576","DOI":"10.1109\/TMM.2017.2766843","article-title":"Speech emotion recognition using deep convolutional neural network and discriminant temporal pyramid matching","volume":"20","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Multimed."},{"key":"ref_155","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TAFFC.2018.2817622","article-title":"EEG emotion recognition using dynamical graph convolutional neural networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_156","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.eij.2020.07.005","article-title":"A 3D-convolutional neural network framework with ensemble learning techniques for multi-modal emotion recognition","volume":"22","author":"Salama","year":"2021","journal-title":"Egypt. Inform. J."},{"key":"ref_157","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_158","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1007\/s12559-021-09865-2","article-title":"When old meets new: Emotion recognition from speech signals","volume":"13","author":"Gloor","year":"2021","journal-title":"Cogn. Comput."},{"key":"ref_159","doi-asserted-by":"crossref","first-page":"622759","DOI":"10.3389\/fnins.2020.622759","article-title":"An investigation of deep learning models for EEG-based emotion recognition","volume":"14","author":"Zhang","year":"2020","journal-title":"Front. Neurosci."},{"key":"ref_160","doi-asserted-by":"crossref","first-page":"114683","DOI":"10.1016\/j.eswa.2021.114683","article-title":"Speech emotion recognition using recurrent neural networks with directional self-attention","volume":"173","author":"Li","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_161","doi-asserted-by":"crossref","unstructured":"Kim, Y., Lee, H., and Provost, E.M. (2013, January 21). Deep learning for robust feature generation in audiovisual emotion recognition. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638346"},{"key":"ref_162","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.inffus.2018.10.009","article-title":"Human emotion recognition using deep belief network architecture","volume":"51","author":"Hassan","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_163","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10723-021-09564-0","article-title":"Speech expression multimodal emotion recognition based on deep belief network","volume":"19","author":"Liu","year":"2021","journal-title":"J. Grid Comput."},{"key":"ref_164","doi-asserted-by":"crossref","first-page":"4525","DOI":"10.1109\/ACCESS.2017.2676238","article-title":"Facial expression recognition utilizing local direction-based robust features and deep belief network","volume":"5","author":"Uddin","year":"2017","journal-title":"IEEE Access"},{"key":"ref_165","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_166","doi-asserted-by":"crossref","unstructured":"Jiang, H., He, P., Chen, W., Liu, X., Gao, J., and Zhao, T. (2020, January 5\u201310). SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Washington, DC, USA.","DOI":"10.18653\/v1\/2020.acl-main.197"},{"key":"ref_167","doi-asserted-by":"crossref","unstructured":"Shukla, A., Vougioukas, K., Ma, P., Petridis, S., and Pantic, M. (2020, January 4\u20138). Visually guided self supervised learning of speech representations. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053415"},{"key":"ref_168","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.neucom.2022.10.013","article-title":"In search of a robust facial expressions recognition model: A large-scale visual cross-corpus study","volume":"514","author":"Ryumina","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_169","doi-asserted-by":"crossref","unstructured":"Chudasama, V., Kar, P., Gudmalwar, A., Shah, N., Wasnik, P., and Onoe, N. (2022, January 21\u201324). M2FNet: Multi-modal fusion network for emotion recognition in conversation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00511"},{"key":"ref_170","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.knosys.2018.07.041","article-title":"Multimodal sentiment analysis using hierarchical fusion with context modeling","volume":"161","author":"Majumder","year":"2018","journal-title":"Knowl.-Based Syst."},{"key":"ref_171","doi-asserted-by":"crossref","unstructured":"Song, X., Zang, L., Zhang, R., Hu, S., and Huang, L. (2022, January 23\u201327). Emotionflow: Capture the dialogue level emotion transitions. Proceedings of the ICASSP 2022\u20142022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746464"},{"key":"ref_172","doi-asserted-by":"crossref","unstructured":"Ding, H., Zhou, S.K., and Chellappa, R. (June, January 30). Facenet2expnet: Regularizing a deep face recognition net for expression recognition. Proceedings of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), Washington, DC, USA.","DOI":"10.1109\/FG.2017.23"},{"key":"ref_173","doi-asserted-by":"crossref","first-page":"2132","DOI":"10.1109\/TAFFC.2022.3188390","article-title":"Classifying emotions and engagement in online learning based on a single facial expression recognition neural network","volume":"13","author":"Savchenko","year":"2022","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_174","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Wang, C., Ling, X., and Deng, W. (2022, January 23\u201327). Learn from all: Erasing attention consistency for noisy label facial expression recognition. Proceedings of the Computer Vision\u2013ECCV 2022: 17th European Conference, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19809-0_24"},{"key":"ref_175","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1109\/TCDS.2020.2999337","article-title":"A novel bi-hemispheric discrepancy model for EEG emotion recognition","volume":"13","author":"Li","year":"2020","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_176","doi-asserted-by":"crossref","unstructured":"Paraskevopoulos, G., Georgiou, E., and Potamianos, A. (2020, January 23\u201327). Mmlatch: Bottom-up top-down fusion for multimodal sentiment analysis. Proceedings of the ICASSP 2022\u20142022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746418"},{"key":"ref_177","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1109\/TMI.2016.2528162","article-title":"Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning","volume":"35","author":"Shin","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_178","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1016\/j.patcog.2010.09.020","article-title":"Survey on speech emotion recognition: Features, classification schemes, and databases","volume":"44","author":"Kamel","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_179","doi-asserted-by":"crossref","unstructured":"Ringeval, F., Sonderegger, A., Sauer, J., and Lalanne, D. (2013, January 22\u201326). Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China.","DOI":"10.1109\/FG.2013.6553805"},{"key":"ref_180","doi-asserted-by":"crossref","unstructured":"Coan, J.A., and Allen, J.J. (2007). Handbook of Emotion Elicitation and Assessment, Oxford University Press.","DOI":"10.1093\/oso\/9780195169157.001.0001"},{"key":"ref_181","unstructured":"Douglas-Cowie, E., Cowie, R., and Schr\u00f6der, M. (2000, January 5\u20137). A new emotion database: Considerations, sources and scope. Proceedings of the ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion, Newcastle, Northern Ireland, UK."},{"key":"ref_182","doi-asserted-by":"crossref","unstructured":"Grimm, M., Kroschel, K., and Narayanan, S. (2008, January 26). The Vera am Mittag German audio-visual emotional speech database. Proceedings of the 2008 IEEE International Conference on Multimedia and Expo, Hannover, Germany.","DOI":"10.1109\/ICME.2008.4607572"},{"key":"ref_183","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/TSA.2004.838534","article-title":"Toward detecting emotions in spoken dialogs","volume":"13","author":"Lee","year":"2005","journal-title":"IEEE Trans. Speech Audio Process."},{"key":"ref_184","doi-asserted-by":"crossref","unstructured":"Dredze, M., Crammer, K., and Pereira, F. (2008, January 5\u20139). Confidence-weighted linear classification. Proceedings of the 25th International Conference on Machine Learning, New York, NY, USA.","DOI":"10.1145\/1390156.1390190"},{"key":"ref_185","unstructured":"Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A.Y., and Potts, C. (2013, January 18\u201321). Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language processing, Seattle, WA, USA."},{"key":"ref_186","unstructured":"Maas, A., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., and Potts, C. (2011, January 19\u201324). Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA."},{"key":"ref_187","doi-asserted-by":"crossref","unstructured":"Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W.F., and Weiss, B. (2005, January 4\u20138). A database of German emotional speech. Proceedings of the Interspeech, Lisbon, Portugal.","DOI":"10.21437\/Interspeech.2005-446"},{"key":"ref_188","unstructured":"Jackson, P. (2014). Surrey Audio-Visual Expressed Emotion (Savee) Database, University of Surrey."},{"key":"ref_189","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1109\/TAFFC.2014.2336244","article-title":"Crema-d: Crowd-sourced emotional multimodal actors dataset","volume":"5","author":"Cao","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_190","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/s10579-008-9076-6","article-title":"IEMOCAP: Interactive emotional dyadic motion capture database","volume":"42","author":"Busso","year":"2008","journal-title":"Lang. Resour. Eval."},{"key":"ref_191","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1109\/TASL.2006.876113","article-title":"Prosody conversion from neutral speech to emotional speech","volume":"14","author":"Tao","year":"2006","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_192","unstructured":"Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., and Mihalcea, R. (August, January 28). MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_193","unstructured":"Batliner, A., Steidl, S., and N\u00f6th, E. (June, January 26\u201327&31). Releasing a Thoroughly Annotated and Processed Spontaneous Emotional Database: The FAU Aibo Emotion Corpus. Proceedings of the Satellite Workshop of LREC 2008 on Corpora for Research on Emotion and Affect, Marrakech, Morocco."},{"key":"ref_194","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/T-AFFC.2011.20","article-title":"The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent","volume":"3","author":"McKeown","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_195","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1007\/s12652-016-0406-z","article-title":"CHEAVD: A Chinese natural emotional audio\u2013visual database","volume":"8","author":"Li","year":"2017","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_196","unstructured":"Pantic, M., Valstar, M., Rademaker, R., and Maat, L. (2005, January 6). Web-based database for facial expression analysis. Proceedings of the 2005 IEEE international conference on multimedia and Expo, Amsterdam, The Netherlands."},{"key":"ref_197","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1016\/j.imavis.2014.06.002","article-title":"Bp4d-spontaneous: A high-resolution spontaneous 3d dynamic facial expression database","volume":"32","author":"Zhang","year":"2014","journal-title":"Image Vis. Comput."},{"key":"ref_198","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_199","doi-asserted-by":"crossref","unstructured":"Zhang, X., Yin, L., Cohn, J.F., Canavan, S., Reale, M., Horowitz, A., and Liu, P. (2013, January 22\u201326). A high-resolution spontaneous 3d dynamic facial expression database. Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China.","DOI":"10.1109\/FG.2013.6553788"},{"key":"ref_200","doi-asserted-by":"crossref","first-page":"1022","DOI":"10.1109\/TPAMI.2019.2944808","article-title":"Sewa db: A rich database for audio-visual emotion and sentiment research in the wild","volume":"43","author":"Kossaifi","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_201","unstructured":"Valstar, M., and Pantic, M. (2010, January 23). Induced disgust, happiness and surprise: An addition to the mmi facial expression database. Proceedings of the 3rd International Workshop on EMOTION (Satellite of LREC): Corpora for Research on Emotion and Affect, Valletta, Malta."},{"key":"ref_202","unstructured":"Lyons, M., Akamatsu, S., Kamachi, M., and Gyoba, J. (1998, January 14\u201316). Coding facial expressions with gabor wavelets. Proceedings of the Proceedings Third IEEE international Conference on Automatic Face and Gesture Recognition, Nara, Japan."},{"key":"ref_203","unstructured":"Yin, L., Wei, X., Sun, Y., Wang, J., and Rosato, M.J. (2006, January 10\u201312). A 3D facial expression database for facial behavior research. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR06), Southampton, UK."},{"key":"ref_204","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"Affectnet: A database for facial expression, valence, and arousal computing in the wild","volume":"10","author":"Mollahosseini","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_205","doi-asserted-by":"crossref","unstructured":"Li, S., Deng, W., and Du, J. (2017, January 22\u201325). Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.277"},{"key":"ref_206","doi-asserted-by":"crossref","unstructured":"Fabian Benitez-Quiroz, C., Srinivasan, R., and Martinez, A.M. (2016, January 27\u201330). Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.600"},{"key":"ref_207","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"Deap: A database for emotion analysis; using physiological signals","volume":"3","author":"Koelstra","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_208","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1109\/TAFFC.2015.2392932","article-title":"DECAF: MEG-based multimodal database for decoding affective physiological responses","volume":"6","author":"Abadi","year":"2015","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_209","first-page":"479","article-title":"Amigos: A dataset for affect, personality and mood research on individuals and groups","volume":"12","author":"Abadi","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_210","doi-asserted-by":"crossref","first-page":"026017","DOI":"10.1088\/1741-2552\/aa5a98","article-title":"A multimodal approach to estimating vigilance using EEG and forehead EOG","volume":"14","author":"Zheng","year":"2017","journal-title":"J. Neural Eng."},{"key":"ref_211","doi-asserted-by":"crossref","first-page":"101835","DOI":"10.1016\/j.bspc.2019.101835","article-title":"Study on the usage feasibility of continuous-wave radar for emotion recognition","volume":"58","author":"Gouveia","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_212","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1038\/s41928-019-0258-6","article-title":"Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor","volume":"2","author":"Mercuri","year":"2019","journal-title":"Nat. Electron."},{"key":"ref_213","doi-asserted-by":"crossref","first-page":"1796","DOI":"10.1049\/rsn2.12297","article-title":"Emotion recognition method using millimetre wave radar based on deep learning","volume":"16","author":"Dang","year":"2022","journal-title":"IET Radar Sonar Navig."},{"key":"ref_214","unstructured":"Zadeh, A.B., Liang, P.P., Poria, S., Cambria, E., and Morency, L.-P. (2018, January 15\u201320). Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia."},{"key":"ref_215","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/T-AFFC.2011.25","article-title":"A multimodal database for affect recognition and implicit tagging","volume":"3","author":"Soleymani","year":"2011","journal-title":"IEEE Trans. Affect. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2455\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:40:04Z","timestamp":1760121604000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2455"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,23]]},"references-count":215,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052455"],"URL":"https:\/\/doi.org\/10.3390\/s23052455","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,23]]}}}