{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T17:52:23Z","timestamp":1771523543806,"version":"3.50.1"},"reference-count":81,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T00:00:00Z","timestamp":1628208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJCE1309"],"award-info":[{"award-number":["JPMJCE1309"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP20H03553"],"award-info":[{"award-number":["JP20H03553"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>During social interaction, humans recognize others\u2019 emotions via individual features and interpersonal features. However, most previous automatic emotion recognition techniques only used individual features\u2014they have not tested the importance of interpersonal features. In the present study, we asked whether interpersonal features, especially time-lagged synchronization features, are beneficial to the performance of automatic emotion recognition techniques. We explored this question in the main experiment (speaker-dependent emotion recognition) and supplementary experiment (speaker-independent emotion recognition) by building an individual framework and interpersonal framework in visual, audio, and cross-modality, respectively. Our main experiment results showed that the interpersonal framework outperformed the individual framework in every modality. Our supplementary experiment showed\u2014even for unknown communication pairs\u2014that the interpersonal framework led to a better performance. Therefore, we concluded that interpersonal features are useful to boost the performance of automatic emotion recognition tasks. We hope to raise attention to interpersonal features in this study.<\/jats:p>","DOI":"10.3390\/s21165317","type":"journal-article","created":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T08:01:42Z","timestamp":1628236902000},"page":"5317","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Incorporating Interpersonal Synchronization Features for Automatic Emotion Recognition from Visual and Audio Data during Communication"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0685-9532","authenticated-orcid":false,"given":"Jingyu","family":"Quan","sequence":"first","affiliation":[{"name":"Department of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoshihiro","family":"Miyake","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tokyo Institute of Technology, Yokohama 226-8502, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6300-4373","authenticated-orcid":false,"given":"Takayuki","family":"Nozawa","sequence":"additional","affiliation":[{"name":"Research Institute for the Earth Inclusive Sensing, Tokyo Institute of Technology, Tokyo 152-8550, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Planalp, S. (1999). Communicating Emotion: Social, Moral, and Cultural Processes, Cambridge University Press.","DOI":"10.1017\/CBO9781316257012"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hazarika, D., Poria, S., Zadeh, A., Cambria, E., Morency, L.P., and Zimmermann, R. (2018, January 1\u20136). Conversational memory network for emotion recognition in dyadic dialogue videos. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1193"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hazarika, D., Poria, S., Mihalcea, R., Cambria, E., and Zimmermann, R. (November, January 31). ICON: Interactive conversational memory network for multimodal emotion detection. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1280"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ko\u0142akowska, A., Landowska, A., Szwoch, M., Szwoch, W., and Wr\u00f3bel, M.R. (2013, January 6\u20138). Emotion recognition and its application in software engineering. Proceedings of the 2013 6th International Conference on Human System Interactions (HSI), Sopot, Poland.","DOI":"10.1109\/HSI.2013.6577877"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Eyben, F., W\u00f6llmer, M., and Schuller, B. (2009, January 10\u201312). OpenEAR\u2014introducing the Munich open-source emotion and affect recognition toolkit. Proceedings of the 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, The Netherlands.","DOI":"10.1109\/ACII.2009.5349350"},{"key":"ref_6","unstructured":"Poria, S., Hazarika, D., Majumder, N., and Mihalcea, R. (2020). Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research. IEEE Trans. Affect. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"70","DOI":"10.3389\/fnhum.2020.00070","article-title":"Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation","volume":"14","author":"Flynn","year":"2020","journal-title":"Front. Hum. Neurosci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1037\/0022-3514.51.4.690","article-title":"Cues and channels in emotion recognition","volume":"51","author":"Wallbott","year":"1986","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1001\/archpsyc.1986.01800030098011","article-title":"Recognition of emotion from vocal cues","volume":"43","author":"Johnson","year":"1986","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1111\/j.2044-835X.2012.02081.x","article-title":"Children\u2019s recognition of emotions from vocal cues","volume":"31","author":"Sauter","year":"2013","journal-title":"Br. J. Dev. Psychol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1177\/1534582302001001003","article-title":"Recognizing emotion from facial expressions: Psychological and neurological mechanisms","volume":"1","author":"Adolphs","year":"2002","journal-title":"Behav. Cogn. Neurosci. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1927","DOI":"10.1098\/rspb.1998.0522","article-title":"Neural structures associated with recognition of facial expressions of basic emotions","volume":"265","author":"Sprengelmeyer","year":"1998","journal-title":"Proc. R. Soc. Lond. Ser. B Biol. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1109\/TAFFC.2018.2874986","article-title":"Survey on emotional body gesture recognition","volume":"12","author":"Noroozi","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Saha, S., Datta, S., Konar, A., and Janarthanan, R. (2014, January 10\u201312). A study on emotion recognition from body gestures using Kinect sensor. Proceedings of the 2014 International Conference on Communication and Signal Processing, Bangkok, Thailand.","DOI":"10.1109\/ICCSP.2014.6949798"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cook, N.D. (2002). Tone of Voice and Mind: The Connections between Intonation, Emotion, Cognition, and Consciousness, John Benjamins Publishing.","DOI":"10.1075\/aicr.47"},{"key":"ref_16","first-page":"328","article-title":"Gaze and Mutual Gaze in Social Encounters: How long\u2014And when\u2014We look others\u201c in the eye\u201d is one of the main signals in nonverbal communication","volume":"65","author":"Cook","year":"1977","journal-title":"Am. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1023\/A:1024969514170","article-title":"Facial-affective behavior, mutual gaze, and emotional experience in dyadic interactions","volume":"21","author":"Merten","year":"1997","journal-title":"J. Nonverbal Behav."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-50960-0","article-title":"Interpersonal synchrony feels good but impedes self-regulation of affect","volume":"9","author":"Galbusera","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"116","DOI":"10.3389\/frobt.2019.00116","article-title":"Synchronization in interpersonal speech","volume":"6","author":"Amiriparian","year":"2019","journal-title":"Front. Robot. AI"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-021-91831-x","article-title":"Interpersonal physiological synchrony is associated with first person and third person subjective assessments of excitement during cooperative joint tasks","volume":"11","author":"Murata","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1006\/jrpe.1993.1012","article-title":"Effects of mutual gaze and touch on attraction, mood, and cardiovascular reactivity","volume":"27","author":"Williams","year":"1993","journal-title":"J. Res. Personal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.neubiorev.2017.05.013","article-title":"Connecting minds and sharing emotions through mimicry: A neurocognitive model of emotional contagion","volume":"80","author":"Prochazkova","year":"2017","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-020-00630-y","article-title":"K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations","volume":"7","author":"Park","year":"2020","journal-title":"Sci. Data"},{"key":"ref_24","unstructured":"Nakatsu, R., Nicholson, J., and Tosa, N. (November, January 30). Emotion recognition and its application to computer agents with spontaneous interactive capabilities. Proceedings of the Seventh ACM International Conference on Multimedia (Part 1), Orlando, FL, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rupauliha, K., Goyal, A., Saini, A., Shukla, A., and Swaminathan, S. (2020, January 24\u201326). Multimodal Emotion Recognition in Polish (Student Consortium). Proceedings of the 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM), New Delhi, India.","DOI":"10.1109\/BigMM50055.2020.00054"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Meng, H., Romera-Paredes, B., and Bianchi-Berthouze, N. (2011, January 21\u201325). Emotion recognition by two view SVM_2K classifier on dynamic facial expression features. Proceedings of the Face and Gesture 2011, Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771362"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mehta, D., Siddiqui, M.F.H., and Javaid, A.Y. (2018). Facial emotion recognition: A survey and real-world user experiences in mixed reality. Sensors, 18.","DOI":"10.3390\/s18020416"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ozdemir, M.A., Elagoz, B., Alaybeyoglu, A., Sadighzadeh, R., and Akan, A. (2019, January 3\u20135). Real time emotion recognition from facial expressions using cnn architecture. Proceedings of the 2019 Medical Technologies Congress (TIPTEKNO), Izmir, Turkey.","DOI":"10.1109\/TIPTEKNO.2019.8895215"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"012004","DOI":"10.1088\/1742-6596\/1193\/1\/012004","article-title":"Emotion recognition from facial expression using deep convolutional neural network","volume":"1193","author":"Liliana","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.jvcir.2018.12.039","article-title":"Video facial emotion recognition based on local enhanced motion history image and CNN-CTSLSTM networks","volume":"59","author":"Hu","year":"2019","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s42452-020-2234-1","article-title":"Facial emotion recognition using convolutional neural networks (FERC)","volume":"2","author":"Mehendale","year":"2020","journal-title":"SN Appl. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.jvcir.2018.05.024","article-title":"Histogram-based local descriptors for facial expression recognition (FER): A comprehensive study","volume":"55","author":"Turan","year":"2018","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dang, K., and Sharma, S. (2017, January 12\u201313). Review and comparison of face detection algorithms. Proceedings of the 2017 7th International Conference on Cloud Computing, Data Science & Engineering-Confluence, Noida, India.","DOI":"10.1109\/CONFLUENCE.2017.7943228"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1007\/s10462-018-9650-2","article-title":"Face detection techniques: A review","volume":"52","author":"Kumar","year":"2019","journal-title":"Artif. Intell. Rev."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Al-Allaf, O.N. (2014). Review of face detection systems based artificial neural networks algorithms. arXiv.","DOI":"10.5121\/ijma.2013.6101"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"St\u0105por, K. (2017, January 20\u201322). Evaluating and comparing classifiers: Review, some recommendations and limitations. Proceedings of the International Conference on Computer Recognition Systems, Polanica-Zdr\u00f3j, Poland.","DOI":"10.1007\/978-3-319-59162-9_2"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"19626","DOI":"10.1109\/ACCESS.2018.2813079","article-title":"Classifiers combination techniques: A comprehensive review","volume":"6","author":"Mohandes","year":"2018","journal-title":"IEEE Access"},{"key":"ref_38","unstructured":"Jung, H., Lee, S., Park, S., Lee, I., Ahn, C., and Kim, J. (2015). Deep temporal appearance-geometry network for facial expression recognition. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ding, H., Zhou, S.K., and Chellappa, R. (June, January 30). Facenet2expnet: Regularizing a deep face recognition net for expression recognition. Proceedings of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), Washington, DC, USA.","DOI":"10.1109\/FG.2017.23"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"e2364","DOI":"10.7717\/peerj.2364","article-title":"Emotion recognition using Kinect motion capture data of human gaits","volume":"4","author":"Li","year":"2016","journal-title":"PeerJ"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1002\/cav.1714","article-title":"Continuous body emotion recognition system during theater performances","volume":"27","author":"Senecal","year":"2016","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Glowinski, D., Camurri, A., Volpe, G., Dael, N., and Scherer, K. (2008, January 23\u201328). Technique for automatic emotion recognition by body gesture analysis. Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Anchorage, AK, USA.","DOI":"10.1109\/CVPRW.2008.4563173"},{"key":"ref_43","unstructured":"Piana, S., Stagliano, A., Odone, F., Verri, A., and Camurri, A. (2014). Real-time automatic emotion recognition from body gestures. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.jvcir.2015.06.013","article-title":"A survey of human pose estimation: The body parts parsing based methods","volume":"32","author":"Liu","year":"2015","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1080\/10798587.2015.1095419","article-title":"A survey on human pose estimation","volume":"22","author":"Zhang","year":"2016","journal-title":"Intell. Autom. Soft Comput."},{"key":"ref_46","unstructured":"Zheng, C., Wu, W., Yang, T., Zhu, S., Chen, C., Liu, R., Shen, J., Kehtarnavaz, N., and Shah, M. (2020). Deep Learning-Based Human Pose Estimation: A Survey. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Aracena, C., Basterrech, S., Sn\u00e1el, V., and Vel\u00e1squez, J. (2015, January 9\u201312). Neural networks for emotion recognition based on eye tracking data. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.460"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wu, S., Du, Z., Li, W., Huang, D., and Wang, Y. (2019, January 14\u201318). Continuous emotion recognition in videos by fusing facial expression, head pose and eye gaze. Proceedings of the 2019 International Conference on Multimodal Interaction, Suzhou, China.","DOI":"10.1145\/3340555.3353739"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s10772-018-9491-z","article-title":"Databases, features and classifiers for speech emotion recognition: A review","volume":"21","author":"Swain","year":"2018","journal-title":"Int. J. Speech Technol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"117327","DOI":"10.1109\/ACCESS.2019.2936124","article-title":"Speech emotion recognition using deep learning techniques: A review","volume":"7","author":"Khalil","year":"2019","journal-title":"IEEE Access"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Chandrasekar, P., Chapaneri, S., and Jayaswal, D. (2014, January 4\u20135). Automatic speech emotion recognition: A survey. Proceedings of the 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA), Mumbai, India.","DOI":"10.1109\/CSCITA.2014.6839284"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1016\/j.patcog.2010.09.020","article-title":"Survey on speech emotion recognition: Features, classification schemes, and databases","volume":"44","author":"Kamel","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"23745","DOI":"10.1007\/s11042-020-09874-7","article-title":"Deep learning approaches for speech emotion recognition: State of the art and research challenges","volume":"80","author":"Jahangir","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Lin, Y.S., and Lee, C.C. (2017, January 20\u201324). Deriving Dyad-Level Interaction Representation Using Interlocutors Structural and Expressive Multimodal Behavior Features. Proceedings of the INTERSPEECH, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-569"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Lee, C.C., Busso, C., Lee, S., and Narayanan, S.S. (2009, January 6\u201310). Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions. Proceedings of the Tenth Annual Conference of the International Speech Communication Association, Brighton, UK.","DOI":"10.21437\/Interspeech.2009-480"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Yeh, S.L., Lin, Y.S., and Lee, C.C. (2019, January 4\u20138). An interaction-aware attention network for speech emotion recognition in spoken dialogs. Proceedings of the ICASSP 2019\u20132019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP.2019.8683293"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Castellano, G., Kessous, L., and Caridakis, G. (2008). Emotion recognition through multiple modalities: Face, body gesture, speech. Affect and Emotion in Human-Computer Interaction, Springer.","DOI":"10.1007\/978-3-540-85099-1_8"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"115831","DOI":"10.1016\/j.image.2020.115831","article-title":"Human emotion recognition by optimally fusing facial expression and speech feature","volume":"84","author":"Wang","year":"2020","journal-title":"Signal Process. Image Commun."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.knosys.2018.07.041","article-title":"Multimodal sentiment analysis using hierarchical fusion with context modeling","volume":"161","author":"Majumder","year":"2018","journal-title":"Knowl. Based Syst."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Pan, Z., Luo, Z., Yang, J., and Li, H. (2020). Multi-modal attention for speech emotion recognition. arXiv.","DOI":"10.21437\/Interspeech.2020-1653"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/s10579-008-9076-6","article-title":"IEMOCAP: Interactive emotional dyadic motion capture database","volume":"42","author":"Busso","year":"2008","journal-title":"Lang. Resour. Eval."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1017\/S0954579405050340","article-title":"The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology","volume":"17","author":"Posner","year":"2005","journal-title":"Dev. Psychopathol."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Wiem, M.B.H., and Lachiri, Z. (2017). Emotion classification in arousal valence model using MAHNOB-HCI database. Int. J. Adv. Comput. Sci. Appl., 8.","DOI":"10.14569\/IJACSA.2017.080344"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"LeCun, Y., Haffner, P., Bottou, L., and Bengio, Y. (1999). Object recognition with gradient-based learning. Shape, Contour and Grouping in Computer Vision, Springer.","DOI":"10.1007\/3-540-46805-6_19"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Krishnan, D., Taylor, G.W., and Fergus, R. (2010, January 13\u201318). Deconvolutional networks. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Lea, C., Flynn, M.D., Vidal, R., Reiter, A., and Hager, G.D. (2017, January 21\u201326). Temporal convolutional networks for action segmentation and detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.113"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1016\/j.physleta.2014.12.036","article-title":"Analysis of detrended time-lagged cross-correlation between two nonstationary time series","volume":"379","author":"Shen","year":"2015","journal-title":"Phys. Lett. A"},{"key":"ref_69","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv."},{"key":"ref_70","unstructured":"Ishida, T., Yamane, I., Sakai, T., Niu, G., and Sugiyama, M. (2020). Do We Need Zero Training Loss After Achieving Zero Training Error?. arXiv."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Farooq, M., Hussain, F., Baloch, N.K., Raja, F.R., Yu, H., and Zikria, Y.B. (2020). Impact of Feature Selection Algorithm on Speech Emotion Recognition Using Deep Convolutional Neural Network. Sensors, 20.","DOI":"10.3390\/s20216008"},{"key":"ref_72","unstructured":"Haq, S., Jackson, P.J., and Edge, J. (2009, January 10\u201313). Speaker-dependent audio-visual emotion recognition. Proceedings of the AVSP, Norwich, UK."},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Ando, S., and Huang, C.Y. (2017, January 18\u201312). Deep over-sampling framework for classifying imbalanced data. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Skopje, North Macedonia.","DOI":"10.1007\/978-3-319-71249-9_46"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","article-title":"A systematic study of the class imbalance problem in convolutional neural networks","volume":"106","author":"Buda","year":"2018","journal-title":"Neural Netw."},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Pouyanfar, S., Tao, Y., Mohan, A., Tian, H., Kaseb, A.S., Gauen, K., Dailey, R., Aghajanzadeh, S., Lu, Y.H., and Chen, S.C. (2018, January 10\u201312). Dynamic sampling in convolutional neural networks for imbalanced data classification. Proceedings of the 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), Miami, FL, USA.","DOI":"10.1109\/MIPR.2018.00027"},{"key":"ref_76","unstructured":"Cao, K., Wei, C., Gaidon, A., Arechiga, N., and Ma, T. (2019). Learning imbalanced datasets with label-distribution-aware margin loss. arXiv."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1109\/TPAMI.2018.2832629","article-title":"Imbalanced deep learning by minority class incremental rectification","volume":"41","author":"Dong","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Khan, S., Hayat, M., Zamir, S.W., Shen, J., and Shao, L. (2019, January 15\u201320). Striking the right balance with uncertainty. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00019"},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"9776","DOI":"10.3390\/ijerph110909776","article-title":"Resampling methods improve the predictive power of modeling in class-imbalanced datasets","volume":"11","author":"Lee","year":"2014","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Zeng, H., Li, X., Borghini, G., Zhao, Y., Aric\u00f2, P., Di Flumeri, G., Sciaraffa, N., Zakaria, W., Kong, W., and Babiloni, F. (2021). An EEG-Based Transfer Learning Method for Cross-Subject Fatigue Mental State Prediction. Sensors, 21.","DOI":"10.3390\/s21072369"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5317\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:41:37Z","timestamp":1760164897000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5317"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,6]]},"references-count":81,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21165317"],"URL":"https:\/\/doi.org\/10.3390\/s21165317","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,6]]}}}