{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T20:17:04Z","timestamp":1776975424015,"version":"3.51.4"},"reference-count":84,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,5,6]],"date-time":"2024-05-06T00:00:00Z","timestamp":1714953600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,6]],"date-time":"2024-05-06T00:00:00Z","timestamp":1714953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","award":["2020.06024.BD"],"award-info":[{"award-number":["2020.06024.BD"]}]},{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","award":["PD\/BDE\/150304\/2019"],"award-info":[{"award-number":["PD\/BDE\/150304\/2019"]}]},{"DOI":"10.13039\/100016218","name":"Ag\u00eancia Regional para o Desenvolvimento da Investiga\u00e7\u00e3o, Tecnologia e Inova\u00e7\u00e3o","doi-asserted-by":"publisher","award":["MAC2\/1.1b\/352"],"award-info":[{"award-number":["MAC2\/1.1b\/352"]}],"id":[{"id":"10.13039\/100016218","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021680","name":"NOVA Laboratory for Computer Science and Informatics","doi-asserted-by":"publisher","award":["UIDB\/04516\/2020"],"award-info":[{"award-number":["UIDB\/04516\/2020"]}],"id":[{"id":"10.13039\/501100021680","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016218","name":"Ag\u00eancia Regional para o Desenvolvimento da Investiga\u00e7\u00e3o, Tecnologia e Inova\u00e7\u00e3o","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016218","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Virtual Reality"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Affective computing has been widely used to detect and recognize emotional states. The main goal of this study was to detect emotional states using machine learning algorithms automatically. The experimental procedure involved eliciting emotional states using film clips in an immersive and non-immersive virtual reality setup. The participants\u2019 physiological signals were recorded and analyzed to train machine learning models to recognize users\u2019 emotional states. Furthermore, two subjective ratings emotional scales were provided to rate each emotional film clip. Results showed no significant differences between presenting the stimuli in the two degrees of immersion. Regarding emotion classification, it emerged that for both physiological signals and subjective ratings, user-dependent models have a better performance when compared to user-independent models. We obtained an average accuracy of 69.29 \u00b1 11.41% and 71.00 \u00b1 7.95% for the subjective ratings and physiological signals, respectively. On the other hand, using user-independent models, the accuracy we obtained was 54.0 \u00b1 17.2% and 24.9 \u00b1 4.0%, respectively. We interpreted these data as the result of high inter-subject variability among participants, suggesting the need for user-dependent classification models. In future works, we intend to develop new classification algorithms and transfer them to real-time implementation. This will make it possible to adapt to a virtual reality environment in real-time, according to the user\u2019s emotional state.<\/jats:p>","DOI":"10.1007\/s10055-024-00989-y","type":"journal-article","created":{"date-parts":[[2024,5,6]],"date-time":"2024-05-06T07:02:49Z","timestamp":1714978969000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Multimodal emotion classification using machine learning in immersive and non-immersive virtual reality"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4030-9526","authenticated-orcid":false,"given":"Rodrigo","family":"Lima","sequence":"first","affiliation":[]},{"given":"Alice","family":"Chirico","sequence":"additional","affiliation":[]},{"given":"Rui","family":"Varandas","sequence":"additional","affiliation":[]},{"given":"Hugo","family":"Gamboa","sequence":"additional","affiliation":[]},{"given":"Andrea","family":"Gaggioli","sequence":"additional","affiliation":[]},{"given":"Sergi Berm\u00fadez","family":"i Badia","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,5,6]]},"reference":[{"key":"989_CR1","doi-asserted-by":"crossref","unstructured":"Achlioptas P, Ovsjanikov M, Haydarov K, et\u00a0al (2021) ArtEmis: affective language for visual art. CoRR arXiv:2101.07396","DOI":"10.1109\/CVPR46437.2021.01140"},{"issue":"3","key":"989_CR2","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1016\/j.biopsycho.2006.09.001","volume":"74","author":"T Aue","year":"2007","unstructured":"Aue T, Flykt A, Scherer KR (2007) First evidence for differential and sequential efferent effects of stimulus relevance and goal conduciveness appraisal. Biol Psychol 74(3):347\u2013357. https:\/\/doi.org\/10.1016\/j.biopsycho.2006.09.001","journal-title":"Biol Psychol"},{"issue":"2","key":"989_CR3","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1097\/YCO.0b013e3283503669","volume":"25","author":"M Berking","year":"2012","unstructured":"Berking M, Wupperman P (2012) Emotion regulation and mental health: Recent findings, current challenges, and future directions. Curr Opin Psychiatry 25(2):128\u2013134. https:\/\/doi.org\/10.1097\/YCO.0b013e3283503669","journal-title":"Curr Opin Psychiatry"},{"key":"989_CR4","doi-asserted-by":"publisher","DOI":"10.3390\/s22176528","author":"A Bernardes","year":"2022","unstructured":"Bernardes A, Couceiro R, Medeiros J et al (2022) How reliable are ultra-short-term HRV measurements during cognitively demanding tasks? Sensors. https:\/\/doi.org\/10.3390\/s22176528","journal-title":"Sensors"},{"key":"989_CR5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0148037","author":"A Betella","year":"2016","unstructured":"Betella A, Verschure PF (2016) The affective slider: a digital self-assessment scale for the measurement of human emotions. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0148037","journal-title":"PLoS ONE"},{"key":"989_CR6","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-016-0174-1","author":"A Borrego","year":"2016","unstructured":"Borrego A, Latorre J, Llorens R et al (2016) Feasibility of a walking virtual reality system for rehabilitation: Objective and subjective parameters. J NeuroEng Rehabil. https:\/\/doi.org\/10.1186\/s12984-016-0174-1","journal-title":"J NeuroEng Rehabil"},{"key":"989_CR7","doi-asserted-by":"publisher","first-page":"140990","DOI":"10.1109\/ACCESS.2019.2944001","volume":"7","author":"PJ Bota","year":"2019","unstructured":"Bota PJ, Wang C, Fred AL et al (2019) A review, current challenges, and future possibilities on emotion recognition using machine learning and physiological signals. IEEE Access 7:140990\u2013141020. https:\/\/doi.org\/10.1109\/ACCESS.2019.2944001","journal-title":"IEEE Access"},{"key":"989_CR8","doi-asserted-by":"publisher","unstructured":"Boucsein W (2012) Electodermal activity, Second edn. https:\/\/doi.org\/10.1007\/978-1-4614-1126-0","DOI":"10.1007\/978-1-4614-1126-0"},{"issue":"1","key":"989_CR9","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/0005-7916(94)90063-9","volume":"25","author":"MM Bradley","year":"1994","unstructured":"Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Therapy Exp Psychiatry 25(1):49\u201359. https:\/\/doi.org\/10.1016\/0005-7916(94)90063-9","journal-title":"J Behav Therapy Exp Psychiatry"},{"issue":"1","key":"989_CR10","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1037\/0003-066x.45.1.16","volume":"45","author":"JT Cacioppo","year":"1990","unstructured":"Cacioppo JT, Tassinary LG (1990) Inferring psychological significance from physiological signals. Am Psychol 45(1):16\u201328. https:\/\/doi.org\/10.1037\/0003-066x.45.1.16","journal-title":"Am Psychol"},{"issue":"4","key":"989_CR11","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/s10484-012-9201-6","volume":"37","author":"S Carvalho","year":"2012","unstructured":"Carvalho S, Leite J, Galdo-\u00c1lvarez S et al (2012) The emotional movie database (EMDB): a self-report and psychophysiological study. Appl Psychophysiol Biofeedback 37(4):279\u2013294. https:\/\/doi.org\/10.1007\/s10484-012-9201-6","journal-title":"Appl Psychophysiol Biofeedback"},{"key":"989_CR12","doi-asserted-by":"publisher","unstructured":"Chanel G, Ansari-Asl K, Pun T (2007) Valence-arousal evaluation using physiological signals in an emotion recall paradigm. In: Conference proceedings - IEEE international conference on systems, man and cybernetics 41(22):2662\u20132667. https:\/\/doi.org\/10.1109\/ICSMC.2007.4413638","DOI":"10.1109\/ICSMC.2007.4413638"},{"key":"989_CR13","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-020-00327-4","author":"RC Chen","year":"2020","unstructured":"Chen RC, Dewi C, Huang SW et al (2020) Selecting critical features for data classification based on machine learning methods. J Big Data. https:\/\/doi.org\/10.1186\/s40537-020-00327-4","journal-title":"J Big Data"},{"issue":"4","key":"989_CR14","doi-asserted-by":"publisher","first-page":"283","DOI":"10.3969\/j.issn.1671-7104.2020.04.001","volume":"44","author":"S Chen","year":"2020","unstructured":"Chen S, Zhang L, Jiang F et al (2020) Emotion recognition based on multiple physiological signals. Zhongguo yi liao qi xie za zhi Chin J Med Instrum 44(4):283\u2013287. https:\/\/doi.org\/10.3969\/j.issn.1671-7104.2020.04.001","journal-title":"Zhongguo yi liao qi xie za zhi Chin J Med Instrum"},{"key":"989_CR15","unstructured":"Kothe C, Medine D, Boulay C, et\u00a0al (2019) LabStreamingLayer. https:\/\/github.com\/sccn\/labstreaminglayer"},{"issue":"3","key":"989_CR16","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/S0167-8760(96)00067-0","volume":"24","author":"HJ Crawford","year":"1996","unstructured":"Crawford HJ, Clarke SW, Kitner-Triolo M (1996) Self-generated happy and sad emotions in low and highly hypnotizable persons during waking and hypnosis: Laterality and regional EEG activity differences. Int J Psychophysiol 24(3):239\u2013266. https:\/\/doi.org\/10.1016\/S0167-8760(96)00067-0","journal-title":"Int J Psychophysiol"},{"issue":"2","key":"989_CR17","doi-asserted-by":"publisher","first-page":"468","DOI":"10.3758\/s13428-011-0064-1","volume":"43","author":"ES Dan-Glauser","year":"2011","unstructured":"Dan-Glauser ES, Scherer KR (2011) The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance. Behav Res Methods 43(2):468\u2013477. https:\/\/doi.org\/10.3758\/s13428-011-0064-1","journal-title":"Behav Res Methods"},{"issue":"6","key":"989_CR18","doi-asserted-by":"publisher","first-page":"823","DOI":"10.1037\/a0016909","volume":"135","author":"TF Denson","year":"2009","unstructured":"Denson TF, Spanovic M, Miller N (2009) Cognitive appraisals and emotions predict cortisol and immune responses: a meta-analysis of acute laboratory social stressors and emotion inductions. Psychol Bull 135(6):823\u2013853. https:\/\/doi.org\/10.1037\/a0016909","journal-title":"Psychol Bull"},{"key":"989_CR19","unstructured":"Donges N (2018) The random forest algorithm. https:\/\/machinelearning-blog.com\/2018\/02\/06\/the-random-forest-algorithm\/https:\/\/towardsdatascience.com\/the-random-forest-algorithm-d457d499ffcd"},{"issue":"May","key":"989_CR20","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.entcs.2019.04.009","volume":"343","author":"M Egger","year":"2019","unstructured":"Egger M, Ley M, Hanke S (2019) Emotion Recognition from Physiological Signal Analysis: A Review. Electron Notes Theor Comput Sci 343(May):35\u201355. https:\/\/doi.org\/10.1016\/j.entcs.2019.04.009","journal-title":"Electron Notes Theor Comput Sci"},{"issue":"3\u20134","key":"989_CR21","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1080\/02699939208411068","volume":"6","author":"P Ekman","year":"1992","unstructured":"Ekman P (1992) An argument for basic emotions. Cogn Emot 6(3\u20134):169\u2013200. https:\/\/doi.org\/10.1080\/02699939208411068","journal-title":"Cogn Emot"},{"key":"989_CR22","doi-asserted-by":"publisher","first-page":"25431","DOI":"10.1109\/ACCESS.2023.3254134","volume":"11","author":"TT Finseth","year":"2023","unstructured":"Finseth TT, Dorneich MC, Vardeman S et al (2023) Real-time personalized physiologically based stress detection for hazardous operations. IEEE Access 11:25431\u201325454","journal-title":"IEEE Access"},{"key":"989_CR23","doi-asserted-by":"publisher","unstructured":"Garcia-Garcia JM, Penichet VM, Lozano MD (2017) Emotion detection: a technology review. In: ACM international conference proceeding series part F1311(October). https:\/\/doi.org\/10.1145\/3123818.3123852","DOI":"10.1145\/3123818.3123852"},{"key":"989_CR24","doi-asserted-by":"publisher","unstructured":"Goncalves A, Borrego A, Latorre J, et\u00a0al (2021) Evaluation of a low-cost virtual reality surround-screen projection system. IEEE Trans Vis Comput Graph, PP. https:\/\/doi.org\/10.1109\/TVCG.2021.3091485","DOI":"10.1109\/TVCG.2021.3091485"},{"issue":"6\u20137","key":"989_CR25","doi-asserted-by":"publisher","first-page":"300","DOI":"10.3109\/03091902.2011.601784","volume":"35","author":"K Gouizi","year":"2011","unstructured":"Gouizi K, Bereksi Reguig F, Maaoui C (2011) Emotion recognition from physiological signals. J Med Eng Technol 35(6\u20137):300\u2013307. https:\/\/doi.org\/10.3109\/03091902.2011.601784","journal-title":"J Med Eng Technol"},{"key":"989_CR26","doi-asserted-by":"publisher","unstructured":"Gu Y, Tan SL, Wong KJ, et\u00a0al (2010) A biometric signature based system for improved emotion recognition using physiological responses from multiple subjects. In: 2010 8th IEEE international conference on industrial informatics, pp 61\u201366. https:\/\/doi.org\/10.1109\/INDIN.2010.5549464","DOI":"10.1109\/INDIN.2010.5549464"},{"issue":"1","key":"989_CR27","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1023\/A:1012487302797","volume":"46","author":"I Guyon","year":"2002","unstructured":"Guyon I, Weston J, Barnhill S et al (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46(1):389\u2013422. https:\/\/doi.org\/10.1023\/A:1012487302797","journal-title":"Mach Learn"},{"key":"989_CR28","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1007\/978-3-540-24842-2_4","volume-title":"Affective dialogue systems","author":"A Haag","year":"2004","unstructured":"Haag A, Goronzy S, Schaich P et al (2004) Emotion Recognition using bio-sensors: first steps towards an automatic system. In: Andr\u00e9 E, Dybkj\u00e6r L, Minker W et al (eds) Affective dialogue systems. Springer, Berlin, pp 36\u201348"},{"issue":"1","key":"989_CR29","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","volume":"143","author":"JA Hanley","year":"1982","unstructured":"Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1):29\u201336. https:\/\/doi.org\/10.1148\/radiology.143.1.7063747","journal-title":"Radiology"},{"key":"989_CR30","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2017.00026","author":"X Hu","year":"2017","unstructured":"Hu X, Yu J, Song M et al (2017) EEG correlates of ten positive emotions. Front Hum Neurosci. https:\/\/doi.org\/10.3389\/fnhum.2017.00026","journal-title":"Front Hum Neurosci"},{"issue":"2","key":"989_CR31","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1111\/aphw.12127","volume":"10","author":"E Inwood","year":"2018","unstructured":"Inwood E, Ferrari M (2018) Mechanisms of change in the relationship between self-compassion, emotion regulation, and mental health: a systematic review. Appl Psychol Health Well Being 10(2):215\u2013235. https:\/\/doi.org\/10.1111\/aphw.12127","journal-title":"Appl Psychol Health Well Being"},{"issue":"3","key":"989_CR32","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1109\/2.485891","volume":"29","author":"AK Jain","year":"1996","unstructured":"Jain AK, Mao J, Mohiuddin KM (1996) Artificial neural networks: a tutorial. Computer 29(3):31\u201344. https:\/\/doi.org\/10.1109\/2.485891","journal-title":"Computer"},{"key":"989_CR33","doi-asserted-by":"publisher","unstructured":"Jerritta S, Murugappan M, Nagarajan R et al (2011) Physiological signals based human emotion recognition: A review. In: Proceedings - 2011 IEEE 7th international colloquium on signal processing and its applications. CSPA 2011:410\u2013415. https:\/\/doi.org\/10.1109\/CSPA.2011.5759912","DOI":"10.1109\/CSPA.2011.5759912"},{"key":"989_CR34","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2018.00305","author":"J Juvrud","year":"2018","unstructured":"Juvrud J, Gredeb\u00e4ck G, \u00c5hs F et al (2018) The immersive virtual reality lab: possibilities for remote experimental manipulations of autonomic activity on a large scale. Front Neurosci. https:\/\/doi.org\/10.3389\/fnins.2018.00305","journal-title":"Front Neurosci"},{"issue":"12","key":"989_CR35","doi-asserted-by":"publisher","first-page":"2067","DOI":"10.1109\/TPAMI.2008.26","volume":"30","author":"J Kim","year":"2008","unstructured":"Kim J, Andr\u00e9 E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30(12):2067\u20132083. https:\/\/doi.org\/10.1109\/TPAMI.2008.26","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"989_CR36","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","volume":"3","author":"S Koelstra","year":"2012","unstructured":"Koelstra S, M\u00fchl C, Soleymani M et al (2012) DEAP: a database for emotion analysis; Using physiological signals. IEEE Trans Affect Comput 3(1):18\u201331. https:\/\/doi.org\/10.1109\/T-AFFC.2011.15","journal-title":"IEEE Trans Affect Comput"},{"key":"989_CR37","unstructured":"Lang PJ, Bradley MM, Cuthbert BN (2005) IAPS: Affective ratings of pictures and instruction manual. Emotion"},{"key":"989_CR38","doi-asserted-by":"publisher","unstructured":"Li L, Chen Jh (2006) Emotion Recognition Using Physiological Signals from Multiple Subjects. In: 2006 international conference on intelligent information hiding and multimedia, pp 355\u2013358. https:\/\/doi.org\/10.1109\/IIH-MSP.2006.265016","DOI":"10.1109\/IIH-MSP.2006.265016"},{"key":"989_CR39","doi-asserted-by":"publisher","unstructured":"Liapis A, Xenos M (2013) The physiological measurements as a critical indicator in users\u2019 experience evaluation. In: ACM international conference proceeding series. https:\/\/doi.org\/10.1145\/2491845.2491883","DOI":"10.1145\/2491845.2491883"},{"key":"989_CR40","doi-asserted-by":"publisher","unstructured":"Lindquist KA, Kober H, Bliss-Moreau E et al (2015) The brain basis of emotion: A meta-analytic review. Behav Brain Sci. 35(3):121\u2013143 https:\/\/doi.org\/10.1017\/S0140525X11000446.The, https:\/\/www.cambridge.org\/core\/product\/identifier\/S0140525X11000446\/type\/journal_article","DOI":"10.1017\/S0140525X11000446"},{"key":"989_CR41","doi-asserted-by":"publisher","unstructured":"Liu C, Rani P, Sarkar N (2005) An empirical study of machine learning techniques for affect recognition in human-robot interaction. In: 2005 IEEE\/RSJ international conference on intelligent robots and systems, pp 2662\u20132667. https:\/\/doi.org\/10.1109\/IROS.2005.1545344","DOI":"10.1109\/IROS.2005.1545344"},{"key":"989_CR42","doi-asserted-by":"publisher","unstructured":"Maaoui C, Pruski A (2010) Emotion recognition through physiological signals for human-machine communication. In: Kordic V (ed) Cutting edge robotics 2010. IntechOpen, Rijeka, chap\u00a020. https:\/\/doi.org\/10.5772\/10312,","DOI":"10.5772\/10312"},{"issue":"4","key":"989_CR43","doi-asserted-by":"publisher","first-page":"1689","DOI":"10.3758\/s13428-020-01516-y","volume":"53","author":"D Makowski","year":"2021","unstructured":"Makowski D, Pham T, Lau ZJ et al (2021) NeuroKit2: a Python toolbox for neurophysiological signal processing. Behav Res Methods 53(4):1689\u20131696. https:\/\/doi.org\/10.3758\/s13428-020-01516-y","journal-title":"Behav Res Methods"},{"key":"989_CR44","doi-asserted-by":"publisher","unstructured":"Mar\u00edn-Morales J, Llinares C, Guixeres J, et\u00a0al (2020) Emotion recognition in immersive virtual reality: from statistics to affective computing. Sensors 20(18). https:\/\/doi.org\/10.3390\/s20185163, https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5163","DOI":"10.3390\/s20185163"},{"issue":"3","key":"989_CR45","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1145\/566654.566630","volume":"21","author":"M Meehan","year":"2002","unstructured":"Meehan M, Insko B, Whitton M et al (2002) Physiological measures of presence in stressful virtual environments. ACM Trans. Graph. 21(3):645\u2013652. https:\/\/doi.org\/10.1145\/566654.566630","journal-title":"ACM Trans. Graph."},{"key":"989_CR46","unstructured":"Mehrabian A, Russell JA (1974) An approach to environmental psychology"},{"key":"989_CR47","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/9\/2\/026022","author":"S Moghimi","year":"2012","unstructured":"Moghimi S, Kushki A, Power S et al (2012) Automatic detection of a prefrontal cortical response to emotionally rated music using multi-channel near-infrared spectroscopy. J Neural Eng. https:\/\/doi.org\/10.1088\/1741-2560\/9\/2\/026022","journal-title":"J Neural Eng"},{"issue":"2","key":"989_CR48","doi-asserted-by":"publisher","first-page":"01","DOI":"10.5121\/ijdkp.2015.5201","volume":"5","author":"H Mohammad","year":"2015","unstructured":"Mohammad H, Nasir MdS (2015) A review on evaluation metrics for data classification evaluations. Int J Data Min Knowl Manage Process 5(2):01\u201311. https:\/\/doi.org\/10.5121\/ijdkp.2015.5201","journal-title":"Int J Data Min Knowl Manage Process"},{"issue":"Jan","key":"989_CR49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnhum.2015.00003","volume":"9","author":"N Naseer","year":"2015","unstructured":"Naseer N, Hong KS (2015) fNIRS-based brain-computer interfaces: a review. Front Human Neurosci 9(Jan):1\u201315. https:\/\/doi.org\/10.3389\/fnhum.2015.00003","journal-title":"Front Human Neurosci"},{"issue":"1","key":"989_CR50","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s10111-003-0143-x","volume":"6","author":"F Nasoz","year":"2004","unstructured":"Nasoz F, Alvarez K, Lisetti CL et al (2004) Emotion recognition from physiological signals using wireless sensors for presence technologies. Cogn Technol Work 6(1):4\u201314. https:\/\/doi.org\/10.1007\/s10111-003-0143-x","journal-title":"Cogn Technol Work"},{"issue":"3","key":"989_CR51","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1109\/TBME.1985.325532","volume":"32","author":"J Pan","year":"1985","unstructured":"Pan J, Tompkins W (1985) A real-time QRS detection algorithm. Biomed Eng IEEE Trans 32(3):230\u2013236","journal-title":"Biomed Eng IEEE Trans"},{"issue":"3","key":"989_CR52","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1049\/htl.2017.0090","volume":"5","author":"L Pecchia","year":"2018","unstructured":"Pecchia L, Castaldo R, Montesinos L et al (2018) Are ultra-short heart rate variability features good surrogates of short-term ones? State-of-the-art review and recommendations. Healthc Technol Lett 5(3):94\u2013100. https:\/\/doi.org\/10.1049\/htl.2017.0090","journal-title":"Healthc Technol Lett"},{"issue":"85","key":"989_CR53","doi-asserted-by":"publisher","first-page":"2825","DOI":"10.1289\/EHP4713","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12(85):2825\u20132830. https:\/\/doi.org\/10.1289\/EHP4713","journal-title":"J Mach Learn Res"},{"issue":"1","key":"989_CR54","doi-asserted-by":"publisher","first-page":"195","DOI":"10.3758\/s13428-018-01193-y","volume":"51","author":"J Peirce","year":"2019","unstructured":"Peirce J, Gray JR, Simpson S et al (2019) PsychoPy2: Experiments in behavior made easy. Behav Res Methods 51(1):195\u2013203. https:\/\/doi.org\/10.3758\/s13428-018-01193-y","journal-title":"Behav Res Methods"},{"issue":"4\u20135","key":"989_CR55","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1177\/053901882021004003","volume":"21","author":"R Plutchik","year":"1982","unstructured":"Plutchik R (1982) A psychoevolutionary theory of emotions. Soc Sci Inf 21(4\u20135):529\u2013553. https:\/\/doi.org\/10.1177\/053901882021004003","journal-title":"Soc Sci Inf"},{"key":"989_CR56","doi-asserted-by":"publisher","DOI":"10.3390\/s20020479","author":"HF Posada-Quintero","year":"2020","unstructured":"Posada-Quintero HF, Chon KH (2020) Innovations in electrodermal activity data collection and signal processing: a systematic review. Sensors (Switzerland). https:\/\/doi.org\/10.3390\/s20020479","journal-title":"Sensors (Switzerland)"},{"issue":"10","key":"989_CR57","doi-asserted-by":"publisher","first-page":"3124","DOI":"10.1007\/s10439-016-1606-6","volume":"44","author":"HF Posada-Quintero","year":"2016","unstructured":"Posada-Quintero HF, Florian JP, Orjuela-Ca\u00f1\u00f3n AD et al (2016) Power spectral density analysis of electrodermal activity for sympathetic function assessment. Ann Biomed Eng 44(10):3124\u20133135. https:\/\/doi.org\/10.1007\/s10439-016-1606-6","journal-title":"Ann Biomed Eng"},{"key":"989_CR58","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2352\/ISSN.2470-1173.2016.16.HVEI-129","volume":"2016","author":"N Ramzan","year":"2016","unstructured":"Ramzan N, Palke S, Cuntz T et al (2016) Emotion Recognition by Physiological Signals. Electronic Imaging 2016:1\u20136. https:\/\/doi.org\/10.2352\/ISSN.2470-1173.2016.16.HVEI-129","journal-title":"Electronic Imaging"},{"key":"989_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-1-4899-7993-3_565-2","volume-title":"Cross-validation","author":"P Refaeilzadeh","year":"2016","unstructured":"Refaeilzadeh P, Tang L, Liu H (2016) Cross-validation. Springer, New York, pp 1\u20137. https:\/\/doi.org\/10.1007\/978-1-4899-7993-3_565-2"},{"issue":"1","key":"989_CR60","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1111\/tops.12292","volume":"11","author":"R Reisenzein","year":"2019","unstructured":"Reisenzein R, Horstmann G, Sch\u00fctzwohl A (2019) The cognitive-evolutionary model of surprise: a review of the evidence. Top Cogn Sci 11(1):50\u201374. https:\/\/doi.org\/10.1111\/tops.12292","journal-title":"Top Cogn Sci"},{"issue":"1","key":"989_CR61","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1162\/pres.19.1.35","volume":"19","author":"Y Renard","year":"2010","unstructured":"Renard Y, Lotte F, Gibert G et al (2010) OpenViBE: an open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence Teleop Virt 19(1):35\u201353. https:\/\/doi.org\/10.1162\/pres.19.1.35","journal-title":"Presence Teleop Virt"},{"issue":"3","key":"989_CR62","doi-asserted-by":"publisher","first-page":"1699","DOI":"10.1016\/j.eswa.2014.10.006","volume":"42","author":"B Rey","year":"2014","unstructured":"Rey B, Clemente M, Wrzesien M et al (2014) Assessing brain activations associated with emotional regulation during virtual reality mood induction procedures. Expert Syst Appl 42(3):1699\u20131709. https:\/\/doi.org\/10.1016\/j.eswa.2014.10.006","journal-title":"Expert Syst Appl"},{"key":"989_CR63","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1007\/978-3-540-73078-1_36","volume-title":"User modeling 2007","author":"G Rigas","year":"2007","unstructured":"Rigas G, Katsis CD, Ganiatsas G et al (2007) A user independent, biosignal based, emotion recognition method. In: Conati C, McCoy K, Paliouras G (eds) User modeling 2007. Springer, Berlin, pp 314\u2013318"},{"key":"989_CR64","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.apergo.2018.01.009","volume":"69","author":"T Rose","year":"2018","unstructured":"Rose T, Nam CS, Chen KB (2018) Immersion of virtual reality for rehabilitation\u2014review. Appl Ergon 69:153\u2013161. https:\/\/doi.org\/10.1016\/j.apergo.2018.01.009","journal-title":"Appl Ergon"},{"issue":"6","key":"989_CR65","doi-asserted-by":"publisher","first-page":"1161","DOI":"10.1037\/h0077714","volume":"39","author":"JA Russell","year":"1980","unstructured":"Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 39(6):1161\u20131178. https:\/\/doi.org\/10.1037\/h0077714","journal-title":"J Pers Soc Psychol"},{"key":"989_CR66","doi-asserted-by":"publisher","unstructured":"Salahuddin L, Cho J, Jeong MG et al (2007) Ultra short term analysis of heart rate variability for monitoring mental stress in mobile settings. In: Annual international conference of the ieee engineering in medicine and biology society ieee engineering in medicine and biology society annual international conference 2007:4656\u20134659. https:\/\/doi.org\/10.1109\/IEMBS.2007.4353378","DOI":"10.1109\/IEMBS.2007.4353378"},{"key":"989_CR67","doi-asserted-by":"publisher","unstructured":"Samala RK, Chan HP, Hadjiiski L, et\u00a0al (2020) Hazards of data leakage in machine learning: a study on classification of breast cancer using deep neural networks. In: Medical imaging 2020: computer-aided diagnosis, AA(University of Michigan), AB(University of Michigan), AC(University of Michigan), AD(University of Michigan), p 1131416, https:\/\/doi.org\/10.1117\/12.2549313, https:\/\/ui.adsabs.harvard.edu\/abs\/2020SPIE11314E..16S","DOI":"10.1117\/12.2549313"},{"key":"989_CR68","doi-asserted-by":"crossref","unstructured":"Schaaff K, Adam MTP (2013) Measuring emotional arousal for online applications: Evaluation of ultra-short term heart rate variability measures. In: 2013 Humaine association conference on affective computing and intelligent interaction pp 362\u2013368","DOI":"10.1109\/ACII.2013.66"},{"issue":"7","key":"989_CR69","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1080\/02699930903274322","volume":"24","author":"A Schaefer","year":"2010","unstructured":"Schaefer A, Nils F, Philippot P et al (2010) Assessing the effectiveness of a large database of emotion-eliciting films: a new tool for emotion researchers. Cogn Emot 24(7):1153\u20131172. https:\/\/doi.org\/10.1080\/02699930903274322","journal-title":"Cogn Emot"},{"key":"989_CR70","doi-asserted-by":"crossref","unstructured":"Scherer KR, Schorr A, Johnstone T (2001) Appraisal processes in emotion: theory, methods, research. Series in affective science, Oxford University Press. https:\/\/books.google.pt\/books?id=IWLnBwAAQBAJ","DOI":"10.1093\/oso\/9780195130072.001.0001"},{"key":"989_CR71","doi-asserted-by":"publisher","DOI":"10.3390\/s19194079","author":"P Schmidt","year":"2019","unstructured":"Schmidt P, Reiss A, D\u00fcrichen R et al (2019) Wearable-based affect recognition-a review. Sensors (Basel, Switzerland). https:\/\/doi.org\/10.3390\/s19194079","journal-title":"Sensors (Basel, Switzerland)."},{"issue":"4","key":"989_CR72","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1016\/0305-0483(96)00010-2","volume":"24","author":"MS Shanker","year":"1996","unstructured":"Shanker MS, Hu MY, Hung MS (1996) Effect of data standardization on neural network training. Omega 24(4):385\u2013397. https:\/\/doi.org\/10.1016\/0305-0483(96)00010-2","journal-title":"Omega"},{"key":"989_CR73","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1007\/978-3-030-43192-1_45","volume":"49","author":"M Sharma","year":"2020","unstructured":"Sharma M, Mathew R (2020) Emotion recognition using physiological signals. Lecture Notes on Data Eng Commun Technol 49:389\u2013396. https:\/\/doi.org\/10.1007\/978-3-030-43192-1_45","journal-title":"Lecture Notes on Data Eng Commun Technol"},{"key":"989_CR74","doi-asserted-by":"publisher","first-page":"813","DOI":"10.1037\/0022-3514.48.4.813","volume":"48","author":"C Smith","year":"1985","unstructured":"Smith C, Ellsworth P (1985) Patterns of cognitive appraisal in emotion. J Pers Soc Psychol 48:813\u2013838. https:\/\/doi.org\/10.1037\/0022-3514.48.4.813","journal-title":"J Pers Soc Psychol"},{"issue":"3","key":"989_CR75","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1037\/0022-3514.56.3.339","volume":"56","author":"CA Smith","year":"1989","unstructured":"Smith CA (1989) Dimensions of appraisal and physiological response in emotion. J Pers Soc Psychol 56(3):339\u2013353. https:\/\/doi.org\/10.1037\/0022-3514.56.3.339","journal-title":"J Pers Soc Psychol"},{"issue":"1","key":"989_CR76","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1109\/T-AFFC.2011.25","volume":"3","author":"M Soleymani","year":"2012","unstructured":"Soleymani M, Lichtenauer J, Pun T et al (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42\u201355. https:\/\/doi.org\/10.1109\/T-AFFC.2011.25","journal-title":"IEEE Trans Affect Comput"},{"issue":"4","key":"989_CR77","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1515\/rehab-2015-0056","volume":"30","author":"A Stasie\u00f1ko","year":"2016","unstructured":"Stasie\u00f1ko A, Sarzy\u0144ska-D\u0142ugosz I (2016) Virtual reality in neurorehabilitation. Postepy Rehabil 30(4):67\u201375. https:\/\/doi.org\/10.1515\/rehab-2015-0056","journal-title":"Postepy Rehabil"},{"issue":"5","key":"989_CR78","doi-asserted-by":"publisher","first-page":"1043","DOI":"10.1161\/01.CIR.93.5.1043","volume":"93","author":"Task Force of the European Society of Cardiology","year":"1996","unstructured":"Task Force of the European Society of Cardiology (1996) Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task force of the European society of cardiology and the north American society of pacing and electrophysiology. Circulation 93(5):1043\u20131065","journal-title":"Circulation"},{"issue":"3","key":"989_CR79","doi-asserted-by":"publisher","first-page":"634","DOI":"10.3758\/s13428-013-0412-4","volume":"46","author":"J Trojan","year":"2014","unstructured":"Trojan J, Diers M, Fuchs X et al (2014) An augmented reality home-training system based on the mirror training and imagery approach. Behav Res Methods 46(3):634\u2013640. https:\/\/doi.org\/10.3758\/s13428-013-0412-4","journal-title":"Behav Res Methods"},{"key":"989_CR80","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2016.00180","author":"MK Uhrig","year":"2016","unstructured":"Uhrig MK, Trautmann N, Baumg\u00e4rtner U et al (2016) Emotion elicitation: a comparison of pictures and films. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2016.00180","journal-title":"Front Psychol"},{"key":"989_CR81","doi-asserted-by":"publisher","unstructured":"Vapnik V (1998) The support vector method of function estimation. In: Nonlinear modeling. p 55\u201385. https:\/\/doi.org\/10.1007\/978-1-4615-5703-6_3","DOI":"10.1007\/978-1-4615-5703-6_3"},{"key":"989_CR82","doi-asserted-by":"publisher","unstructured":"Varandas R, Lima R, Berm\u00fadez I Badia S, et\u00a0al (2022) Automatic cognitive fatigue detection using wearable fNIRS and machine learning. Sensors.https:\/\/doi.org\/10.3390\/s22114010. https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4010","DOI":"10.3390\/s22114010"},{"key":"989_CR83","doi-asserted-by":"publisher","unstructured":"Wioleta S (2013) Using physiological signals for emotion recognition. In: 2013 6th international conference on human system interactions, HSI 2013 pp 556\u2013561. https:\/\/doi.org\/10.1109\/HSI.2013.6577880","DOI":"10.1109\/HSI.2013.6577880"},{"issue":"9","key":"989_CR84","doi-asserted-by":"publisher","first-page":"2839","DOI":"10.1016\/j.patcog.2015.03.009","volume":"48","author":"TT Wong","year":"2015","unstructured":"Wong TT (2015) Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recogn 48(9):2839\u20132846. https:\/\/doi.org\/10.1016\/j.patcog.2015.03.009","journal-title":"Pattern Recogn"}],"container-title":["Virtual Reality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-024-00989-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10055-024-00989-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-024-00989-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T19:39:17Z","timestamp":1719517157000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10055-024-00989-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,6]]},"references-count":84,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["989"],"URL":"https:\/\/doi.org\/10.1007\/s10055-024-00989-y","relation":{},"ISSN":["1434-9957"],"issn-type":[{"value":"1434-9957","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,6]]},"assertion":[{"value":"30 June 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 March 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Authors Rui Varandas and Hugo Gamboa are affiliated with PLUX Wireless Biosignals S.A., the company that produces the biosignalsplux acquisition device used in this work and also the OpenSignals software. The remaining authors have no Conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This work was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of University of Madeira (approved on the 17 of February of 2022).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"A written informed consent was obtained from all the participants included in this work.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}}],"article-number":"107"}}