{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T14:03:44Z","timestamp":1762956224453,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T00:00:00Z","timestamp":1578268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["RTI2018-098085-B-C42"],"award-info":[{"award-number":["RTI2018-098085-B-C42"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Assessing emotional state is an emerging application field boosting research activities on the topic of analysis of non-invasive biosignals to find effective markers to accurately determine the emotional state in real-time. Nowadays using wearable sensors, electrocardiogram and thoracic impedance measurements can be recorded, facilitating analyzing cardiac and respiratory functions directly and autonomic nervous system function indirectly. Such analysis allows distinguishing between different emotional states: neutral, sadness, and disgust. This work was specifically focused on the proposal of a k-fold approach for selecting features while training the classifier that reduces the loss of generalization. The performance of the proposed algorithm used as the selection criterion was compared to the commonly used standard error function. The proposed k-fold approach outperforms the conventional method with 4% hit success rate improvement, reaching an accuracy near to 78%. Moreover, the proposed selection criterion method allows the classifier to produce the best performance using a lower number of features at lower computational cost. A reduced number of features reduces the risk of overfitting while a lower computational cost contributes to implementing real-time systems using wearable electronics.<\/jats:p>","DOI":"10.3390\/s20010309","type":"journal-article","created":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T10:34:46Z","timestamp":1578306886000},"page":"309","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A Wrapper Feature Selection Algorithm: An Emotional Assessment Using Physiological Recordings from Wearable Sensors"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5144-3472","authenticated-orcid":false,"given":"Inma","family":"Mohino-Herranz","sequence":"first","affiliation":[{"name":"Department of Signal Theory and Communications, University of Alcal\u00e1,  Alcal\u00e1 de Henares, 28805 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1790-3834","authenticated-orcid":false,"given":"Roberto","family":"Gil-Pita","sequence":"additional","affiliation":[{"name":"Department of Signal Theory and Communications, University of Alcal\u00e1,  Alcal\u00e1 de Henares, 28805 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0890-789X","authenticated-orcid":false,"given":"Joaqu\u00edn","family":"Garc\u00eda-G\u00f3mez","sequence":"additional","affiliation":[{"name":"Department of Signal Theory and Communications, University of Alcal\u00e1,  Alcal\u00e1 de Henares, 28805 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manuel","family":"Rosa-Zurera","sequence":"additional","affiliation":[{"name":"Department of Signal Theory and Communications, University of Alcal\u00e1,  Alcal\u00e1 de Henares, 28805 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6995-967X","authenticated-orcid":false,"given":"Fernando","family":"Seoane","sequence":"additional","affiliation":[{"name":"Institute for Clinical Science, Intervention and Technology, Karolinska Institutet, 17177 Solna Stockholm, Sweden"},{"name":"Department of Medical Care Technology, Karolinska University Hospital, 14157 Huddinge, Sweden"},{"name":"Textile Materials Technology, Department of Textile Technology, Faculty of Textiles, Engineering and Businees Swedish School of Textiles, University of Boras, 50190 Boras, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1126\/science.270.5236.644","article-title":"Central command neurons of the sympathetic nervous system: Basis of the fight-or-flight response","volume":"270","author":"Jansen","year":"1995","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1037\/0033-295X.107.3.411","article-title":"Biobehavioral responses to stress in females: Tend-and-befriend, not fight-or-flight","volume":"107","author":"Taylor","year":"2000","journal-title":"Psychol. Rev."},{"key":"ref_3","unstructured":"Cai, J., Liu, G., and Hao, M. (2009, January 25\u201326). The research on emotion recognition from ECG signal. Proceedings of the International Conference on Information Technology and Computer Science, Kiev, Ukraine."},{"key":"ref_4","first-page":"280","article-title":"The effect of emotions on electrocardiogram","volume":"1","author":"Dhillon","year":"2011","journal-title":"Acad. Res. Int."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lee, C., Yoo, S.K., Park, Y., Kim, N., Jeong, K., and Lee, B. (2006, January 17\u201318). Using neural network to recognize human emotions from heart rate variability and skin resistance. Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China.","DOI":"10.1109\/IEMBS.2005.1615734"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ma, C.-W., and Liu, G.-Y. (2009, January 6\u20137). Feature extraction, feature selection and classification from electrocardiography to emotions. Proceedings of the 2009 International Conference on Computational Intelligence and Natural Computing, Wuhan, China.","DOI":"10.1109\/CINC.2009.126"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1080\/026999300378905","article-title":"Heartbeat detection and the experience of emotions","volume":"14","author":"Wiens","year":"2000","journal-title":"Cogn. Emot."},{"key":"ref_8","unstructured":"Calhoun, C., and Solomon, R.C. (1984). What is an Emotion: Classic Readings in Philosophical Psychology, Oxford University Press."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1016\/j.neuroimage.2016.03.041","article-title":"Mindful attention to breath regulates emotions via increased amygdala\u2013prefrontal cortex connectivity","volume":"134","author":"Doll","year":"2016","journal-title":"Neuroimage"},{"key":"ref_10","first-page":"1011","article-title":"Breathing rhythms and emotions Experimental physiology","volume":"93","author":"Homma","year":"2008","journal-title":"Wiley Online Lib."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.envres.2014.01.001","article-title":"Thoracic Bioelectrical Impedance Assessment of the Hemodynamic Reactions to Recorded Road-Traffic Noise in Young Adults","volume":"129","author":"Paunovic","year":"2014","journal-title":"Environ. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7120","DOI":"10.3390\/s140407120","article-title":"Wearable biomedical measurement systems for assessment of mental stress of combatants in real time","volume":"14","author":"Seoane","year":"2014","journal-title":"Sensors"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/0301-0511(94)90024-8","article-title":"Effect of mental stress throughout the day on cardiac autonomic control","volume":"37","author":"Sloan","year":"1994","journal-title":"Biol. Psychol."},{"key":"ref_14","first-page":"4061","article-title":"Smart watch-based coaching with tiotropium and olodaterol ameliorates physical activity in patients with chronic obstructive pulmonary disease","volume":"14","author":"Hataji","year":"2017","journal-title":"Exp. Ther. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1538","DOI":"10.1109\/TBME.2014.2309951","article-title":"Unobtrusive sensing and wearable devices for health informatics","volume":"61","author":"Zheng","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.mser.2017.02.001","article-title":"Recent advances in wearable tactile sensors: Materials, sensing mechanisms, and device performance","volume":"115","author":"Yang","year":"2017","journal-title":"Mater. Sci. Eng. R Rep."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1109\/72.317727","article-title":"Weight smoothing to improve network generalization","volume":"5","author":"Jean","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1016\/j.apm.2006.04.014","article-title":"A mended hybrid learning algorithm for radial basis function neural networks to improve generalization capability","volume":"31","author":"Zhao","year":"2007","journal-title":"Appl. Math. Model."},{"key":"ref_19","unstructured":"Anguita, D., Ridella, S., and Rivieccio, F. (August, January 31). K-fold generalization capability assessment for support vector classifiers. Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"08TR02","DOI":"10.1088\/1361-6579\/aad57e","article-title":"Passive BCI beyond the lab: Current trends and future directions","volume":"39","author":"Borghini","year":"2018","journal-title":"Physiol. Meas."},{"key":"ref_21","first-page":"249","article-title":"Synthetical enlargement of mfcc based training sets for emotion recognition","volume":"6","year":"2014","journal-title":"Int. J. Comput. Sci. Inf. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0004-3702(97)00043-X","article-title":"Wrappers for feature subset selection","volume":"97","author":"Kohavi","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.artmed.2004.04.002","article-title":"Data mining and genetic algorithm based gene\/SNP selection","volume":"31","author":"Shah","year":"2004","journal-title":"Artif. Intell. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1111\/j.1439-0388.2011.00917.x","article-title":"Dimension reduction and variable selection for genomic selection: Application to predicting milk yield in Holsteins","volume":"128","author":"Long","year":"2011","journal-title":"J. Anim. Breed. Genet."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1016\/j.ejor.2004.09.010","article-title":"Using simulated annealing to optimize the feature selection problem in marketing applications","volume":"171","author":"Meiri","year":"2006","journal-title":"Eur. J. Oper. Res."},{"key":"ref_26","first-page":"90","article-title":"A genetic embedded approach for gene selection and classification of microarray data","volume":"7","author":"Hernandez","year":"2007","journal-title":"EvoBIO"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TITS.2005.848368","article-title":"Detecting stress during real-world driving tasks using physiological sensors","volume":"6","author":"Healey","year":"2005","journal-title":"Intell. Transp. Syst. IEEE Trans."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1016\/S0002-9149(99)80309-9","article-title":"The effects of emotions on short-term power spectrum analysis of heart rate variability","volume":"76","author":"McCraty","year":"1995","journal-title":"Am. J. Cardiol."},{"key":"ref_29","first-page":"825","article-title":"Power spectral analysis of heart rate variability in obese subjects: Evidence of decreased cardiac sympathetic responsiveness","volume":"20","author":"Piccirillo","year":"1996","journal-title":"Int. J. Obes. Relat. Metab. Disord. J. Int. Assoc. Study Obes."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"26","DOI":"10.3389\/fphys.2013.00026","article-title":"The LF\/HF ratio does not accurately measure cardiac sympatho-vagal balance","volume":"4","author":"Billman","year":"2013","journal-title":"Front. Physiol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"25607","DOI":"10.3390\/s151025607","article-title":"Assessment of mental, emotional and physical stress through analysis of physiological signals using smartphones","volume":"15","author":"Ferreira","year":"2015","journal-title":"Sensors"},{"key":"ref_32","unstructured":"Mehrnia, A., and Willson, A.N. (2004, January 23\u201326). On optimal IFIR filter design. Proceedings of the 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512), Vancouver, BC, Canada."},{"key":"ref_33","unstructured":"Davis, L. Handbook of Genetic Algorithms, Van Nostrand Reinhold."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mohino-Herranz, I., Gil-Pita, R., Rosa-Zurera, M., and Seoane, F. (2019). Activity Recognition Using Wearable Physiological Measurements: Selection of Features from a Comprehensive Literature Study. Sensors, 19.","DOI":"10.3390\/s19245524"},{"key":"ref_35","unstructured":"Prabhakar, V., Varde, P., and Michael, G. (2019). Risk-Based Engineering: An Integrated Approach to Complex Systems-special Reference to Nuclear Plants, Springer."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e2","DOI":"10.2196\/mental.5626","article-title":"Using Real-Time Social Media Technologies to Monitor Levels of Perceived Stress and Emotional State in College Students: A Web-Based Questionnaire Study","volume":"4","author":"Liu","year":"2017","journal-title":"JMIR Ment. Health"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1109\/TAFFC.2017.2660485","article-title":"Real-time movie-induced discrete emotion recognition from EEG signals","volume":"9","author":"Liu","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Turabzadeh, S., Meng, H., Swash, R.M., Pleva, M., and Juhar, J. (2017, January 16\u201318). Real-time emotional state detection from facial expression on embedded devices. Proceedings of the 2017 Seventh International Conference on Innovative Computing Technology (INTECH), Luton, UK.","DOI":"10.1109\/INTECH.2017.8102423"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Keogh, E., and Mueen, A. (2011). Curse of dimensionality. Encyclopedia of Machine Learning, Springer.","DOI":"10.1007\/978-0-387-30164-8_192"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhuo, L., Zheng, J., Li, X., Wang, F., Ai, B., and Qian, J. (2008, January 28\u201329). A genetic algorithm based wrapper feature selection method for classification of hyperspectral images using support vector machine. Proceedings of the Geoinformatics 2008 and Joint Conference on GIS and Built Environment: Geo-Simulation and Virtual GIS Environments, Guangzhou, China.","DOI":"10.1117\/12.813256"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1825","DOI":"10.1016\/j.patrec.2007.05.011","article-title":"A hybrid genetic algorithm for feature selection wrapper based on mutual information","volume":"28","author":"Huang","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_42","unstructured":"Kohavi, R., and Sommerfield, D. (1995, January 20\u201321). Feature Subset Selection Using theWrapper Method: Overfitting and Dynamic Search Space Topology. Proceedings of the First International Conference on Knowledge Discovery and Data Mining, Montreal, QC, Canada."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/309\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:28:53Z","timestamp":1760362133000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/309"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,6]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010309"],"URL":"https:\/\/doi.org\/10.3390\/s20010309","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,1,6]]}}}