{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T02:11:32Z","timestamp":1780366292437,"version":"3.54.1"},"reference-count":111,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,9]],"date-time":"2022-09-09T00:00:00Z","timestamp":1662681600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"AFOSR","award":["#FA9550-18-1-0465"],"award-info":[{"award-number":["#FA9550-18-1-0465"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 \u2213 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 \u2213 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.<\/jats:p>","DOI":"10.3390\/s22186834","type":"journal-article","created":{"date-parts":[[2022,9,13]],"date-time":"2022-09-13T04:05:41Z","timestamp":1663041941000},"page":"6834","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Investigating Methods for Cognitive Workload Estimation for Assistive Robots"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8157-2219","authenticated-orcid":false,"given":"Ayca","family":"Aygun","sequence":"first","affiliation":[{"name":"Department of Computer Science, Tufts University, Medford, MA 02155, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thuan","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tufts University, Medford, MA 02155, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zachary","family":"Haga","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tufts University, Medford, MA 02155, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuchin","family":"Aeron","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Tufts University, Medford, MA 02155, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0064-2789","authenticated-orcid":false,"given":"Matthias","family":"Scheutz","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tufts University, Medford, MA 02155, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1923","DOI":"10.3758\/s13423-021-01961-6","article-title":"Using Past and Present Indicators of Human Workload to Explain Variance in Human Performance","volume":"28","author":"Howard","year":"2021","journal-title":"Psychon. Bull. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1109\/THMS.2017.2782483","article-title":"A survey of workload assessment algorithms","volume":"48","author":"Heard","year":"2018","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_3","first-page":"B231","article-title":"EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks","volume":"78","author":"Berka","year":"2007","journal-title":"Aviat. Space Environ. Med."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"So, W.K., Wong, S.W., Mak, J.N., and Chan, R.H. (2017). An evaluation of mental workload with frontal EEG. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0174949"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/0001-6918(90)90067-P","article-title":"Eye movement indices of mental workload","volume":"75","author":"May","year":"1990","journal-title":"Acta Psychol."},{"key":"ref_6","unstructured":"Greef, T.D., Lafeber, H., Oostendorp, H.V., and Lindenberg, J. (2009, January 19\u201324). Eye movement as indicators of mental workload to trigger adaptive automation. Proceedings of the International Conference on Foundations of Augmented Cognition, San Diego, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"389","DOI":"10.3389\/fnhum.2017.00389","article-title":"Multisubject \u201clearning\u201d for mental workload classification using concurrent EEG, fNIRS, and physiological measures","volume":"11","author":"Liu","year":"2017","journal-title":"Front. Hum. Neurosci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Scheutz, M., Aeron, S., Aygun, A., de Ruiter, J., Fantini, S., Fernandez, C., Haga, Z., Nguyen, T., Lyu, B., and Rife, J. (2022). Estimating Individual Cognitive States from a Mixture of Physiological and Brain Signals. Trends Cogn. Sci., under review.","DOI":"10.1111\/tops.12669"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Aygun, A., Lyu, B., Nguyen, T., Haga, Z., Aeron, S., and Scheutz, M. (2022, January 7\u201311). Cognitive Workload Assessment via Eye Gaze and EEG in an Interactive Multi-Modal Driving Task. Proceedings of the 24th ACM International Conference on Multi-Modal Interaction, Bengaluru, India.","DOI":"10.1145\/3536221.3556610"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P., and Rao, R.P. (2008, January 5\u201310). Feasibility and pragmatics of classifying working memory load with an electroencephalograph. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy.","DOI":"10.1145\/1357054.1357187"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1207\/S15326985EP3801_8","article-title":"Cognitive load measurement as a means to advance cognitive load theory","volume":"38","author":"Paas","year":"2003","journal-title":"Educ. Psychol."},{"key":"ref_12","first-page":"272","article-title":"Ageing drivers\u2019 mental workload in real-time driving task based on subjective and objective measures","volume":"9","author":"Dawal","year":"2021","journal-title":"J. Eng. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research","volume":"52","author":"Hart","year":"1988","journal-title":"Adv. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0166-4115(08)62387-0","article-title":"The subjective workload assessment technique: A scaling procedure for measuring mental workload","volume":"Volume 52","author":"Reid","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1080\/00140139608964470","article-title":"Diagnosticity and multidimensional subjective workload ratings","volume":"39","author":"Tsang","year":"1996","journal-title":"Ergonomics"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tao, D., Tan, H., Wang, H., Zhang, X., Qu, X., and Zhang, T. (2019). A systematic review of physiological measures of mental workload. Int. J. Environ. Res. Public Health, 16.","DOI":"10.3390\/ijerph16152716"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1177\/0018720811400601","article-title":"Influence of task combination on EEG spectrum modulation for driver workload estimation","volume":"53","author":"Lei","year":"2011","journal-title":"Hum. Factors"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1016\/j.ergon.2005.04.005","article-title":"Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic","volume":"35","author":"Ryu","year":"2005","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Qu, H., Shan, Y., Liu, Y., Pang, L., Fan, Z., Zhang, J., and Wanyan, X. (2020). Mental workload classification method based on EEG independent component features. Appl. Sci., 10.","DOI":"10.3390\/app10093036"},{"key":"ref_20","first-page":"17","article-title":"Artifact removal from EEG signals","volume":"77","author":"Reddy","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jiang, X., Bian, G.B., and Tian, Z. (2019). Removal of artifacts from EEG signals: A review. Sensors, 19.","DOI":"10.3390\/s19050987"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"109494","DOI":"10.1016\/j.jneumeth.2022.109494","article-title":"Designing and comparing cleaning pipelines for TMS-EEG data: A theoretical overview and practical example","volume":"371","author":"Rogasch","year":"2022","journal-title":"J. Neurosci. Methods"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.jneumeth.2003.10.009","article-title":"EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis","volume":"134","author":"Delorme","year":"2004","journal-title":"J. Neurosci. Methods"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1016\/j.bspc.2011.07.004","article-title":"Real-time detection of workload changes using heart rate variability","volume":"7","author":"Hoover","year":"2012","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"565","DOI":"10.3389\/fphys.2019.00565","article-title":"Mental workload alters heart rate variability, lowering non-linear dynamics","volume":"10","author":"Delliaux","year":"2019","journal-title":"Front. Physiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.ergon.2018.02.015","article-title":"Analysis of the sensitivity of heart rate variability and subjective workload measures in a driving simulator: The case of highway work zones","volume":"66","author":"Shakouri","year":"2018","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.ijpsycho.2014.01.010","article-title":"Short-term cardiovascular measures for driver support: Increasing sensitivity for detecting changes in mental workload","volume":"92","author":"Stuiver","year":"2014","journal-title":"Int. J. Psychophysiol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/s00421-004-1055-z","article-title":"The effect of mental stress on heart rate variability and blood pressure during computer work","volume":"92","author":"Hjortskov","year":"2004","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1016\/j.ergon.2006.04.002","article-title":"Using eye movement activity as a correlate of cognitive workload","volume":"36","author":"Ahlstrom","year":"2006","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Palinko, O., Kun, A.L., Shyrokov, A., and Heeman, P. (2010, January 22\u201324). Estimating cognitive load using remote eye tracking in a driving simulator. Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, Austin, TX, USA.","DOI":"10.1145\/1743666.1743701"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Palinko, O., and Kun, A.L. (2012, January 28\u201330). Exploring the effects of visual cognitive load and illumination on pupil diameter in driving simulators. Proceedings of the Symposium on Eye Tracking Research and Applications, Santa Barbara, CA, USA.","DOI":"10.1145\/2168556.2168650"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1037\/0033-2909.91.2.276","article-title":"Task-evoked pupillary responses, processing load, and the structure of processing resources","volume":"91","author":"Beatty","year":"1982","journal-title":"Psychol. Bull."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Pfleging, B., Fekety, D.K., Schmidt, A., and Kun, A.L. (2016, January 7\u201312). A model relating pupil diameter to mental workload and lighting conditions. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2858036.2858117"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Das, S., Prudhvi, K., and Maiti, J. (2022). Assessing Mental Workload Using Eye Tracking Technology and Deep Learning Models. Handbook of Intelligent Computing and Optimization for Sustainable Development, Wiley Online Library.","DOI":"10.1002\/9781119792642.ch1"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"103193","DOI":"10.1016\/j.ergon.2021.103193","article-title":"The ability of eye-tracking metrics to classify and predict the perceived driving workload","volume":"86","author":"Bitkina","year":"2021","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Pang, L., Fan, Y., Deng, Y., Wang, X., and Wang, T. (2020, January 17\u201319). Mental Workload Classification By Eye Movements In Visual Search Tasks. Proceedings of the 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Chengdu, China.","DOI":"10.1109\/CISP-BMEI51763.2020.9263668"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kosch, T., Hassib, M., Buschek, D., and Schmidt, A. (2018, January 21\u201316). Look into my eyes: Using pupil dilation to estimate mental workload for task complexity adaptation. Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3170427.3188643"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Appel, T., Scharinger, C., Gerjets, P., and Kasneci, E. (2018, January 14\u201317). Cross-subject workload classification using pupil-related measures. Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, Warsaw, Poland.","DOI":"10.1145\/3204493.3204531"},{"key":"ref_39","unstructured":"Khedher, A.B., Jraidi, I., and Frasson, C. (2019, January 19\u201322). Predicting learners\u2019 performance using EEG and eye tracking features. Proceedings of the Thirty-Second International Flairs Conference, Sarasota, FL, USA."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MC.2015.314","article-title":"Combining EEG with pupillometry to improve cognitive workload detection","volume":"48","author":"Rozado","year":"2015","journal-title":"Computer"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.neuroimage.2011.07.091","article-title":"The effects of day-to-day variability of physiological data on operator functional state classification","volume":"59","author":"Christensen","year":"2012","journal-title":"NeuroImage"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"359","DOI":"10.3389\/fnhum.2017.00359","article-title":"Measuring mental workload with EEG+ fNIRS","volume":"11","author":"Aghajani","year":"2017","journal-title":"Front. Hum. Neurosci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1080\/2326263X.2017.1304020","article-title":"Mental workload classification with concurrent electroencephalography and functional near-infrared spectroscopy","volume":"4","author":"Liu","year":"2017","journal-title":"Brain-Comput. Interfaces"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Herff, C., Fortmann, O., Tse, C.Y., Cheng, X., Putze, F., Heger, D., and Schultz, T. (2015, January 22\u201324). Hybrid fNIRS-EEG based discrimination of 5 levels of memory load. Proceedings of the 2015 7th International IEEE\/EMBS Conference on Neural Engineering (NER), Montpellier, France.","DOI":"10.1109\/NER.2015.7146546"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Borys, M., Plechawska-W\u00f3jcik, M., Wawrzyk, M., and Weso\u0142owska, K. (2017, January 12\u201314). Classifying cognitive workload using eye activity and EEG features in arithmetic tasks. Proceedings of the International Conference on Information and Software Technologies, Druskininkai, Lithuania.","DOI":"10.1007\/978-3-319-67642-5_8"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1822","DOI":"10.1177\/1071181312561367","article-title":"Measuring workload using a combination of electroencephalography and near infrared spectroscopy","volume":"56","author":"Coffey","year":"2012","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1542","DOI":"10.1109\/TCYB.2019.2939399","article-title":"Multimodal fusion for objective assessment of cognitive workload: A review","volume":"51","author":"Debie","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1109\/TCDS.2016.2628702","article-title":"Quantifying cognitive workload in simulated flight using passive, dry EEG measurements","volume":"10","author":"Blanco","year":"2016","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Cheema, B.S., Samima, S., Sarma, M., and Samanta, D. (2018, January 15\u201320). Mental workload estimation from EEG signals using machine learning algorithms. Proceedings of the International Conference on Engineering Psychology and Cognitive Ergonomics, Las Vegas, NV, USA.","DOI":"10.1007\/978-3-319-91122-9_23"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kaczorowska, M., Plechawska-W\u00f3jcik, M., and Tokovarov, M. (2021). Interpretable machine learning models for three-way classification of cognitive workload levels for eye-tracking features. Brain Sci., 11.","DOI":"10.3390\/brainsci11020210"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1177\/1071181311551042","article-title":"Workload classification across subjects using EEG","volume":"55","author":"Hope","year":"2011","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Duraisingam, A., Palaniappan, R., and Andrews, S. (2017, January 3\u20134). Cognitive task difficulty analysis using EEG and data mining. Proceedings of the 2017 Conference on Emerging Devices and Smart Systems (ICEDSS), Mallasamudram, India.","DOI":"10.1109\/ICEDSS.2017.8073658"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Pandey, V., Choudhary, D.K., Verma, V., Sharma, G., Singh, R., and Chandra, S. (2020, January 26\u201327). Mental Workload Estimation Using EEG. Proceedings of the 2020 Fifth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Bangalore, India.","DOI":"10.1109\/ICRCICN50933.2020.9296150"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Almogbel, M.A., Dang, A.H., and Kameyama, W. (2019, January 17\u201320). Cognitive workload detection from raw EEG-signals of vehicle driver using deep learning. Proceedings of the 2019 21st International Conference on Advanced Communication Technology (ICACT), PyeongChang, Korea.","DOI":"10.23919\/ICACT.2019.8702048"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1940","DOI":"10.1109\/TNSRE.2017.2701002","article-title":"Task-independent mental workload classification based upon common multiband EEG cortical connectivity","volume":"25","author":"Dimitrakopoulos","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"14819","DOI":"10.1109\/ACCESS.2017.2731784","article-title":"An EEG-based cognitive load assessment in multimedia learning using feature extraction and partial directed coherence","volume":"5","author":"Mazher","year":"2017","journal-title":"IEEE Access"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"046020","DOI":"10.1088\/1741-2560\/12\/4\/046020","article-title":"Cognitive workload modulation through degraded visual stimuli: A single-trial EEG study","volume":"12","author":"Yu","year":"2015","journal-title":"J. Neural Eng."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Singh, U., and Ahirwal, M.K. (2021, January 17\u201319). Mental Workload Classification for Multitasking Test using Electroencephalogram Signal. Proceedings of the 2021 IEEE International Conference on Technology, Research, and Innovation for Betterment of Society (TRIBES), Raipur, India.","DOI":"10.1109\/TRIBES52498.2021.9751676"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.bspc.2016.11.013","article-title":"Cross-session classification of mental workload levels using EEG and an adaptive deep learning model","volume":"33","author":"Yin","year":"2017","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1109\/TAMD.2015.2441960","article-title":"Beyond Subjective Self-Rating: EEG Signal Classification of Cognitive Workload","volume":"7","author":"Zarjam","year":"2015","journal-title":"IEEE Trans. Auton. Ment. Dev."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.patrec.2017.05.020","article-title":"Deep long short-term memory structures model temporal dependencies improving cognitive workload estimation","volume":"94","author":"Hefron","year":"2017","journal-title":"Pattern Recognit. Lett."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Rahman, H., Ahmed, M.U., Barua, S., Funk, P., and Begum, S. (2021). Vision-based driver\u2019s cognitive load classification considering eye movement using machine learning and deep learning. Sensors, 21.","DOI":"10.3390\/s21238019"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.compbiomed.2019.04.034","article-title":"Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders","volume":"109","author":"Yang","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"103094","DOI":"10.1016\/j.bspc.2021.103094","article-title":"Recognition of driver\u2019s mental workload based on physiological signals, a comparative study","volume":"71","author":"Huang","year":"2022","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Islam, M.R., Barua, S., Ahmed, M.U., Begum, S., and Flumeri, G.D. (2019, January 14\u201315). Deep learning for automatic EEG feature extraction: An application in drivers\u2019 mental workload classification. Proceedings of the International Symposium on Human Mental Workload: Models and Applications, Rome, Italy.","DOI":"10.1007\/978-3-030-32423-0_8"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.apergo.2018.08.028","article-title":"Measuring mental workload using physiological measures: A systematic review","volume":"74","author":"Charles","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"2615","DOI":"10.1007\/s00221-020-05915-7","article-title":"Pupil diameter as a biomarker of effort in goal-directed gait","volume":"238","author":"Hassan","year":"2020","journal-title":"Exp. Brain Res."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1111\/aas.13129","article-title":"Pain assessment by pupil dilation reflex in response to noxious stimulation in anaesthetized adults","volume":"62","author":"Wildemeersch","year":"2018","journal-title":"Acta Anaesthesiol. Scand."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Smallwood, J., Brown, K.S., Tipper, C., Giesbrecht, B., Franklin, M.S., Mrazek, M.D., Carlson, J.M., and Schooler, J.W. (2011). Pupillometric evidence for the decoupling of attention from perceptual input during offline thought. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0018298"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Prieur-Coloma, Y., Reinoso-Leblanch, R.A., Mayeta-Revilla, L., Delisle-Rodr\u00edguez, D., Bastos, T., L\u00f3pez-Delis, A., Balart-Fern\u00e1ndez, L., and Falk, T.H. (2020, January 7\u20139). Enhancing shoulder pre-movements recognition through EEG Riemannian covariance matrices for a BCI-based exoskeleton. Proceedings of the 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy.","DOI":"10.1109\/ICHMS49158.2020.9209345"},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Yu, Y. (2020, January 25\u201327). A study on the classification of left-and righthanded eeg signals based on motor imagination. Proceedings of the 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), Dalian, China.","DOI":"10.1109\/AEECA49918.2020.9213510"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Li, R., and Principe, J.C. (September, January 30). Blinking artifact removal in cognitive EEG data using ICA. Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA.","DOI":"10.1109\/IEMBS.2006.260605"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_74","unstructured":"Cerliani, M. (2022, June 15). Tsmoothie. Available online: https:\/\/github.com\/cerlymarco\/tsmoothie."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"1404","DOI":"10.1088\/0967-3334\/37\/9\/1404","article-title":"Probabilistic model-based approach for heart beat detection","volume":"37","author":"Chen","year":"2016","journal-title":"Physiol. Meas."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"De Morais Borges, G., and Brusamarello, V. (2014, January 12\u201315). Bayesian fusion of multiple sensors for reliable heart rate detection. Proceedings of the 2014 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings, Montevideo, Uruguay.","DOI":"10.1109\/I2MTC.2014.6860957"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1088\/0967-3334\/37\/7\/1120","article-title":"Correlation between RR, inter-systolic and inter-diastolic intervals and their differences for the analysis of spontaneous heart rate variability","volume":"37","author":"Dias","year":"2016","journal-title":"Physiol. Meas."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-019-0134-9","article-title":"Real-world heart rate norms in the Health eHeart study","volume":"2","author":"Avram","year":"2019","journal-title":"NPJ Digit. Med."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Zhao, M., Gao, H., Wang, W., Qu, J., and Chen, L. (2020, January 20\u201322). Study on the identification of irritability emotion based on the percentage change in pupil size. Proceedings of the 2020 2nd International Conference on Image, Video and Signal Processing, Singapore.","DOI":"10.1145\/3388818.3389166"},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"012030","DOI":"10.1088\/1757-899X\/557\/1\/012030","article-title":"Analysis of EEG spectrum bands using power spectral density for pleasure and displeasure state","volume":"Volume 557","author":"Ameera","year":"2019","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"012032","DOI":"10.1088\/1757-899X\/557\/1\/012032","article-title":"PSD-based features extraction for EEG signal during typing task","volume":"Volume 557","author":"Ng","year":"2019","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Lim, W.L., Sourina, O., Liu, Y., and Wang, L. (2015, January 2\u20134). EEG-based mental workload recognition related to multitasking. Proceedings of the 2015 10th International Conference on Information, Communications and Signal Processing (ICICS), Singapore.","DOI":"10.1109\/ICICS.2015.7459834"},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.paid.2017.03.002","article-title":"Metrics for individual differences in EEG response to cognitive workload: Optimizing performance prediction","volume":"118","author":"Matthews","year":"2017","journal-title":"Personal. Individ. Differ."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"e14009","DOI":"10.1111\/psyp.14009","article-title":"EEG power spectral measures of cognitive workload: A meta-analysis","volume":"59","author":"Chikhi","year":"2022","journal-title":"Psychophysiology"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Foroozan, F., Mohan, M., and Wu, J.S. (2018, January 15\u201320). Robust beat-to-beat detection algorithm for pulse rate variability analysis from wrist photoplethysmography signals. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8462286"},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.cmpb.2013.07.024","article-title":"Kubios HRV\u2013heart rate variability analysis software","volume":"113","author":"Tarvainen","year":"2014","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1111\/jch.13304","article-title":"Blood pressure variability: Clinical relevance and application","volume":"20","author":"Parati","year":"2018","journal-title":"J. Clin. Hypertens."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"e6000","DOI":"10.1097\/MD.0000000000006000","article-title":"Association between beat-to-beat blood pressure variability and vascular elasticity in normal young adults during the cold pressor test","volume":"96","author":"Xia","year":"2017","journal-title":"Medicine"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1016\/j.jocn.2019.03.003","article-title":"Beat-to-beat blood pressure variability and heart rate variability in relation to autonomic dysregulation in patients with acute mild-moderate ischemic stroke","volume":"64","author":"Tian","year":"2019","journal-title":"J. Clin. Neurosci."},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Zawadka-Kunikowska, M., Rzepi\u0144ski, \u0141., Newton, J.L., Zalewski, P., and S\u0142omko, J. (2020). Cardiac Autonomic Modulation Is Different in Terms of Clinical Variant of Multiple Sclerosis. J. Clin. Med., 9.","DOI":"10.3390\/jcm9103176"},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Qin, X., Zheng, Y., and Chen, B. (2019, January 22\u201324). Extract EEG Features by Combining Power Spectral Density and Correntropy Spectral Density. Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China.","DOI":"10.1109\/CAC48633.2019.8996873"},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"012031","DOI":"10.1088\/1757-899X\/1077\/1\/012031","article-title":"Development of Unified Neuro-Affective Classification Tool (UNACT)","volume":"Volume 1077","author":"Hossain","year":"2021","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_93","doi-asserted-by":"crossref","first-page":"265","DOI":"10.3923\/jbs.2016.265.271","article-title":"Classification of EEG signals based on different motor movement using multi-layer Perceptron artificial neural network","volume":"16","author":"Hamzah","year":"2016","journal-title":"J. Biol. Sci."},{"key":"ref_94","first-page":"419","article-title":"Classification of human emotions from electroencephalogram (EEG) signal using deep neural network","volume":"8","author":"Hosny","year":"2017","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_95","unstructured":"Stoica, P., and Moses, R.L. (2005). Spectral Analysis of Signals, Pearson Prentice Hall."},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Rozado, D., Duenser, A., and Howell, B. (2015). Improving the performance of an EEG-based motor imagery brain computer interface using task evoked changes in pupil diameter. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0121262"},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Plechawska-W\u00f3jcik, M., and Borys, M. (2016, January 6\u20138). An analysis of EEG signal combined with pupillary response in the dynamics of human cognitive processing. Proceedings of the 2016 9th International Conference on Human System Interactions (HSI), Portsmouth, UK.","DOI":"10.1109\/HSI.2016.7529661"},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Lobo, J.L., Ser, J.D., De Simone, F., Presta, R., Collina, S., and Moravek, Z. (2016, January 14\u201316). Cognitive workload classification using eye-tracking and EEG data. Proceedings of the International Conference on Human-Computer Interaction in Aerospace, Paris, France.","DOI":"10.1145\/2950112.2964585"},{"key":"ref_99","first-page":"127","article-title":"Cross-validation: A review","volume":"9","author":"Stone","year":"1978","journal-title":"Stat. A J. Theor. Appl. Stat."},{"key":"ref_100","doi-asserted-by":"crossref","unstructured":"Saadati, M., Nelson, J., and Ayaz, H. (2019, January 13\u201316). Mental Workload Classification From Spatial Representation of FNIRS Recordings Using Convolutional Neural Networks. Proceedings of the 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), Pittsburgh, PA, USA.","DOI":"10.1109\/MLSP.2019.8918861"},{"key":"ref_101","doi-asserted-by":"crossref","unstructured":"Mughal, N.E., Khalil, K., and Khan, M.J. (2021, January 28\u201330). fNIRS Based Multi-Class Mental Workload Classification Using Recurrence Plots and CNN-LSTM. Proceedings of the 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), Bandung, Indonesia.","DOI":"10.1109\/AIMS52415.2021.9466084"},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: A compact convolutional neural network for EEG-based brain\u2013computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.neunet.2019.04.014","article-title":"Multivariate LSTM-FCNs for time series classification","volume":"116","author":"Karim","year":"2019","journal-title":"Neural Netw."},{"key":"ref_104","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1007\/s10618-020-00710-y","article-title":"Inceptiontime: Finding alexnet for time series classification","volume":"34","author":"Lucas","year":"2020","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_105","unstructured":"Agarap, A.F. (2018). Deep learning using rectified linear units (relu). arXiv."},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1007\/s10618-020-00727-3","article-title":"The great multivariate time series classification bake off: A review and experimental evaluation of recent algorithmic advances","volume":"35","author":"Ruiz","year":"2021","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_108","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0169-7439(89)80095-4","article-title":"Analysis of variance (ANOVA)","volume":"6","author":"St","year":"1989","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_110","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1037\/0033-2909.96.3.589","article-title":"Pairwise multiple comparison procedures: A review","volume":"96","author":"Jaccard","year":"1984","journal-title":"Psychol. Bull."},{"key":"ref_111","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: A practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. Ser. B"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6834\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:28:35Z","timestamp":1760142515000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6834"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,9]]},"references-count":111,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186834"],"URL":"https:\/\/doi.org\/10.3390\/s22186834","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,9]]}}}