{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T18:36:17Z","timestamp":1772303777437,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,10,8]],"date-time":"2022-10-08T00:00:00Z","timestamp":1665187200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>There is high demand for techniques to estimate human mental workload during some activities for productivity enhancement or accident prevention. Most studies focus on a single physiological sensing modality and use univariate methods to analyse multi-channel electroencephalography (EEG) data. This paper proposes a new framework that relies on the features of hybrid EEG\u2013functional near-infrared spectroscopy (EEG\u2013fNIRS), supported by machine-learning features to deal with multi-level mental workload classification. Furthermore, instead of the well-used univariate power spectral density (PSD) for EEG recording, we propose using bivariate functional brain connectivity (FBC) features in the time and frequency domains of three bands: delta (0.5\u20134 Hz), theta (4\u20137 Hz) and alpha (8\u201315 Hz). With the assistance of the fNIRS oxyhemoglobin and deoxyhemoglobin (HbO and HbR) indicators, the FBC technique significantly improved classification performance at a 77% accuracy for 0-back vs. 2-back and 83% for 0-back vs. 3-back using a public dataset. Moreover, topographic and heat-map visualisation indicated that the distinguishing regions for EEG and fNIRS showed a difference among the 0-back, 2-back and 3-back test results. It was determined that the best region to assist the discrimination of the mental workload for EEG and fNIRS is different. Specifically, the posterior area performed the best for the posterior midline occipital (POz) EEG in the alpha band and fNIRS had superiority in the right frontal region (AF8).<\/jats:p>","DOI":"10.3390\/s22197623","type":"journal-article","created":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T05:12:21Z","timestamp":1665378741000},"page":"7623","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["EEG\/fNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2121-7631","authenticated-orcid":false,"given":"Jun","family":"Cao","sequence":"first","affiliation":[{"name":"School of Aerospace, Transport and Manufacturing, Cranfield University, Bedfordshire MK43 0AL, UK"}]},{"given":"Enara Martin","family":"Garro","sequence":"additional","affiliation":[{"name":"School of Aerospace, Transport and Manufacturing, Cranfield University, Bedfordshire MK43 0AL, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2383-5724","authenticated-orcid":false,"given":"Yifan","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Aerospace, Transport and Manufacturing, Cranfield University, Bedfordshire MK43 0AL, UK"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1111\/j.1467-8659.2011.01928.x","article-title":"A user study of visualization effectiveness using EEG and cognitive load","volume":"30","author":"Anderson","year":"2011","journal-title":"Comput. Graph. Forum"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bashivan, P., Yeasin, M., and Bidelman, G.M. (2016, January 1). Single trial prediction of normal and excessive cognitive load through EEG feature fusion. Proceedings of the 2015 IEEE Signal Processing in Medicine and Biology Symposium, Philadelphia, PA, USA.","DOI":"10.1109\/SPMB.2015.7405422"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1007\/s10111-017-0430-6","article-title":"A deep learning scheme for mental workload classification based on restricted Boltzmann machines","volume":"19","author":"Zhang","year":"2017","journal-title":"Cogn. Technol. Work"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hussain, I., and Park, S.J. (2021). Quantitative evaluation of task-induced neurological outcome after stroke. Brain Sci., 11.","DOI":"10.3390\/brainsci11070900"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hussain, I., Hossain, A., Jany, R., Bari, A., Uddin, M., Raihan, A., Kamal, M., Ku, Y., and Kim, J. (2022). Quantitative Evaluation of EEG-Biomarkers for Prediction of Sleep Stages. Sensors, 22.","DOI":"10.3390\/s22083079"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hussain, I., Young, S., Kim, C.H., Benjamin, H.C.M., and Park, S.J. (2021). Quantifying physiological biomarkers of a microwave brain stimulation device. Sensors, 21.","DOI":"10.3390\/s21051896"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3824","DOI":"10.1109\/JBHI.2021.3085131","article-title":"EEG Fingerprints of Task-Independent Mental Workload Discrimination","volume":"25","author":"Kakkos","year":"2021","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"641","DOI":"10.3389\/fnhum.2017.00641","article-title":"Signal processing in functional near-infrared spectroscopy (fNIRS): Methodological differences lead to different statistical results","volume":"11","author":"Pfeifer","year":"2018","journal-title":"Front. Hum. Neurosci."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kuanar, S., Athitsos, V., Pradhan, N., Mishra, A., and Rao, K.R. (2018, January 15\u201320). Cognitive Analysis of Working Memory Load from Eeg, by a Deep Recurrent Neural Network. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8462243"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"359","DOI":"10.3389\/fnhum.2017.00359","article-title":"Measuring mental workload with EEG+fNIRS","volume":"11","author":"Aghajani","year":"2017","journal-title":"Front. Hum. Neurosci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"So, W.K.Y., Wong, S.W.H., Mak, J.N., and Chan, R.H.M. (2017). An evaluation of mental workload with frontal EEG. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0174949"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Maimon, N.B., Molcho, L., Intrator, N., and Lamy, D. (2020). Single-channel EEG features during n-back task correlate with working memory load. arXiv.","DOI":"10.1109\/LifeTech52111.2021.9391963"},{"key":"ref_13","first-page":"114","article-title":"EEG-based workload estimation across affective contexts","volume":"8","author":"Jeunet","year":"2014","journal-title":"Front. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"860","DOI":"10.1002\/hbm.25683","article-title":"Brain functional and effective connectivity based on electroencephalography recordings: A review","volume":"43","author":"Cao","year":"2021","journal-title":"Hum. Brain Mapp."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cao, J., Grajcar, K., Shan, X., Zhao, Y., Zou, J., Chen, L., Li, Z., Grunewald, R., Zis, P., and De Marco, M. (2021). Using interictal seizure-free EEG data to recognise patients with epilepsy based on machine learning of brain functional connectivity. Biomed. Signal Process. Control, 67.","DOI":"10.1016\/j.bspc.2021.102554"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1109\/TNSRE.2021.3076311","article-title":"Tracking non-stationary association of two electroencephalography signals using a Revised Hilbert-Huang Transformation","volume":"29","author":"Shan","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng. Shan"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1109\/TMI.2019.2953584","article-title":"Imaging of Nonlinear and Dynamic Functional Brain Connectivity Based on EEG Recordings with the Application on the Diagnosis of Alzheimer\u2019s Disease","volume":"39","author":"Zhao","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5815\/ijigsp.2021.05.01","article-title":"Classification of FNIRS Using Wigner-ville Distribution and CNN","volume":"13","author":"Zaman","year":"2021","journal-title":"Int. J. Image, Graph. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.7717\/peerj-cs.537","article-title":"An effective classification framework for brain-computer interface system design based on combining of fNIRS and EEG signals","volume":"7","author":"Alhudhaif","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1111\/nyas.13948","article-title":"The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience","volume":"1464","author":"Pinti","year":"2020","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Herold, F., Wiegel, P., Scholkmann, F., and M\u00fcller, N.G. (2018). Applications of functional near-infrared spectroscopy (fNIRS) neuroimaging in exercise--cognition science: A systematic, methodology-focused review. J. Clin. Med., 7.","DOI":"10.3390\/jcm7120466"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"935","DOI":"10.3389\/fnhum.2013.00935","article-title":"Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS","volume":"7","author":"Herff","year":"2014","journal-title":"Front. Hum. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.biopsycho.2016.10.002","article-title":"Neural and psychophysiological correlates of human performance under stress and high mental workload","volume":"121","author":"Mandrick","year":"2016","journal-title":"Biol. Psychol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Saadati, M., Nelson, J., and Ayaz, H. (2019, January 13\u201316). Mental Workload Classification from Spatial Representation of FNIRS Recordings Using Convolutional Neural Networks. Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing, Pittsburgh, PA, USA.","DOI":"10.1109\/MLSP.2019.8918861"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2367","DOI":"10.1109\/TNSRE.2020.3026991","article-title":"A unified analytical framework with multiple fNIRS features for mental workload assessment in the prefrontal cortex","volume":"28","author":"Lim","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Putze, F., Herff, C., Tremmel, C., Schultz, T., and Krusienski, D.J. (2019, January 23\u201327). Decoding Mental Workload in Virtual Environments: A fNIRS Study using an Immersive n-back Task. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8856386"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"143250","DOI":"10.1109\/ACCESS.2019.2944965","article-title":"Assessment and Classification of Mental Workload in the Prefrontal Cortex (PFC) Using Fixed-Value Modified Beer-Lambert Law","volume":"7","author":"Asgher","year":"2019","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1080\/2326263X.2017.1304020","article-title":"Mental workload classification with concurrent electroencephalography and functional near-infrared spectroscopy","volume":"4","author":"Liu","year":"2016","journal-title":"Brain-Comput. Interfaces"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/978-3-030-20473-0_21","article-title":"Multimodal fNIRS-EEG classification using deep learning algorithms for brain-computer interfaces purposes","volume":"953","author":"Saadati","year":"2020","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/sdata.2018.3","article-title":"Simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset","volume":"5","author":"Shin","year":"2018","journal-title":"Sci. Data"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"198","DOI":"10.3389\/fnins.2010.00198","article-title":"The Berlin brain--computer interface: Non-medical uses of BCI technology","volume":"4","author":"Blankertz","year":"2010","journal-title":"Front. Neurosci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"244","DOI":"10.3389\/fnhum.2014.00244","article-title":"Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface","volume":"8","author":"Khan","year":"2014","journal-title":"Front. Hum. Neurosci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"045008","DOI":"10.1088\/1741-2560\/9\/4\/045008","article-title":"Estimating workload using EEG spectral power and ERPs in the n-back task","volume":"9","author":"Brouwer","year":"2012","journal-title":"J. Neural Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1016\/S0167-2789(00)00087-7","article-title":"Mean phase coherence as a measure for phase synchronization and its application to the EEG of epilepsy patients","volume":"144","author":"Mormann","year":"2000","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"031005","DOI":"10.1088\/1741-2552\/aab2f2","article-title":"A review of classification algorithms for EEG-based brain-computer interfaces: A 10 year update","volume":"15","author":"Lotte","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"16230","DOI":"10.1038\/srep16230","article-title":"Interval analysis of interictal EEG: Pathology of the alpha rhythm in focal epilepsy","volume":"5","author":"Pyrzowski","year":"2015","journal-title":"Sci. Rep."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1016\/j.clinph.2016.10.002","article-title":"Functional and effective brain connectivity for discrimination between Alzheimer\u2019s patients and healthy individuals: A study on resting state EEG rhythms","volume":"128","author":"Blinowska","year":"2017","journal-title":"Clin. Neurophysiol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"43933","DOI":"10.1038\/srep43933","article-title":"Utilization of a combined EEG\/NIRS system to predict driver drowsiness","volume":"7","author":"Nguyen","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chu, H., Cao, Y., Jiang, J., Yang, J., Huang, M., Li, Q., Jiang, C., and Jiao, X. (2022, August 30). Optimized EEG-fNIRS Based Mental Workload Detection Method for Practical Applications. Available online: https:\/\/assets.researchsquare.com\/files\/rs-683529\/v1_covered.pdf?c=1631874746.","DOI":"10.21203\/rs.3.rs-683529\/v1"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"103613","DOI":"10.1016\/j.bandc.2019.103613","article-title":"Assessing the brain \u2018on the line\u2019: An ecologically-valid assessment of the impact of repetitive assembly line work on hemodynamic response and fine motor control using fNIRS","volume":"136","author":"Han","year":"2019","journal-title":"Brain Cogn."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"102580","DOI":"10.1016\/j.ijhcs.2020.102580","article-title":"Measuring Mental Workload Variations in Office Work Tasks using fNIRS","volume":"147","author":"Midha","year":"2021","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"393","DOI":"10.3389\/fnhum.2019.00393","article-title":"Toward a Hybrid Passive BCI for the Modulation of Sustained Attention Using EEG and fNIRS","volume":"13","author":"Karran","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_43","first-page":"1","article-title":"EEG-Based Multiclass Workload Identification Using Feature Fusion and Selection","volume":"70","author":"Pei","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7623\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:48:14Z","timestamp":1760143694000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7623"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,8]]},"references-count":43,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197623"],"URL":"https:\/\/doi.org\/10.3390\/s22197623","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,8]]}}}