{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T11:03:05Z","timestamp":1781953385620,"version":"3.54.5"},"reference-count":54,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,9,30]],"date-time":"2021-09-30T00:00:00Z","timestamp":1632960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019YFA0706200"],"award-info":[{"award-number":["2019YFA0706200"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62072219"],"award-info":[{"award-number":["62072219"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61632014"],"award-info":[{"award-number":["61632014"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Basic Research Program of China","award":["2014CB744600"],"award-info":[{"award-number":["2014CB744600"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>With the increasing pressure of current life, fatigue caused by high-pressure work has deeply affected people and even threatened their lives. In particular, fatigue driving has become a leading cause of traffic accidents and deaths. This paper investigates electroencephalography (EEG)-based fatigue detection for driving by mining the latent information through the spatial-temporal changes in the relations between EEG channels. First, EEG data are partitioned into several segments to calculate the covariance matrices of each segment, and then we feed these matrices into a recurrent neural network to obtain high-level temporal information. Second, the covariance matrices of whole signals are leveraged to extract two kinds of spatial features, which will be fused with temporal characteristics to obtain comprehensive spatial-temporal information. Experiments on an open benchmark showed that our method achieved an excellent classification accuracy of 93.834% and performed better than several novel methods. These experimental results indicate that our method enables better reliability and feasibility in the detection of fatigued driving.<\/jats:p>","DOI":"10.3390\/e23101298","type":"journal-article","created":{"date-parts":[[2021,10,1]],"date-time":"2021-10-01T10:55:40Z","timestamp":1633085740000},"page":"1298","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Fatigue Detection with Spatial-Temporal Fusion Method on Covariance Manifolds of Electroencephalography"],"prefix":"10.3390","volume":"23","author":[{"given":"Nan","family":"Zhao","sequence":"first","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dawei","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Optics and Electronics, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kechen","family":"Hou","sequence":"additional","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meifei","family":"Chen","sequence":"additional","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangyu","family":"Wei","sequence":"additional","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8562-416X","authenticated-orcid":false,"given":"Xiaowei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3514-5413","authenticated-orcid":false,"given":"Bin","family":"Hu","sequence":"additional","affiliation":[{"name":"Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"},{"name":"CAS Center for Excellence in Brain Science and Institutes for Biological Sciences, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai 200031, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1001\/archneur.1989.00520460115022","article-title":"The fatigue severity scale: Application to patients with multiple sclerosis and systemic lupus erythematosus","volume":"46","author":"Krupp","year":"1989","journal-title":"Arch. Neurol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/0091-7435(86)90037-X","article-title":"The epidemiology of self-perceived fatigue among adults","volume":"15","author":"Chen","year":"1986","journal-title":"Prev. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1136\/jech.46.2.92","article-title":"The epidemiology of fatigue: More questions than answers","volume":"46","author":"Lewis","year":"1992","journal-title":"J. Epidemiol. Community Health"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2213","DOI":"10.1001\/archinte.1988.00380100077017","article-title":"The mental health of patients with a chief complaint of chronic fatigue: A prospective evaluation and follow-up","volume":"148","author":"Manu","year":"1988","journal-title":"Arch. Intern. Med."},{"key":"ref_5","first-page":"104","article-title":"Relationship between fatigue driving and traffic accident","volume":"10","author":"Li","year":"2010","journal-title":"J. Traffic Transp. Eng. (Xi\u2019an Shaanxi)"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/S1087-0792(03)90004-9","article-title":"The hazards and prevention of driving while sleepy","volume":"7","author":"MacLean","year":"2003","journal-title":"Sleep Med. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4386","DOI":"10.1109\/TCYB.2020.2987575","article-title":"Emotion Recognition From Multimodal Physiological Signals Using a Regularized Deep Fusion of Kernel Machine","volume":"51","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1109\/TII.2018.2868431","article-title":"Brain network analysis of compressive sensed high-density EEG signals in AD and MCI subjects","volume":"15","author":"Mammone","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"11761","DOI":"10.1109\/ACCESS.2019.2963113","article-title":"Emotion recognition from body movement","volume":"8","author":"Ahmed","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","unstructured":"Gu, H., Ji, Q., and Zhu, Z. (2002, January 4). Active facial tracking for fatigue detection. Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, Orlando, FL, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zeng, H., Li, X., Borghini, G., Zhao, Y., Aric\u00f2, P., Di Flumeri, G., Sciaraffa, N., Zakaria, W., Kong, W., and Babiloni, F. (2021). An EEG-Based Transfer Learning Method for Cross-Subject Fatigue Mental State Prediction. Sensors, 21.","DOI":"10.3390\/s21072369"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hu, J., and Wang, P. (2017). Noise robustness analysis of performance for EEG-based driver fatigue detection using different entropy feature sets. Entropy, 19.","DOI":"10.3390\/e19080385"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Han, C., Sun, X., Yang, Y., Che, Y., and Qin, Y. (2019). Brain complex network characteristic analysis of fatigue during simulated driving based on electroencephalogram signals. Entropy, 21.","DOI":"10.3390\/e21040353"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"046002","DOI":"10.1088\/1741-2552\/aa6c6f","article-title":"Effective brain network analysis with resting-state EEG data: A comparison between heroin abstinent and non-addicted subjects","volume":"14","author":"Hu","year":"2017","journal-title":"J. Neural Eng."},{"key":"ref_15","unstructured":"Zhang, X., Pan, J., Shen, J., Din, Z.U., Li, J., Lu, D., Wu, M., and Hu, B. (2020). Fusing of Electroencephalogram and Eye Movement with Group Sparse Canonical Correlation Analysis for Anxiety Detection. IEEE Trans. Affect. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mishra, P.K., Jagadish, B., Kiran, M.P.R.S., Rajalakshmi, P., and Reddy, D.S. (2018, January 17\u201320). A novel classification for EEG based four class motor imagery using kullback-leibler regularized Riemannian manifold. Proceedings of the 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom), Ostrava, Czech Republic.","DOI":"10.1109\/HealthCom.2018.8531086"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.neucom.2012.12.039","article-title":"Classification of covariance matrices using a Riemannian-based kernel for BCI applications","volume":"112","author":"Barachant","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1080\/2326263X.2017.1297192","article-title":"Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review","volume":"4","author":"Congedo","year":"2017","journal-title":"Brain-Comput. Interfaces"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Huang, Z., and Van Gool, L. (2017, January 4\u20139). A riemannian network for spd matrix learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10866"},{"key":"ref_20","unstructured":"Jozefowicz, R., Zaremba, W., and Sutskever, I. (2015, January 7\u20139). An empirical exploration of recurrent network architectures. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sundermeyer, M., Schl\u00fcter, R., and Ney, H. (2012, January 9\u201313). LSTM neural networks for language modeling. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.","DOI":"10.21437\/Interspeech.2012-65"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/j.neucom.2018.08.067","article-title":"LSTM-based traffic flow prediction with missing data","volume":"318","author":"Tian","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.ins.2021.04.023","article-title":"Augmented skeleton based contrastive action learning with momentum lstm for unsupervised action recognition","volume":"569","author":"Rao","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xu, S., Rao, H., Peng, H., Jiang, X., Guo, Y., Hu, X., and Hu, B. (2020). Attention based multi-level co-occurrence graph convolutional lstm for 3d action recognition. IEEE Internet Things J., 1.","DOI":"10.1109\/JIOT.2020.3042986"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Miah, A.S.M., Rahim, M.A., and Shin, J. (2020). Motor-imagery classification using riemannian geometry with median absolute deviation. Electronics, 9.","DOI":"10.3390\/electronics9101584"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Park, Y., and Chung, W. (2019, January 18\u201320). Optimal channel selection using covariance matrix and cross-combining region in EEG-based BCI. Proceedings of the 2019 7th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Korea.","DOI":"10.1109\/IWW-BCI.2019.8737257"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"170","DOI":"10.3389\/fnins.2017.00170","article-title":"Multiple kernel based region importance learning for neural classification of gait states from EEG signals","volume":"11","author":"Zhang","year":"2017","journal-title":"Front. Neurosci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1109\/TNSRE.2019.2905894","article-title":"EEG signal processing in MI-BCI applications with improved covariance matrix estimators","volume":"27","author":"Olias","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1137\/050637996","article-title":"Geometric means in a novel vector space structure on symmetric positive-definite matrices","volume":"29","author":"Arsigny","year":"2007","journal-title":"SIAM J. Matrix Anal. Appl."},{"key":"ref_30","unstructured":"Sra, S. A new metric on the manifold of kernel matrices with application to matrix geometric means. Proceedings of the 25th International Conference on Neural Information Processing Systems\u2014Volume 1."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3497","DOI":"10.1109\/TII.2020.3020694","article-title":"Fatigue Detection With Covariance Manifolds of Electroencephalography in Transportation Industry","volume":"17","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1038\/nn.4504","article-title":"Magnetoencephalography for brain electrophysiology and imaging","volume":"20","author":"Baillet","year":"2017","journal-title":"Nat. Neurosci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"116893","DOI":"10.1016\/j.neuroimage.2020.116893","article-title":"Predictive regression modeling with MEG\/EEG: From source power to signals and cognitive states","volume":"222","author":"Sabbagh","year":"2020","journal-title":"NeuroImage"},{"key":"ref_34","unstructured":"Du, Y., Wang, W., and Wang, L. (2015, January 7\u201312). Hierarchical recurrent neural network for skeleton based action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., and Xie, X. (2016, January 12\u201317). Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10451"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2330","DOI":"10.1109\/TMM.2018.2802648","article-title":"Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks","volume":"20","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhang, X., Lu, D., Shen, J., Gao, J., Huang, X., and Wu, M. (2020, January 16\u201319). Spatial-temporal joint optimization network on covariance manifolds of electroencephalography for fatigue detection. Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Seoul, Korea.","DOI":"10.1109\/BIBM49941.2020.9313257"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-019-0027-4","article-title":"Multi-channel EEG recordings during a sustained-attention driving task","volume":"6","author":"Cao","year":"2019","journal-title":"Sci. Data"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Huang, R.S., Jung, T.P., and Makeig, S. (2009, January 19\u201324). Tonic changes in EEG power spectra during simulated driving. Proceedings of the International Conference on Foundations of Augmented Cognition, San Diego, CA, USA.","DOI":"10.1007\/978-3-642-02812-0_47"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2044","DOI":"10.1109\/TCSI.2012.2185290","article-title":"Generalized EEG-based drowsiness prediction system by using a self-organizing neural fuzzy system","volume":"59","author":"Lin","year":"2012","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1114","DOI":"10.1109\/TBME.2019.2930186","article-title":"Evaluation of artifact subspace reconstruction for automatic artifact components removal in multi-channel EEG recordings","volume":"67","author":"Chang","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1109\/TNSRE.2015.2415520","article-title":"EEG-based attention tracking during distracted driving","volume":"23","author":"Wang","year":"2015","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"046059","DOI":"10.1088\/1741-2552\/abfa71","article-title":"A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation","volume":"18","author":"Yang","year":"2021","journal-title":"J. Neural Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2263","DOI":"10.1109\/TNSRE.2019.2945794","article-title":"EEG-based driver drowsiness estimation using feature weighted episodic training","volume":"27","author":"Cui","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.knosys.2015.01.007","article-title":"An EEG-based perceptual function integration network for application to drowsy driving","volume":"80","author":"Chuang","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1016\/S0013-4694(97)00147-8","article-title":"Volume conduction effects in EEG and MEG","volume":"106","author":"Reinders","year":"1998","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Hajinoroozi, M., Zhang, J.M., and Huang, Y. (2017, January 5\u20138). Driver\u2019s fatigue prediction by deep covariance learning from EEG. Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada.","DOI":"10.1109\/SMC.2017.8122609"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Saha, P., Fels, S., and Abdul-Mageed, M. (2019, January 12\u201317). Deep learning the eeg manifold for phonological categorization from active thoughts. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682330"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Duma, G.M., Mento, G., Manari, T., Martinelli, M., and Tressoldi, P. (2017). Driving with intuition: A preregistered study about the EEG anticipation of simulated random car accidents. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0170370"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"066028","DOI":"10.1088\/1741-2560\/12\/6\/066028","article-title":"EEG-based decoding of error-related brain activity in a real-world driving task","volume":"12","author":"Zhang","year":"2015","journal-title":"J. Neural Eng."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Kim, J.W., Kim, I.H., and Lee, S.W. (2015, January 9\u201312). Detection of braking intention during simulated driving based on EEG analysis: Online study. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.163"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Ary, J.P., Klein, S.A., and Fender, D.H. (1981). Location of sources of evoked scalp potentials: Corrections for skull and scalp thicknesses. IEEE Trans. Biomed. Eng., 447\u2013452.","DOI":"10.1109\/TBME.1981.324817"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.neuroimage.2013.10.067","article-title":"On the interpretation of weight vectors of linear models in multivariate neuroimaging","volume":"87","author":"Haufe","year":"2014","journal-title":"Neuroimage"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1093\/biomet\/83.4.891","article-title":"A forerunner of the t-distribution","volume":"83","author":"Statistics","year":"1996","journal-title":"Biometrika"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1298\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:08:18Z","timestamp":1760166498000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1298"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,30]]},"references-count":54,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["e23101298"],"URL":"https:\/\/doi.org\/10.3390\/e23101298","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,30]]}}}