{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T14:20:41Z","timestamp":1781619641093,"version":"3.54.5"},"reference-count":34,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T00:00:00Z","timestamp":1696982400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62102076"],"award-info":[{"award-number":["62102076"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["20220402033GH"],"award-info":[{"award-number":["20220402033GH"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Science and Technology Development Plan of Jilin Province, China","award":["62102076"],"award-info":[{"award-number":["62102076"]}]},{"name":"Science and Technology Development Plan of Jilin Province, China","award":["20220402033GH"],"award-info":[{"award-number":["20220402033GH"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A variety of technologies that could enhance driving safety are being actively explored, with the aim of reducing traffic accidents by accurately recognizing the driver\u2019s state. In this field, three mainstream detection methods have been widely applied, namely visual monitoring, physiological indicator monitoring and vehicle behavior analysis. In order to achieve more accurate driver state recognition, we adopted a multi-sensor fusion approach. We monitored driver physiological signals, electroencephalogram (EEG) signals and electrocardiogram (ECG) signals to determine fatigue state, while an in-vehicle camera observed driver behavior and provided more information for driver state assessment. In addition, an outside camera was used to monitor vehicle position to determine whether there were any driving deviations due to distraction or fatigue. After a series of experimental validations, our research results showed that our multi-sensor approach exhibited good performance for driver state recognition. This study could provide a solid foundation and development direction for future in-depth driver state recognition research, which is expected to further improve road safety.<\/jats:p>","DOI":"10.3390\/s23208386","type":"journal-article","created":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T08:18:57Z","timestamp":1697012337000},"page":"8386","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["EEG and ECG-Based Multi-Sensor Fusion Computing for Real-Time Fatigue Driving Recognition Based on Feedback Mechanism"],"prefix":"10.3390","volume":"23","author":[{"given":"Ling","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fangjie","family":"Song","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3258-0244","authenticated-orcid":false,"given":"Tie Hua","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiayu","family":"Hao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0394-9054","authenticated-orcid":false,"given":"Keun Ho","family":"Ryu","sequence":"additional","affiliation":[{"name":"Data Science Laboratory, Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam"},{"name":"Biomedical Engineering Institute, Chiang Mai University, Chiang Mai 50200, Thailand"},{"name":"Department of Computer Science, College of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhang, Z.Y. (2022, January 12\u201314). Study on the Influence of Driving Fatigue on Drivers\u2019 Effective Reaction Time. Proceedings of the 10th International Conference on Traffic and Logistic Engineering (ICTLE), Electr Network, Macau, China.","DOI":"10.1109\/ICTLE55577.2022.9902106"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"19999","DOI":"10.1109\/TITS.2022.3189346","article-title":"A Systematic Survey of Driving Fatigue Monitoring","volume":"23","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Akiduki, T., Nagasawa, J., Zhang, Z., Omae, Y., Arakawa, T., and Takahashi, H. (2022). Inattentive Driving Detection Using Body-Worn Sensors: Feasibility Study. Sensors, 22.","DOI":"10.3390\/s22010352"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2743","DOI":"10.1109\/TNSRE.2022.3208374","article-title":"Reliability of EEG Measures in Driving Fatigue","volume":"30","author":"Harvy","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1186\/s13640-021-00575-1","article-title":"Fatigue driving detection based on electrooculography: A review","volume":"2021","author":"Tian","year":"2021","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"106830","DOI":"10.1016\/j.aap.2022.106830","article-title":"Detecting driver fatigue using heart rate variability: A systematic review","volume":"178","author":"Lu","year":"2022","journal-title":"Accid. Anal. Prev."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Murugan, S., Sivakumar, P.K., Kavitha, C., Harichandran, A., and Lai, W.C. (2023). An Electro-Oculogram (EOG) Sensor\u2019s Ability to Detect Driver Hypovigilance Using Machine Learning. Sensors, 23.","DOI":"10.3390\/s23062944"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, G., and Chung, W.Y. (2022). Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review. Sensors, 22.","DOI":"10.3390\/s22031100"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"10811","DOI":"10.1109\/JSEN.2021.3058658","article-title":"EEG Driving Fatigue Detection With PDC-Based Brain Functional Network","volume":"21","author":"Wang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Qin, Y.M., Hu, Z.Y., Chen, Y., Liu, J., Jiang, L.J., Che, Y.Q., and Han, C.X. (2022). Directed Brain Network Analysis for Fatigue Driving Based on EEG Source Signals. Entropy, 24.","DOI":"10.3390\/e24081093"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, T., Chen, J.C., He, E.Q., and Wang, H. (2021). Sample-Entropy-Based Method for Real Driving Fatigue Detection with Multichannel Electroencephalogram. Appl. Sci., 11.","DOI":"10.3390\/app112110279"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"618408","DOI":"10.3389\/fnbot.2021.618408","article-title":"EEG-Based Driving Fatigue Detection Using a Two-Level Learning Hierarchy Radial Basis Function","volume":"15","author":"Ren","year":"2021","journal-title":"Front. Neurorobotics"},{"key":"ref_13","first-page":"4175","article-title":"A Product Fuzzy Convolutional Network for Detecting Driving Fatigue","volume":"14","author":"Du","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Butkeviciute, E., Michalkovic, A., and Bikulciene, L. (2022). ECG Signal Features Classification for the Mental Fatigue Recognition. Mathematics, 10.","DOI":"10.3390\/math10183395"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Halomoan, J., Ramli, K., Sudiana, D., Gunawan, T.S., and Salman, M. (2023). A New ECG Data Processing Approach to Developing an Accurate Driving Fatigue Detection Framework with Heart Rate Variability Analysis and Ensemble Learning. Information, 14.","DOI":"10.3390\/info14040210"},{"key":"ref_16","unstructured":"Garcia-Perez, S., Rodriguez, M.D., and Lopez-Nava, I.H. (December, January 29). Towards Recognition of Driver Drowsiness States by Using ECG Signals. Proceedings of the International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI), Cordoba, Spain."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ebrahimian, S., Nahvi, A., Tashakori, M., Salmanzadeh, H., Mohseni, O., and Leppanen, T. (2022). Multi-Level Classification of Driver Drowsiness by Simultaneous Analysis of ECG and Respiration Signals Using Deep Neural Networks. Int. J. Environ. Res. Public Health, 19.","DOI":"10.3390\/ijerph191710736"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dolezalek, E., Farnan, M., and Min, C.H. (2021, January 9\u201311). Physiological Signal Monitoring System to Analyze Driver Attentiveness. Proceedings of the IEEE International Midwest Symposium on Circuits and Systems (MWSCAS), Electr Network, Lansing, MI, USA.","DOI":"10.1109\/MWSCAS47672.2021.9531871"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhu, T.J., Zhang, C., Wu, T.L., Ouyang, Z., Li, H.Z., Na, X.X., Liang, J.G., and Li, W.H. (2022). Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences. Appl. Sci., 12.","DOI":"10.3390\/app12042224"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1049\/ipr2.12373","article-title":"Real time detection of driver fatigue based on CNN-LSTM","volume":"16","author":"Liu","year":"2022","journal-title":"IET Image Process."},{"key":"ref_21","unstructured":"Li, Z., and Ren, J. (August, January 30). Face Detection and Tracking Algorithm Based on Fatigue Driving. Proceedings of the 18th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Fuzhou, China."},{"key":"ref_22","unstructured":"Yin, J., Zhang, J., Cao, X.L., Yao, S.Y., and Hu, X.Y. (2022, January 20\u201322). Design of Anti-fatigue Driving System Based on Image Recognition. Proceedings of the 2nd International Conference on Optics and Image Processing (ICOIP), Taian, China."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xiao, W.C., Liu, H.L., Ma, Z.J., Chen, W.H., Sun, C.L., and Shi, B. (2022). Fatigue Driving Recognition Method Based on Multi-Scale Facial Landmark Detector. Electronics, 11.","DOI":"10.3390\/electronics11244103"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dong, B.T., Lin, H.Y., and Chang, C.C. (2022). Driver Fatigue and Distracted Driving Detection Using Random Forest and Convolutional Neural Network. Appl. Sci., 12.","DOI":"10.3390\/app12178674"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"113204","DOI":"10.1016\/j.eswa.2020.113204","article-title":"Driver fatigue transition prediction in highly automated driving using physiological features","volume":"147","author":"Zhou","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, F., Wu, S.C., Zhang, W.W., Xu, Z.F., Zhang, Y.H., and Chu, H. (2020). Multiple nonlinear features fusion based driving fatigue detection. Biomed. Signal Process. Control, 62.","DOI":"10.1016\/j.bspc.2020.102075"},{"key":"ref_27","unstructured":"(2023, September 07). Available online: https:\/\/figshare.com\/articles\/dataset\/The_original_EEG_data_for_driver_fatigue_detection\/5202739."},{"key":"ref_28","unstructured":"(2019, February 19). Available online: https:\/\/data.mendeley.com\/datasets\/dpgvc22rth\/1."},{"key":"ref_29","unstructured":"(2018, July 31). Available online: http:\/\/www.drozy.ulg.ac.be\/."},{"key":"ref_30","unstructured":"(2016, April 06). Available online: https:\/\/www.kaggle.com\/code\/azzaali\/system-for-distraction-detection-and-monitoring#Importing-libraries."},{"key":"ref_31","unstructured":"(2018, October 20). Available online: https:\/\/github.com\/TuSimple\/tusimple-benchmark\/issues\/3."},{"key":"ref_32","unstructured":"Ke, G.L., Meng, Q., Finley, T., Wang, T.F., Chen, W., Ma, W.D., Ye, Q.W., and Liu, T.Y. (2017, January 4\u20139). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nguyen, K.H., Ebbatson, M., Tran, Y., Craig, A., Nguyen, H., and Chai, R.F. (2023). Source-Space Brain Functional Connectivity Features in Electroencephalogram-Based Driver Fatigue Classification. Sensors, 23.","DOI":"10.3390\/s23052383"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1049\/bme2.12108","article-title":"Optimal feature-algorithm combination research for EEG fatigue driving detection based on functional brain network","volume":"12","author":"Zhou","year":"2023","journal-title":"IET Biom."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8386\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:04:47Z","timestamp":1760130287000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8386"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,11]]},"references-count":34,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23208386"],"URL":"https:\/\/doi.org\/10.3390\/s23208386","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,11]]}}}