{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T09:53:19Z","timestamp":1785405199363,"version":"3.56.0"},"reference-count":43,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007835","name":"Silesian University of Technology","doi-asserted-by":"publisher","award":["02\/050\/SDU\/10-21-12"],"award-info":[{"award-number":["02\/050\/SDU\/10-21-12"]}],"id":[{"id":"10.13039\/501100007835","id-type":"DOI","asserted-by":"publisher"}]},{"award":["02\/050\/SDU\/10-21-12"],"award-info":[{"award-number":["02\/050\/SDU\/10-21-12"]}],"id":[{"id":"https:\/\/ror.org\/02dyjk442","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Vision-based driver fatigue detection remains challenging because facial signals associated with fatigue are often ambiguous, while geometric indicators such as Eye Aspect Ratio (EAR) and Percentage of Eye Closure (PERCLOS) are prone to false positives caused by normal facial activity, including smiling or speaking. This paper proposes a context-aware framework that integrates behavioural, geometric, and emotional information for robust fatigue assessment. Facial landmarks are extracted using MediaPipe Face Mesh, while adaptive eye-closure detection is performed through multi-stage validation combining EAR trajectories, mouth activity, head-pose analysis, and event-level filtering. Emotion recognition is achieved using an EfficientNet-B0 convolutional neural network trained on the AffectNet dataset, enabling frame-level estimation of facial expression probabilities. These predictions are aggregated into descriptors representing emotional variability and fatigue-related emotional relevance over time. Behavioural information obtained from blinking, yawning, head nodding, and validated PERCLOS is fused with emotional context to construct a multi-level fatigue assessment model. The final Driver Fatigue Risk Index combines physiological eye-closure information with contextual behavioural\u2013emotional analysis, providing an interpretable estimation of driver state rather than a binary classification alone. Experimental evaluation on the NTHU-DDD dataset achieved 94% accuracy and demonstrated improved robustness under non-frontal head poses and expressive facial behaviour.<\/jats:p>","DOI":"10.3390\/s26134120","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T08:26:15Z","timestamp":1782807975000},"page":"4120","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Emotion-Aware Contextual Modelling for Robust Driver Fatigue Detection"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7723-8063","authenticated-orcid":false,"given":"Sebastian","family":"Budzan","sequence":"first","affiliation":[{"name":"Department of Measurements and Control Systems, Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3855-9988","authenticated-orcid":false,"given":"Roman","family":"Wy\u017cgolik","sequence":"additional","affiliation":[{"name":"Department of Measurements and Control Systems, Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"ref_1","unstructured":"Goldenbeld, C., and Nikolaou, D. (2019). Driver Fatigue, Institute for Road Safety Research (SWOV). ESRA2 Thematic Report No. 4."},{"key":"ref_2","unstructured":"Tefft, B.C. (2024). Drowsy Driving in Fatal Crashes, United States, 2017\u20132021 (Research Brief), AAA Foundation for Traffic Safety."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.trf.2008.11.005","article-title":"Driver Fatigue: The Importance of Identifying Causal Factors of Fatigue When Considering Detection and Countermeasure Technologies","volume":"12","author":"May","year":"2009","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_4","first-page":"100865","article-title":"Structural Analysis of Driver Fatigue Behavior: A Systematic Review","volume":"21","author":"Zhang","year":"2023","journal-title":"Transp. Res. Interdiscip. Perspect."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"61904","DOI":"10.1109\/ACCESS.2019.2914373","article-title":"A Survey on State-of-the-Art Drowsiness Detection Techniques","volume":"7","author":"Ramzan","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Albadawi, Y., Takruri, M., and Awad, M. (2022). A Review of Recent Developments in Driver Drowsiness Detection Systems. Sensors, 22.","DOI":"10.3390\/s22052069"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"97969","DOI":"10.1109\/ACCESS.2024.3424654","article-title":"A Systematic Review on Driver Drowsiness Detection Using Eye Activity Measures","volume":"12","author":"Kolus","year":"2024","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ed-Doughmi, Y., Idrissi, N., and Hbali, Y. (2020). Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network. J. Imaging, 6.","DOI":"10.3390\/jimaging6030008"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Huynh, X.P., Park, S.M., and Kim, Y.G. (2017). Detection of Driver Drowsiness Using 3D Deep Neural Network and Semi-Supervised Gradient Boosting Machine. Computer Vision\u2014ACCV 2016 Workshops; Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/978-3-319-54526-4_10"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Alonazi, M., Alshahrani, H.J., Alotaibi, F.A., Maray, M., Alghamdi, M., and Sayed, A. (2023). Automated Facial Emotion Recognition Using the Pelican Optimization Algorithm with a Deep Convolutional Neural Network. Electronics, 12.","DOI":"10.3390\/electronics12224608"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.ifacol.2019.08.071","article-title":"Emotion Monitoring System for Drivers","volume":"52","author":"Kowalczuk","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"102319","DOI":"10.1016\/j.sysarc.2021.102319","article-title":"A Survey on Vision-Based Driver Distraction Analysis","volume":"121","author":"Li","year":"2021","journal-title":"J. Syst. Archit."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5449","DOI":"10.1109\/TPAMI.2024.3366769","article-title":"From Simple to Complex Scenes: Learning Robust Feature Representations for Accurate Human Parsing","volume":"46","author":"Liu","year":"2024","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","first-page":"5005712","article-title":"Mask-Guided Mamba Fusion for Drone-Based Visible-Infrared Vehicle Detection","volume":"62","author":"Wang","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103083","DOI":"10.1016\/j.ergon.2021.103083","article-title":"A Method for Fatigue Detection Based on Driver\u2019s Steering Wheel Grip","volume":"82","author":"Li","year":"2021","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5537","DOI":"10.1109\/TVT.2021.3072936","article-title":"Can Steering Wheel Detect Your Driving Fatigue?","volume":"70","author":"Lu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"101027","DOI":"10.1016\/j.rineng.2023.101027","article-title":"A Systematic Review on Automated Human Emotion Recognition Using Electroencephalogram Signals and Artificial Intelligence","volume":"18","author":"Vempati","year":"2023","journal-title":"Results Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1938","DOI":"10.1016\/j.procs.2019.09.366","article-title":"ECG Sensor for Detection of Driver\u2019s Drowsiness","volume":"159","author":"Gromer","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Vaussenat, F., Bhattacharya, A., Payette, J., Saidi, A., Bellemin, V., Renaud-Dumoulin, G.G., Cloutier, S.G., and Gagnon, G. (2026). Early Drowsiness Detection via Second-Order Derivative Analysis of Heart Rate Variability: A Non-Contact ECG Approach with Machine Learning. Sensors, 26.","DOI":"10.20944\/preprints202601.2310.v1"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2650","DOI":"10.1038\/s41598-022-05810-x","article-title":"Driver Drowsiness Estimation Using EEG Signals with a Dynamical Encoder\u2013Decoder Modelling Framework","volume":"12","author":"Arefnezhad","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"12240","DOI":"10.1038\/s41598-025-86709-1","article-title":"Optimized Driver Fatigue Detection Method Using Multimodal Neural Networks","volume":"15","author":"Cao","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chandiwala, J., and Agarwal, S. (2021, January 19\u201320). Driver\u2019s Real-Time Drowsiness Detection Using Adaptable Eye Aspect Ratio and Smart Alarm System. Proceedings of the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.","DOI":"10.1109\/ICACCS51430.2021.9441756"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"114334","DOI":"10.1016\/j.eswa.2020.114334","article-title":"Efficient and Compact Face Descriptor for Driver Drowsiness Detection","volume":"168","author":"Moujahid","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_24","first-page":"678786","article-title":"Driver\u2019s Fatigue Detection Based on Yawning Extraction","volume":"2014","author":"Alioua","year":"2014","journal-title":"Int. J. Veh. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Weng, C.-H., Lai, Y.-H., and Lai, S.-H. (2016, January 20\u201324). Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. Proceedings of the Asian Conference on Computer Vision Workshops, Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54526-4_9"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., and Hariri, B. (2014, January 19\u201321). YawDD: A Yawning Detection Dataset. Proceedings of the ACM Multimedia Systems Conference, Singapore.","DOI":"10.1145\/2557642.2563678"},{"key":"ref_27","unstructured":"Oghamudu, A., Haythornthwaite, C., and Barshan, B. (2020, January 11\u201314). Driver Drowsiness Detection Based on Joint Monitoring of Yawning, Blinking and Nodding. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada."},{"key":"ref_28","unstructured":"Soukupov\u00e1, T., and \u010cech, J. (July, January 26). Real-Time Eye Blink Detection Using Facial Landmarks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), Las Vegas, NV, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Azim, T., Jaffar, M.A., Ramzan, M., and Mirza, A.M. (2009). Automatic Fatigue Detection of Drivers through Yawning Analysis. Proceedings of the International Conference on Signal Processing (SIP); Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/978-3-642-10546-3_16"},{"key":"ref_30","first-page":"123","article-title":"Driver Drowsiness Monitoring Based on Yawning Detection","volume":"Volume 63","author":"Abtahi","year":"2014","journal-title":"Proceedings of the 2011 IEEE International Instrumentation and Measurement Technology Conference"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cichocka, S., and Rumi\u0144ski, J. (2024, January 8\u201311). Driver Fatigue Detection Method Based on Facial Image Analysis. Proceedings of the 2024 16th International Conference on Human System Interaction (HSI), Paris, France.","DOI":"10.1109\/HSI61632.2024.10613597"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"179396","DOI":"10.1109\/ACCESS.2019.2958667","article-title":"A Real-Time Driving Drowsiness Detection Algorithm with Individual Differences Consideration","volume":"7","author":"You","year":"2019","journal-title":"IEEE Access"},{"key":"ref_33","unstructured":"Krishna, G.S., Supriya, K., and Vardhan, J. (2022). Vision Transformers and YOLOv5 Based Driver Drowsiness Detection Framework. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ghoddoosian, R., Galib, M., and Athitsos, V. (2019). A Realistic Dataset and Baseline Temporal Model for Early Drowsiness Detection. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE.","DOI":"10.1109\/CVPRW.2019.00027"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3089","DOI":"10.1109\/TITS.2020.2979527","article-title":"Vision-Based Fatigue Driving Recognition Method Integrating Heart Rate and Facial Features","volume":"22","author":"Du","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, L., Huang, Y., Wang, F., Zhu, L., and Liu, Z. (2026). Exploring the Impact of Emotional States on Fatigue Evolution in Metro Drivers: A Physiological Signal-Based Approach. Appl. Sci., 16.","DOI":"10.3390\/app16062653"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"zsaa289","DOI":"10.1093\/sleep\/zsaa289","article-title":"The Effect of Sleep Deprivation and Restriction on Mood, Emotion, and Emotion Regulation: Three Meta-Analyses in One","volume":"44","author":"Tomaso","year":"2021","journal-title":"Sleep"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"26","DOI":"10.3390\/electronics12010026","article-title":"Driver Emotion and Fatigue State Detection Based on Time Series Fusion","volume":"12","author":"Shang","year":"2023","journal-title":"Electronics"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild","volume":"10","author":"Mollahosseini","year":"2019","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, W., Qian, J., Yao, Z., Jiao, X., and Pan, J. (2019). Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection. Future Internet, 11.","DOI":"10.3390\/fi11050115"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4206","DOI":"10.1109\/TITS.2018.2883823","article-title":"Driver Drowsiness Detection Using Condition-Adaptive Representation Learning Framework","volume":"20","author":"Yu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3155","DOI":"10.1007\/s00521-020-05209-7","article-title":"Deep CNN Models-Based Ensemble Approach to Driver Drowsiness Detection","volume":"33","author":"Dua","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Albadawi, Y., AlRedhaei, A., and Takruri, M. (2023). Real-Time Machine Learning-Based Driver Drowsiness Detection Using Visual Features. J. Imaging, 9.","DOI":"10.3390\/jimaging9050091"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/13\/4120\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T08:42:31Z","timestamp":1782808951000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/13\/4120"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":43,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["s26134120"],"URL":"https:\/\/doi.org\/10.3390\/s26134120","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]}}}