{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T14:53:53Z","timestamp":1782831233095,"version":"3.54.5"},"reference-count":55,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,11,1]],"date-time":"2025-11-01T00:00:00Z","timestamp":1761955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62027810"],"award-info":[{"award-number":["62027810"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Driver fatigue is a primary factor in traffic accidents and poses a serious threat to road safety. To address this issue, this paper proposes a multi-feature fusion fatigue detection method based on an improved YOLOv8 model. First, the method uses an enhanced YOLOv8 model to achieve high-precision face detection. Then, it crops the detected face regions. Next, the lightweight PFLD (Practical Facial Landmark Detector) model performs keypoint detection on the cropped images, extracting 68 facial feature points and calculating key indicators related to fatigue status. These indicators include the eye aspect ratio (EAR), eyelid closure percentage (PERCLOS), mouth aspect ratio (MAR), and head posture ratio (HPR). To mitigate the impact of individual differences on detection accuracy, the paper introduces a novel sliding window model that combines a dynamic threshold adjustment strategy with an exponential weighted moving average (EWMA) algorithm. Based on this framework, blink frequency (BF), yawn frequency (YF), and nod frequency (NF) are calculated to extract time-series behavioral features related to fatigue. Finally, the driver\u2019s fatigue state is determined using a comprehensive fatigue assessment algorithm. Experimental results on the WIDER FACE and YAWDD datasets demonstrate this method\u2019s significant advantages in improving detection accuracy and computational efficiency. By striking a better balance between real-time performance and accuracy, the proposed method shows promise for real-world driving applications.<\/jats:p>","DOI":"10.3390\/jimaging11110385","type":"journal-article","created":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T14:40:48Z","timestamp":1762180848000},"page":"385","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Lightweight and Real-Time Driver Fatigue Detection Based on MG-YOLOv8 with Facial Multi-Feature Fusion"],"prefix":"10.3390","volume":"11","author":[{"given":"Chengming","family":"Chen","sequence":"first","affiliation":[{"name":"College of Engineering Science and Technology, Shanghai Ocean University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-5513-5491","authenticated-orcid":false,"given":"Xinyue","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Engineering Science and Technology, Shanghai Ocean University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meng","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Engineering Science and Technology, Shanghai Ocean University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhijian","family":"Li","sequence":"additional","affiliation":[{"name":"College of Engineering Science and Technology, Shanghai Ocean University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanqi","family":"Du","sequence":"additional","affiliation":[{"name":"College of Engineering Science and Technology, Shanghai Ocean University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1859-1411","authenticated-orcid":false,"given":"Yandan","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Illuminating Engineering & Light Sources, School of Information Science and Technology, Fudan University, Shanghai 201203, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,1]]},"reference":[{"key":"ref_1","first-page":"453","article-title":"DeepVeil: Deep learning for identification of face, gender, expression recognition under veiled conditions","volume":"14","author":"Hassanat","year":"2022","journal-title":"Int. J. Biom."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6297","DOI":"10.1364\/AO.461627","article-title":"Improved YOLOX detection algorithm for contraband in X-ray images","volume":"61","author":"Zhang","year":"2022","journal-title":"Appl. Opt."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1142\/S0129054122420205","article-title":"An improved helmet detection algorithm based on YOLO V4","volume":"33","author":"Yang","year":"2022","journal-title":"Int. J. Found. Comput. Sci."},{"key":"ref_4","first-page":"3","article-title":"Face detection and recognition system based on hybrid statistical, machine learning and nature-based computing","volume":"14","author":"Vinodini","year":"2022","journal-title":"Int. J. Biom."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"67219","DOI":"10.1109\/ACCESS.2021.3076365","article-title":"Driver eye location and state estimation based on a robust model and data augmentation","volume":"9","author":"Ling","year":"2021","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"13965","DOI":"10.1007\/s00521-021-06038-y","article-title":"Vehicle driver drowsiness detection method using wearable EEG based on convolution neural network","volume":"33","author":"Zhu","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1109\/TITS.2018.2889962","article-title":"Vigilance estimation using a wearable EOG device in real driving environment","volume":"21","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jeon, Y., Kim, B., and Baek, Y. (2021). Ensemble CNN to detect drowsy driving with in-vehicle sensor data. Sensors, 21.","DOI":"10.3390\/s21072372"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"113778","DOI":"10.1016\/j.eswa.2020.113778","article-title":"Applying deep neural networks for multi-level classification of driver drowsiness using Vehicle-based measures","volume":"162","author":"Arefnezhad","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3089","DOI":"10.1109\/TITS.2020.2979527","article-title":"Vision-based fatigue driving recognition method integrating heart rate and facial features","volume":"22","author":"Du","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"64136","DOI":"10.1109\/ACCESS.2019.2917382","article-title":"Fatigue state detection based on multi-index fusion and state recognition network","volume":"7","author":"Ji","year":"2019","journal-title":"IEEE Access"},{"key":"ref_12","unstructured":"Zhao, Z., Bai, H., Ke, B., Cui, Y., Deng, L., Zhang, Y., Zhang, K., and Schindler, K. (2025). A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking. arXiv."},{"key":"ref_13","unstructured":"Zhao, Z., Deng, L., Bai, H., Cui, Y., Zhang, Z., Zhang, Y., Qin, H., Chen, D., Zhang, J., and Wang, P. (2024). Image fusion via vision-language model. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Zhang, K., Xu, S., Chen, D., Timofte, R., and Van Gool, L. (2024, January 16\u201322). Equivariant multi-modality image fusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02448"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Bai, H., Zhu, Y., Zhang, J., Xu, S., Zhang, Y., Zhang, K., Meng, D., Timofte, R., and Van Gool, L. (2023, January 1\u20136). DDFM: Denoising diffusion model for multi-modality image fusion. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.00742"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., and Van Gool, L. (2023, January 17\u201324). Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00572"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1023\/B:VISI.0000013087.49260.fb","article-title":"Robust real-time face detection","volume":"57","author":"Viola","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst., 28."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). Ssd: Single shot multibox detector. Proceedings of the Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"147054","DOI":"10.1109\/ACCESS.2021.3123388","article-title":"Fatigue driving detection based on deep learning and multi-index fusion","volume":"9","author":"Jia","year":"2021","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1016\/j.procs.2022.01.135","article-title":"A Review of Yolo algorithm developments","volume":"199","author":"Jiang","year":"2022","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13640-018-0403-6","article-title":"Research on key technologies of intelligent transportation based on image recognition and anti-fatigue driving","volume":"2019","author":"Wang","year":"2019","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"22","DOI":"10.20473\/jisebi.7.1.22-30","article-title":"Fatigue detection on face image using FaceNet algorithm and K-nearest neighbor classifier","volume":"7","author":"Adhinata","year":"2021","journal-title":"J. Inf. Syst. Eng. Bus. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Deng, J., Guo, J., Ververas, E., Kotsia, I., and Zafeiriou, S. (2020, January 13\u201319). Retinaface: Single-shot multi-level face localisation in the wild. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00525"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"102723","DOI":"10.1016\/j.jvcir.2019.102723","article-title":"Driver fatigue detection based on deeply-learned facial expression representation","volume":"71","author":"Liu","year":"2020","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"101244","DOI":"10.1109\/ACCESS.2020.2998363","article-title":"A fatigue driving detection algorithm based on facial multi-feature fusion","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"8851485","DOI":"10.1155\/2020\/8851485","article-title":"A fatigue driving detection algorithm based on facial motion information entropy","volume":"2020","author":"You","year":"2020","journal-title":"J. Adv. Transp."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00138-014-0644-z","article-title":"Spatio-temporal features for the automatic control of driver drowsiness state and lack of concentration","volume":"26","author":"Akrout","year":"2015","journal-title":"Mach. Vis. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"13883","DOI":"10.1007\/s00521-022-07209-1","article-title":"A novel drowsiness detection model using composite features of head, eye, and facial expression","volume":"34","author":"Pandey","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"13821","DOI":"10.1109\/TCYB.2021.3110813","article-title":"Two-stream spatial\u2013temporal graph convolutional networks for driver drowsiness detection","volume":"52","author":"Bai","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pandey, N.N., and Muppalaneni, N.B. (2021, January 25\u201327). Real-time drowsiness identification based on eye state analysis. Proceedings of the 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India.","DOI":"10.1109\/ICAIS50930.2021.9395975"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2287","DOI":"10.1007\/s11554-021-01114-x","article-title":"Temporal and spatial feature based approaches in drowsiness detection using deep learning technique","volume":"18","author":"Pandey","year":"2021","journal-title":"J. Real-Time Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"12491","DOI":"10.1109\/ACCESS.2020.2963960","article-title":"Real time driver fatigue detection system based on multi-task ConNN","volume":"8","author":"Becerikli","year":"2020","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5383573","DOI":"10.1155\/2021\/5383573","article-title":"Driver fatigue detection based on facial key points and LSTM","volume":"2021","author":"Chen","year":"2021","journal-title":"Secur. Commun. Netw."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/LSENS.2021.3070419","article-title":"Estimation of driver vigilance status using real-time facial expression and deep learning","volume":"5","author":"Tamanani","year":"2021","journal-title":"IEEE Sens. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Joshi, A., Kyal, S., Banerjee, S., and Mishra, T. (2020, January 13). In-the-wild drowsiness detection from facial expressions. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304579"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4755","DOI":"10.1109\/TCSVT.2019.2958188","article-title":"Driver drowsiness recognition via 3D conditional GAN and two-level attention Bi-LSTM","volume":"30","author":"Hu","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhang, M., Liu, D., Wang, Q., Zhao, B., Bai, O., and Sun, J. (2022). Detection of alertness-related EEG signals based on decision fused BP neural network. Biomed. Signal Process. Control, 74.","DOI":"10.1016\/j.bspc.2022.103479"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"20425","DOI":"10.1007\/s11042-022-12433-x","article-title":"Development and validation of a deep learning-based algorithm for drowsiness detection in facial photographs","volume":"81","author":"Husain","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ghoddoosian, R., Galib, M., and Athitsos, V. (2019, January 16\u201317). A realistic dataset and baseline temporal model for early drowsiness detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00027"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13\u201319). ECA-Net: Efficient channel attention for deep convolutional neural networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"ref_43","unstructured":"Jocher, G., Mattioli, F., Qaddoumi, B., Laughing, Q., and Munawar, M. (2024, January 14\u201316). YOLOv9: A Leap Forward in Object Detection Technology. Proceedings of the 1st International Conference on Engineering Management, Information Technology and Intelligence, Shanghai, China."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Liao, H.Y.M., Wu, Y.H., Chen, P.Y., Hsieh, J.W., and Yeh, I.H. (2020, January 14\u201319). CSPNet: A new backbone that can enhance learning capability of CNN. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00203"},{"key":"ref_45","unstructured":"Wang, C.-Y., Liao, H.-Y.M., and Yeh, I.-H. (2022). Designing network design strategies through gradient path analysis. arXiv."},{"key":"ref_46","unstructured":"Wang, C.Y., Yeh, I.H., and Mark Liao, H.Y. (October, January 29). Yolov9: Learning what you want to learn using programmable gradient information. Proceedings of the European Conference on Computer Vision, Milan, Italy."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., and Ren, D. (2020, January 7\u201312). Distance-IoU loss: Faster and better learning for bounding box regression. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6999"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zhang, F., Su, J., Geng, L., and Xiao, Z. (2017, January 17\u201319). Driver fatigue detection based on eye state recognition. Proceedings of the 2017 International Conference on Machine Vision and Information Technology (CMVIT), Singapore.","DOI":"10.1109\/CMVIT.2017.25"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.neucom.2022.07.042","article-title":"Focal and efficient IOU loss for accurate bounding box regression","volume":"506","author":"Zhang","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/S0736-0266(01)00079-1","article-title":"Active range of motion of the head and cervical spine: A three-dimensional investigation in healthy young adults","volume":"20","author":"Ferrario","year":"2002","journal-title":"J. Orthop. Res."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wu, W., Qian, C., Yang, S., Wang, Q., Cai, Y., and Zhou, Q. (2018, January 18\u201323). Look at boundary: A boundary-aware face alignment algorithm. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00227"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., and Hariri, B. (2014, January 19\u201321). YawDD: A yawning detection dataset. Proceedings of the 5th ACM Multimedia Systems Conference, Singapore.","DOI":"10.1145\/2557642.2563678"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"80565","DOI":"10.1109\/ACCESS.2022.3192454","article-title":"Privacy-preserving federated transfer learning for driver drowsiness detection","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"15374","DOI":"10.1016\/j.ifacol.2020.12.2357","article-title":"A real-time driver fatigue detection method based on two-stage convolutional neural network","volume":"53","author":"He","year":"2020","journal-title":"IFAC-PapersOnLine"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"5956","DOI":"10.1109\/JSEN.2023.3239029","article-title":"Fatigue working detection based on facial multifeature fusion","volume":"23","author":"Yi","year":"2023","journal-title":"IEEE Sens. J."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/11\/385\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,4]],"date-time":"2025-11-04T05:29:53Z","timestamp":1762234193000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/11\/385"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,1]]},"references-count":55,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["jimaging11110385"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11110385","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,1]]}}}