{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T16:37:21Z","timestamp":1778258241386,"version":"3.51.4"},"reference-count":38,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,8,25]],"date-time":"2018-08-25T00:00:00Z","timestamp":1535155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003134","name":"Fonds pour la Formation \u00e0 la Recherche dans l\u2019Industrie et dans l\u2019Agriculture","doi-asserted-by":"publisher","award":["n\/a"],"award-info":[{"award-number":["n\/a"]}],"id":[{"id":"10.13039\/501100003134","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Drowsiness is a major cause of fatal accidents, in particular in transportation. It is therefore crucial to develop automatic, real-time drowsiness characterization systems designed to issue accurate and timely warnings of drowsiness to the driver. In practice, the least intrusive, physiology-based approach is to remotely monitor, via cameras, facial expressions indicative of drowsiness such as slow and long eye closures. Since the system\u2019s decisions are based upon facial expressions in a given time window, there exists a trade-off between accuracy (best achieved with long windows, i.e., at long timescales) and responsiveness (best achieved with short windows, i.e., at short timescales). To deal with this trade-off, we develop a multi-timescale drowsiness characterization system composed of four binary drowsiness classifiers operating at four distinct timescales (5 s, 15 s, 30 s, and 60 s) and trained jointly. We introduce a multi-timescale ground truth of drowsiness, based on the reaction times (RTs) performed during standard Psychomotor Vigilance Tasks (PVTs), that strategically enables our system to characterize drowsiness with diverse trade-offs between accuracy and responsiveness. We evaluated our system on 29 subjects via leave-one-subject-out cross-validation and obtained strong results, i.e., global accuracies of 70%, 85%, 89%, and 94% for the four classifiers operating at increasing timescales, respectively.<\/jats:p>","DOI":"10.3390\/s18092801","type":"journal-article","created":{"date-parts":[[2018,8,27]],"date-time":"2018-08-27T10:56:04Z","timestamp":1535367364000},"page":"2801","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Multi-Timescale Drowsiness Characterization Based on a Video of a Driver\u2019s Face"],"prefix":"10.3390","volume":"18","author":[{"given":"Quentin","family":"Massoz","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Faculty of Applied Science, University of Li\u00e8ge, B-4000 Li\u00e8ge, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacques G.","family":"Verly","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Faculty of Applied Science, University of Li\u00e8ge, B-4000 Li\u00e8ge, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6260-6487","authenticated-orcid":false,"given":"Marc","family":"Van Droogenbroeck","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Faculty of Applied Science, University of Li\u00e8ge, B-4000 Li\u00e8ge, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Banks, S., Dorrian, J., Basner, M., and Dinges, D. (2017). Sleep Deprivation. Principles and Practice of Sleep Medicine, Elsevier. [6th ed.].","DOI":"10.1016\/B978-0-323-24288-2.00005-2"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1037\/1076-898X.6.3.236","article-title":"The Impact of Sleep Deprivation on Decision Making: A Review","volume":"6","author":"Harrison","year":"2000","journal-title":"J. Exp. Psychol. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dorrian, J., Rogers, N., and Dinges, D. (2005). Psychomotor Vigilance Performance: Neurocognitive Assay Sensitive to Sleep Loss. Sleep Deprivation: Clinical Issues, Pharmacology, and Sleep Loss Effects, Marcel Dekker.","DOI":"10.3109\/9780203998007-4"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1046\/j.1365-2869.1996.00013.x","article-title":"Sleepiness and performance of professional drivers in a truck simulator\u2014Comparisons between day and night driving","volume":"5","author":"Gillberg","year":"1996","journal-title":"J. Sleep Res."},{"key":"ref_5","unstructured":"National Highway Traffic Safety Administration (2017). Asleep At the Wheel\u2014A National Compendium of Efforts to Eliminate Drowsy Driving."},{"key":"ref_6","unstructured":"Dinges, D., Mallis, M., Maislin, G., and Powell, J. (1998). PERCLOS, a Valid Psychophysiological Measure of Alertness as Assessed by Psychomotor Vigilance, Technical Report FHWA-MCRT-98-006."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1080\/00140130701817062","article-title":"Blinks and saccades as indicators of fatigue in sleepiness warnings: Looking tired?","volume":"51","author":"Schleicher","year":"2008","journal-title":"Ergonomics"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"29","DOI":"10.3109\/00207459008994241","article-title":"Subjective and objective sleepiness in the active individual","volume":"52","author":"Gillberg","year":"1990","journal-title":"Int. J. Neurosci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1093\/sleep\/34.5.581","article-title":"Maximizing Sensitivity of the Psychomotor Vigilance Test (PVT) to Sleep Loss","volume":"34","author":"Basner","year":"2011","journal-title":"Sleep"},{"key":"ref_10","unstructured":"Kircher, A., Uddman, M., and Sandin, J. (2002). Vehicle Control and Drowsiness, VTI. Technical Report."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1007\/s11517-015-1448-7","article-title":"Drowsiness detection using heart rate variability","volume":"54","author":"Vicente","year":"2016","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1111\/j.1365-2869.2011.00984.x","article-title":"Electrodermal lability as an indicator for subjective sleepiness during total sleep deprivation","volume":"21","author":"Michael","year":"2012","journal-title":"J. Sleep Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1111\/j.1479-8425.2010.00450.x","article-title":"A new perspective on sleepiness","volume":"8","author":"Johns","year":"2010","journal-title":"Sleep Biol. Rhythms"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1016\/j.aap.2015.09.002","article-title":"Driver drowsiness detection based on non-intrusive metrics considering individual specifics","volume":"95","author":"Wang","year":"2016","journal-title":"Accid. Anal. Prev."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ebrahim, P., Abdellaoui, A., Stolzmann, W., and Yang, B. (2014, January 5\u20138). Eyelid-based Driver State Classification under Simulated and Real Driving Conditions. Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), San Diego, CA, USA.","DOI":"10.1109\/SMC.2014.6974419"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Takeda, K., Erdogan, H., Hansen, J., and Abut, H. (2009). Machine Learning Systems for Detecting Driver Drowsiness. In-Vehicle Corpus and Signal Processing for Driver Behavior, Springer.","DOI":"10.1007\/978-0-387-79582-9"},{"key":"ref_17","unstructured":"Liang, Y., Horrey, W., Howard, M., Lee, M., Anderson, C., Shreeve, M., O\u2019Brien, C., and Czeisler, C. (2017). Prediction of drowsiness events in night shift workers during morning driving. Accid. Anal. Prev."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fran\u00e7ois, C., Hoyoux, T., Langohr, T., Wertz, J., and Verly, J. (2016). Tests of a New Drowsiness Characterization and Monitoring System Based on Ocular Parameters. Int. J. Environ. Res. Public Health, 13.","DOI":"10.3390\/ijerph13020174"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Garc\u00eda, I., Bronte, S., Bergasa, L.M., Almaz\u00e1n, J., and Yebes, J. (2012, January 3\u20137). Vision-based drowsiness detector for Real Driving Conditions. Proceedings of the 2012 IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain.","DOI":"10.1109\/IVS.2012.6232222"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nopsuwanchai, R., Noguchi, Y., Ohsuga, M., Kamakura, Y., and Inoue, Y. (2008, January 12\u201315). Driver-Independent Assessment of Arousal States from Video Sequences Based on the Classification of Eyeblink Patterns. Proceedings of the 2008 11th International IEEE Conference on Intelligent Transportation Systems, Beijing, China.","DOI":"10.1109\/ITSC.2008.4732622"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Weng, C.H., Lai, Y.H., and Lai, S.H. (2016, January 20\u201324). Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. Proceedings of the Asian Conference on Computer Vision Workshops (ACCV Workshops), Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54526-4_9"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shih, T.H., and Hsu, C.T. (2016, January 20\u201324). MSTN: Multistage Spatial-Temporal Network for Driver Drowsiness Detection. Proceedings of the Asian Conference on Computer Vision Workshops (ACCV Workshops), Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54526-4_11"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Huynh, X.P., Park, S.M., and Kim, Y.G. (2016, January 20\u201324). Detection of Driver Drowsiness Using 3D Deep Neural Network and Semi-Supervised Gradient Boosting Machine. Proceedings of the Asian Conference on Computer Vision Workshops (ACCV Workshops), Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54526-4_10"},{"key":"ref_24","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."},{"key":"ref_25","unstructured":"Fisher, D., Monty, R., and Senders, J. (1981). Oculomotor Procrastination. Eye Movements: Cognition and Visual Perception, Lawrence Erlbaum."},{"key":"ref_26","first-page":"122","article-title":"The OpenCV Library","volume":"25","author":"Bradski","year":"2000","journal-title":"Dr. Dobb\u2019s J. Softw. Tools"},{"key":"ref_27","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid Object Detection using a Boosted Cascade of Simple Features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA."},{"key":"ref_28","first-page":"1755","article-title":"Dlib-ml: A Machine Learning Toolkit","volume":"10","author":"King","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kazemi, V., and Sullivan, J. (2014, January 23\u201328). One Millisecond Face Alignment with an Ensemble of Regression Trees. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.241"},{"key":"ref_30","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Internation Conference Computer Vision Pattern Recognit, (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1016\/j.imavis.2009.08.002","article-title":"Multi-PIE","volume":"28","author":"Gross","year":"2010","journal-title":"Image Vis. Comput."},{"key":"ref_33","unstructured":"Tieleman, T., and Hinton, G. (2012). Lecture 6.5\u2014RMSProp: Divide the Gradient by a Running Average of Its Recent Magnitude. COURSERA: Neural Networks for Machine Learning, University of Toronto."},{"key":"ref_34","unstructured":"Kingma, D., and Ba, J. (arXiv, 2014). Adam: A Method for Stochastic Optimization, arXiv."},{"key":"ref_35","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Massoz, Q., Langohr, T., Fran\u00e7ois, C., and Verly, J. (2016, January 7\u201310). The ULg Multimodality Drowsiness Database (called DROZY) and Examples of Use. Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477715"},{"key":"ref_37","first-page":"1871","article-title":"LIBLINEAR: A library for large linear classification","volume":"9","author":"Fan","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1111\/j.1469-8986.1996.tb00428.x","article-title":"Counteracting driver sleepiness: Effects of napping, caffeine, and placebo","volume":"33","author":"Horne","year":"1996","journal-title":"Psychophysiology"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2801\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:21:05Z","timestamp":1760196065000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2801"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,25]]},"references-count":38,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18092801"],"URL":"https:\/\/doi.org\/10.3390\/s18092801","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,25]]}}}