{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T19:00:57Z","timestamp":1784574057152,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,19]],"date-time":"2022-07-19T00:00:00Z","timestamp":1658188800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology of Taiwan, R.O.C.","award":["108-2218-E-005-017"],"award-info":[{"award-number":["108-2218-E-005-017"]}]},{"name":"Ministry of Science and Technology of Taiwan, R.O.C.","award":["110-2221-E-005-085"],"award-info":[{"award-number":["110-2221-E-005-085"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Accidents caused by fatigue occur frequently, and numerous scholars have devoted tremendous efforts to investigate methods to reduce accidents caused by fatigued driving. Accordingly, the assessment of the spirit status of the driver through the eyes blinking frequency and the measurement of physiological signals have emerged as effective methods. In this study, a drowsiness detection system is proposed to combine the detection of LF\/HF ratio from heart rate variability (HRV) of photoplethysmographic imaging (PPGI) and percentage of eyelid closure over the pupil over time (PERCLOS), and to utilize the advantages of both methods to improve the accuracy and robustness of drowsiness detection. The proposed algorithm performs three functions, including LF\/HF ratio from HRV status judgment, eye state detection, and drowsiness judgment. In addition, this study utilized a near-infrared webcam to obtain a facial image to achieve non-contact measurement, alleviate the inconvenience of using a contact wearable device, and for use in a dark environment. Furthermore, we selected the appropriate RGB channel under different light sources to obtain LF\/HF ratio from HRV of PPGI. The main drowsiness judgment basis of the proposed drowsiness detection system is the use of algorithm to obtain sympathetic\/parasympathetic nervous balance index and percentage of eyelid closure. In the experiment, there are 10 awake samples and 30 sleepy samples. The sensitivity is 88.9%, the specificity is 93.5%, the positive predictive value is 80%, and the system accuracy is 92.5%. In addition, an electroencephalography signal was used as a contrast to validate the reliability of the proposed method.<\/jats:p>","DOI":"10.3390\/s22145380","type":"journal-article","created":{"date-parts":[[2022,7,19]],"date-time":"2022-07-19T23:10:22Z","timestamp":1658272222000},"page":"5380","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Drowsiness Detection System Based on PERCLOS and Facial Physiological Signal"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3233-8087","authenticated-orcid":false,"given":"Robert Chen-Hao","family":"Chang","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan"},{"name":"Department of Electrical Engineering, National Chi Nan University, Nantou 54561, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chia-Yu","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei-Ting","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng-Di","family":"Chiu","sequence":"additional","affiliation":[{"name":"Neurosurgical Department and Spine Center, China Medical University Hospital, Taichung 404332, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,19]]},"reference":[{"key":"ref_1","unstructured":"(2017, July 01). The National Highway Traffic Safety Administration(NHTSA), Available online: http:\/\/www.nhtsa.gov\/."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.trf.2006.03.003","article-title":"Sleepy at the wheel: Knowledge, symptoms and 51 behaviour among car drivers","volume":"10","author":"Nordbakke","year":"2007","journal-title":"Transp. Res. Part F"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cho, D., and Lee, B. (2016, January 16\u201320). Non-contact robust heeart rate estimation using HSV color model and matrix based IIR filter in the face video imaging. Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591567"},{"key":"ref_4","first-page":"1297","article-title":"Bayesian nonnegative CP decomposition-based feature extraction algorithm for drowsiness detection","volume":"25","author":"Qian","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1109\/TBME.2018.2879346","article-title":"Heart rate vartiability-based driver drowsiness detection and its validation with EEG","volume":"66","author":"Fujiwara","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_6","first-page":"112","article-title":"A study to vital signs measured using a webcam","volume":"6","author":"Guo","year":"2012","journal-title":"JITA"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chang, R.C.-H., Wang, C.-Y., and Kao, Y.-Y. (2021, January 15\u201317). Implementation of a novel intelligent drowsiness detection warning system. Proceedings of the 2021 IEEE International Conference on Consumer Electronics, Penghu, Taiwan.","DOI":"10.1109\/ICCE-TW52618.2021.9602872"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3709","DOI":"10.1109\/JSEN.2019.2960158","article-title":"Comprehensive drowsiness level detection model combining multimodal information","volume":"20","author":"Sunagawa","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_9","unstructured":"Chang, C.W. (2014). Hough Transform Based Drowsy Driver Detection System and Embedded System Implementation. [Master\u2019s Thesis, Nat. Chung Hsing University]."},{"key":"ref_10","unstructured":"Chen, C.H. (2004). The Study of the Variation Trend for Diastolic Pressure of the Surgical Patients utilizing Non-Invasive Plethysmography Signal. [Master\u2019s Thesis, Sun Yat-sen Univiversity]."},{"key":"ref_11","unstructured":"Lin, C.H. (2015). Contact-Free Measurement of Cardiac Pulse Using IR Video Imaging. [Master\u2019s thesis, Nat. Chung Hsing University]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1088\/0967-3334\/35\/5\/807","article-title":"Non-contact video-based vital sign monitoring using ambient light and auto-regressive models","volume":"35","author":"Tarassenko","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10762","DOI":"10.1364\/OE.18.010762","article-title":"Non-contact, automated cardiac pulse measurements using video imaging and blind source separation","volume":"18","author":"Poh","year":"2010","journal-title":"Opt. Express"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1093\/qjmed\/hci018","article-title":"Heart rate variability measurements and the prediction of ventricular arrhythmias","volume":"98","author":"Reed","year":"2005","journal-title":"Qjm"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1126\/science.6166045","article-title":"Power spectrum analysis of heart rate fluctuation: A quantitative probe of beat-to-beat cardiovascular control","volume":"213","author":"Akselrod","year":"1981","journal-title":"Science"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1016\/0002-9149(92)90788-Z","article-title":"Correlations among time and frequency domain measures of heart period variability two weeks after acute myocardial infarction","volume":"69","author":"Bigger","year":"1992","journal-title":"Am. J. Cardiol."},{"key":"ref_17","unstructured":"(2017, July 01). Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology-Heart Rate Variability. Available online: https:\/\/circ.ahajournals.org\/content\/93\/5\/1043."},{"key":"ref_18","first-page":"290","article-title":"Analysis of Heart Rhythm Variability","volume":"52","author":"Wang","year":"2009","journal-title":"Taiwan Med. J."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pinheiro, N., Couceiro, R., Henriques, J., Muehlsteff, J., Quintal, I., Gon\u00e7alves, L., and Carvalho, P. (2016, January 6\u201320). Can PPG be used for HRV analysis?. Proceedings of the 2016 IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591347"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sengthipphany, T., Tretriluxana, S., and Chitsakul, K. (2015, January 24\u201327). Comparison of Heart Rate statistical parameters from Photoplethysmographic signal in resting and exercise conditions. Proceedings of the Electrical Engineering\/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), 2015 12th International Conference, Hua Hin, Thailand.","DOI":"10.1109\/ECTICon.2015.7207074"},{"key":"ref_21","unstructured":"(2022, May 01). KingNet National Internet Hospital \u201cSympathetic Nerve\u201d and \u201cParasympathetic Nerve\u201d. Available online: http:\/\/hospital.kingnet.com.tw\/essay\/essay.html?pid=18479."},{"key":"ref_22","unstructured":"(2022, May 01). Autonomic Nervous Medicine Association Health Education Manual for HRV Detection and Treatment of Autonomic Nervous Disorders. Available online: http:\/\/hrvtw.blogspot.tw\/2010\/09\/hrvq.html."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Heseltine, T., Pears, N., and Austin, J. (2002, January 16\u201318). Evaluation of image pre-processing techniques for eigenface based face recognition. Proceedings of the Second International Conference on Image and Graphics (SPIE), Heifei, China.","DOI":"10.1117\/12.477052"},{"key":"ref_24","unstructured":"Park, I., Ahn, J.-H., and Byun, H. (2006, January 20\u201324). Efficient measurement of eye blinking under various illumination conditions for drowsiness detection systems. Proceedings of the ICPR 2006\u201418th International Conference on Pattern Recognition, HongKong."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sivanath, S.K., Muralikrishnan, S.A., Thothadri, P., and Raja, V. (2012, January 27\u201329). Eyeball and blink controlled firing system for military tank using LabVIEW. Proceedings of the 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI), Kharagpur, India.","DOI":"10.1109\/IHCI.2012.6481807"},{"key":"ref_26","unstructured":"(2022, May 01). International Organization of Societies for Electrophysiological Technology. Available online: https:\/\/www.oset.org\/Guidelines.html."},{"key":"ref_27","unstructured":"Nayak, C.S., and Anilkumar, A.C. (2022). EEG Normal Waveforms. [Updated 8 May 2022]. StatPearls [Internet], StatPearls Publishing."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"St. Louis, E.K., and Frey, L.C. (2016). Electroencephalography (EEG): An Introductory Text and Atlas of Normal and Abnormal Findings in Adults, Children, and Infants [Internet], American Epilepsy Society.","DOI":"10.5698\/978-0-9979756-0-4"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1588","DOI":"10.1016\/j.neuroscience.2009.09.057","article-title":"Event-related delta and theta synchronization during explicit and implicit emotion processing","volume":"164","author":"Knyazev","year":"2009","journal-title":"Neuroscience"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.biopsycho.2009.10.008","article-title":"EEG theta\/beta ratio in relation to fear-modulated response-inhibition, attentional control, and affective traits","volume":"83","author":"Putman","year":"2010","journal-title":"Biol. Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Del Giudice, R., Blume, C., Wislowska, M., Wielek, T., Heib, D.P., and Schabus, M. (2016). The Voice of Anger: Oscillatory EEG Responses to Emotional Prosody. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0159429"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.neuropsychologia.2017.02.021","article-title":"Characterizing the roles of alpha and theta oscillations in multisensory attention","volume":"99","author":"Keller","year":"2017","journal-title":"Neuropsychologia"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.biopsycho.2013.11.010","article-title":"Frontal theta activity reflects distinct aspects of mental fatigue","volume":"96","author":"Wascher","year":"2013","journal-title":"Biol. Psychol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/S0165-0173(98)00056-3","article-title":"EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis","volume":"29","author":"Klimesch","year":"1999","journal-title":"Brain Res. Rev."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.neuropharm.2004.03.017","article-title":"Effects of diazepam and zolpidem on EEG. beta frequencies are behavior-specific in rats","volume":"47","author":"Drinkenburg","year":"2004","journal-title":"Neuropharmacology"},{"key":"ref_36","unstructured":"NeuroSky, Inc. (2009). Brain Wave Signal (EEG), NeuroSky, Inc."},{"key":"ref_37","unstructured":"Shih, M.H. (2014). Design and Embedded System Implementation of Driver Eye Detection Algorithm with Facial Features. [Master\u2019s Thesis, Nat. Chung Hsing University]."},{"key":"ref_38","first-page":"33","article-title":"Local gradient pattern-A novel feature representation for facial expression recognition","volume":"2","author":"Islam","year":"2014","journal-title":"JAIDM"},{"key":"ref_39","unstructured":"Szypulska, M., and Piotrowski, Z. (2012, January 24\u201326). Prediction of fatigue and sleep onset using HRV analysis. Proceedings of the 19th International Conference Mixed Design of Integrated Circuits and Systems-MIXDES 2012, Warsaw, Poland."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1161\/01.CIR.94.2.122","article-title":"Sex-related differences in autonomic modulation of heart rate in middle-aged subjects","volume":"94","author":"Huikuri","year":"1996","journal-title":"Circulation"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5380\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:53:56Z","timestamp":1760140436000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5380"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,19]]},"references-count":40,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145380"],"URL":"https:\/\/doi.org\/10.3390\/s22145380","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,19]]}}}