{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T15:00:37Z","timestamp":1761663637080,"version":"build-2065373602"},"reference-count":56,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2017,6,30]],"date-time":"2017-06-30T00:00:00Z","timestamp":1498780800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The necessity for the classification of open and closed eyes is increasing in various fields, including analysis of eye fatigue in 3D TVs, analysis of the psychological states of test subjects, and eye status tracking-based driver drowsiness detection. Previous studies have used various methods to distinguish between open and closed eyes, such as classifiers based on the features obtained from image binarization, edge operators, or texture analysis. However, when it comes to eye images with different lighting conditions and resolutions, it can be difficult to find an optimal threshold for image binarization or optimal filters for edge and texture extraction. In order to address this issue, we propose a method to classify open and closed eye images with different conditions, acquired by a visible light camera, using a deep residual convolutional neural network. After conducting performance analysis on both self-collected and open databases, we have determined that the classification accuracy of the proposed method is superior to that of existing methods.<\/jats:p>","DOI":"10.3390\/s17071534","type":"journal-article","created":{"date-parts":[[2017,6,30]],"date-time":"2017-06-30T10:04:58Z","timestamp":1498817098000},"page":"1534","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["A Study of Deep CNN-Based Classification of Open and Closed Eyes Using a Visible Light Camera Sensor"],"prefix":"10.3390","volume":"17","author":[{"given":"Ki","family":"Kim","sequence":"first","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Hyung","family":"Hong","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Gi","family":"Nam","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Kang","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2110","DOI":"10.3390\/s140202110","article-title":"Gaze tracking system for user wearing glasses","volume":"14","author":"Gwon","year":"2014","journal-title":"Sensors"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1109\/TCE.2011.6131137","article-title":"New computer interface combining gaze tracking and brainwave measurements","volume":"57","author":"Bang","year":"2011","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.compbiomed.2016.01.012","article-title":"A practical efficient human computer interface based on saccadic eye movements for people with disabilities","volume":"70","author":"Soltani","year":"2016","journal-title":"Comput. Biol. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.bbr.2015.11.028","article-title":"The dopamine D1 receptor agonist SKF-82958 effectively increases eye blinking count in common marmosets","volume":"300","author":"Kotani","year":"2016","journal-title":"Behav. Brain Res."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Appel, T., Santini, T., and Kasneci, E. (2016, January 12\u201316). Brightness- and motion-based blink detection for head-mounted eye trackers. Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany.","DOI":"10.1145\/2968219.2968341"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1080\/10739149.2013.796560","article-title":"An improved and portable eye-blink duration detection system to warn of driver fatigue","volume":"41","author":"Hsieh","year":"2013","journal-title":"Instrum. Sci. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1109\/TPAMI.2010.86","article-title":"Eye movement analysis for activity recognition using electrooculography","volume":"33","author":"Bulling","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chittaro, L., and Sioni, R. (2013, January 2\u20135). Exploring eye-blink startle response as a physiological measure for affective computing. Proceedings of the Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland.","DOI":"10.1109\/ACII.2013.44"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Champaty, B., Pal, K., and Dash, A. (2013, January 4\u20136). Functional electrical stimulation using voluntary eyeblink for foot drop correction. Proceedings of the International Conference on Microelectronics, Communication and Renewable Energy, Kerala, India.","DOI":"10.1109\/AICERA-ICMiCR.2013.6575966"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1088\/0967-3334\/37\/3\/401","article-title":"An unsupervised eye blink artifact detection method for real-time electroencephalogram processing","volume":"37","author":"Chang","year":"2016","journal-title":"Physiol. Meas."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lalonde, M., Byrns, D., Gagnon, L., Teasdale, N., and Laurendeau, D. (2007, January 28\u201330). Real-time eye blink detection with GPU-based SIFT tracking. Proceedings of the Fourth Canadian Conference on Computer and Robot Vision, Montreal, QC, Canada.","DOI":"10.1109\/CRV.2007.54"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mohanakrishnan, J., Nakashima, S., Odagiri, J., and Yu, S. (2013, January 15\u201317). A novel blink detection system for user monitoring. Proceedings of the 1st IEEE Workshop on User-Centered Computer Vision, Tampa, FL, USA.","DOI":"10.1109\/UCCV.2013.6530806"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1016\/j.jneumeth.2010.08.034","article-title":"Blink detection robust to various facial poses","volume":"193","author":"Lee","year":"2010","journal-title":"J. Neurosci. Methods"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"20:1","DOI":"10.1145\/1314303.1314305","article-title":"Robust tracking and remapping of eye appearance with passive computer vision","volume":"3","author":"Colombo","year":"2007","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"17507","DOI":"10.3390\/s150717507","article-title":"Evaluation of fear using nonintrusive measurement of multimodal sensors","volume":"15","author":"Choi","year":"2015","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1007\/s10209-011-0256-6","article-title":"Eye-blink detection system for human\u2013computer interaction","volume":"11","year":"2012","journal-title":"Univ. Access. Inf. Soc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"033103","DOI":"10.1117\/1.OE.54.3.033103","article-title":"Segmentation method of eye region based on fuzzy logic system for classifying open and closed eyes","volume":"54","author":"Kim","year":"2015","journal-title":"Opt. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fuhl, W., Santini, T., Geisler, D., K\u00fcbler, T., Rosenstiel, W., and Kasneci, E. (2016, January 12\u201316). Eyes wide open? Eyelid location and eye aperture estimation for pervasive eye tracking in real-world scenarios. Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany.","DOI":"10.1145\/2968219.2968334"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fuhl, W., and Santini, T. (2017, January 24\u201331). Fast and robust eyelid outline and aperture detection in real-world scenarios. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.126"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1275","DOI":"10.1007\/s00138-016-0776-4","article-title":"Pupil detection for head-mounted eye tracking in the wild: An evaluation of the state of the art","volume":"27","author":"Fuhl","year":"2016","journal-title":"Mach. Vis. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Fuhl, W., Santini, T.C., K\u00fcbler, T., and Kasneci, E. (2016, January 14\u201317). ElSe: Ellipse selection for robust pupil detection in real-world environments. Proceedings of the 9th Biennial ACM Symposium on Eye Tracking Research & Applications, Charleston, SC, USA.","DOI":"10.1145\/2857491.2857505"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"127202","DOI":"10.1117\/1.3657506","article-title":"Vision-based method for detecting driver drowsiness and distraction in driver monitoring system","volume":"50","author":"Jo","year":"2011","journal-title":"Opt. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1312","DOI":"10.1109\/TCE.2008.4637622","article-title":"Statistical models of appearance for eye tracking and eye-blink detection and measurement","volume":"54","author":"Bacivarov","year":"2008","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"17:1","DOI":"10.1145\/2010325.2010327","article-title":"Modeling and animating eye blinks","volume":"8","author":"Trutoiu","year":"2011","journal-title":"ACM Trans. Appl. Percept."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Missimer, E., and Betke, M. (2010, January 23\u201325). Blink and wink detection for mouse pointer control. Proceedings of the 3rd International Conference on Pervasive Technologies Related to Assistive Environments, Samos, Greece.","DOI":"10.1145\/1839294.1839322"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Miluzzo, E., Wang, T., and Campbell, A.T. (2010, January 30). EyePhone: Activating mobile phones with your eyes. Proceedings of the Second ACM SIGCOMM Workshop on Networking, Systems, and Applications on Mobile Handhelds, New Delhi, India.","DOI":"10.1145\/1851322.1851328"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"8:1","DOI":"10.1145\/1671962.1671964","article-title":"An eye localization, tracking and blink pattern recognition system: Algorithm and evaluation","volume":"6","author":"Wu","year":"2010","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s12555-012-0212-0","article-title":"Driver\u2019s eye blinking detection using novel color and texture segmentation algorithms","volume":"10","author":"Lenskiy","year":"2012","journal-title":"Int. J. Control Autom. Syst."},{"key":"ref_29","unstructured":"Hoang, L., Thanh, D., and Feng, L. (2013, January 9\u201311). Eye blink detection for smart glasses. Proceedings of the IEEE International Symposium on Multimedia, Anaheim, CA, USA."},{"key":"ref_30","unstructured":"(2017, May 17). Histogram of Oriented Gradient. Available online: https:\/\/www.mathworks.com\/help\/vision\/ref\/extracthogfeatures.html."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pauly, L., and Sankar, D. (2015, January 20\u201322). Detection of drowsiness based on HOG features and SVM classifiers. Proceedings of the IEEE International Conference on Research in Computational Intelligence and Communication Networks, Kolkata, India.","DOI":"10.1109\/ICRCICN.2015.7434232"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1016\/j.compeleceng.2016.09.008","article-title":"Blink detection using Adaboost and contour circle for fatigue recognition","volume":"58","author":"Wang","year":"2017","journal-title":"Comput. Electr. Eng."},{"key":"ref_33","unstructured":"(2017, May 17). Multilayer Perceptron. Available online: http:\/\/deeplearning.net\/tutorial\/mlp.html."},{"key":"ref_34","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, Curran Associates, Inc."},{"key":"ref_35","unstructured":"(2017, May 17). Support Vector Machine. Available online: https:\/\/en.wikipedia.org\/wiki\/Support_vector_machine."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Parkhi, O.M., Vedaldi, A., and Zisserman, A. (2015, January 7\u201310). Deep face recognition. Proceedings of the British Machine Vision Conference, Swansea, UK.","DOI":"10.5244\/C.29.41"},{"key":"ref_37","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Reconition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Reconition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_40","unstructured":"(2017, May 17). Compute Image Mean. Available online: http:\/\/caffe.berkeleyvision.org\/gathered\/examples\/imagenet.html."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_42","unstructured":"Heaton, J. (2015). Artificial Intelligence for Humans, Volume 3: Deep Learning and Neural Networks, Heaton Research, Inc."},{"key":"ref_43","unstructured":"(2017, May 17). Softmax Function. Available online: https:\/\/en.wikipedia.org\/wiki\/Softmax_function."},{"key":"ref_44","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_45","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel."},{"key":"ref_46","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_47","unstructured":"(2017, May 17). Fully-Connected, Locally-Connected and Shared Weights Layer in Neural Networks Easy Explained. Available online: https:\/\/pennlio.wordpress.com\/2014\/04\/11\/fully-connected-locally-connected-and-shared-weights-layer-in-neural-networks\/."},{"key":"ref_48","unstructured":"(2017, May 17). Softmax Regression. Available online: http:\/\/ufldl.stanford.edu\/wiki\/index.php\/Softmax_Regression."},{"key":"ref_49","unstructured":"(2017, May 17). Stochastic Gradient Descent. Available online: https:\/\/en.wikipedia.org\/wiki\/Stochastic_gradient_descent."},{"key":"ref_50","unstructured":"(2017, May 17). TrainingOptions. Available online: http:\/\/kr.mathworks.com\/help\/nnet\/ref\/trainingoptions.html."},{"key":"ref_51","unstructured":"(2017, May 17). Dongguk Open and Closed Eyes Database (DOCE-DB1) & CNN Model. Available online: http:\/\/dm.dgu.edu\/link.html."},{"key":"ref_52","unstructured":"(2017, May 17). Webcam C600. Available online: http:\/\/www.logitech.com\/en-us\/support\/5869."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Pan, G., Sun, L., Wu, Z., and Lao, S. (2007, January 14\u201320). Eyeblink-based anti-spoofing in face recognition from a generic webcamera. Proceedings of the 11th IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil.","DOI":"10.1109\/ICCV.2007.4409068"},{"key":"ref_54","unstructured":"(2017, May 17). Geforce GTX 1070. Available online: https:\/\/www.nvidia.com\/en-us\/geforce\/products\/10series\/geforce-gtx-1070\/."},{"key":"ref_55","unstructured":"(2017, May 17). CUDA. Available online: https:\/\/en.wikipedia.org\/wiki\/CUDA."},{"key":"ref_56","unstructured":"(2017, February 17). Caffe. Available online: http:\/\/caffe.berkeleyvision.org."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1534\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:41:04Z","timestamp":1760208064000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1534"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,30]]},"references-count":56,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2017,7]]}},"alternative-id":["s17071534"],"URL":"https:\/\/doi.org\/10.3390\/s17071534","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2017,6,30]]}}}