{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T18:07:25Z","timestamp":1772042845942,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T00:00:00Z","timestamp":1737244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Blink detection is considered a useful indicator both for clinical conditions and drowsiness state. In this work, we propose and compare deep learning architectures for the task of detecting blinks in video frame sequences. The first step is the training and application of an eye detector that extracts the eye regions from each video frame. The cropped eye regions are organized as three-dimensional (3D) input with the third dimension spanning time of 300 ms. Two different 3D convolutional neural networks are utilized (a simple 3D CNN and 3D ResNet), as well as a 3D autoencoder combined with a classifier coupled to the latent space. Finally, we propose the usage of a frame prediction accumulator combined with morphological processing and watershed segmentation to detect blinks and determine their start and stop frame in previously unseen videos. The proposed framework was trained on ten (9) different participants and tested on five (8) different ones, with a total of 162,400 frames and 1172 blinks for each eye. The start and end frame of each blink in the dataset has been annotate by specialized ophthalmologist. Quantitative comparison with state-of-the-art blink detection methodologies provide favorable results for the proposed neural architectures coupled with the prediction accumulator, with the 3D ResNet being the best as well as the fastest performer.<\/jats:p>","DOI":"10.3390\/jimaging11010027","type":"journal-article","created":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T07:47:37Z","timestamp":1737359257000},"page":"27","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Blink Detection Using 3D Convolutional Neural Architectures and Analysis of Accumulated Frame Predictions"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5571-4292","authenticated-orcid":false,"given":"George","family":"Nousias","sequence":"first","affiliation":[{"name":"Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1055-3007","authenticated-orcid":false,"given":"Konstantinos K.","family":"Delibasis","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4364-1943","authenticated-orcid":false,"given":"Georgios","family":"Labiris","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, General University Hospital of Alexandroupolis, 68131 Alexandroupolis, Greece"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.cviu.2004.07.010","article-title":"Eye gaze tracking techniques for interactive applications","volume":"98","author":"Morimoto","year":"2005","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1177\/001872089403600209","article-title":"Blink rate: A possible measure of fatigue","volume":"36","author":"Stern","year":"1994","journal-title":"Hum. Factors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.ijpsycho.2017.11.009","article-title":"Spontaneous eye blink rate: An index of dopaminergic component of sustained attention and fatigue","volume":"123","author":"Maffei","year":"2018","journal-title":"Int. J. Psychophysiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2784","DOI":"10.1152\/jn.00557.2002","article-title":"Eyelid movements: Behavioral studies of blinking in humans under different stimulus conditions","volume":"89","author":"VanderWerf","year":"2003","journal-title":"J. Neurophysiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/S1542-0124(11)70007-6","article-title":"Spontaneous eyeblink activity","volume":"9","author":"Cruz","year":"2011","journal-title":"Ocul. Surf."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1001\/archopht.1997.01100150633010","article-title":"The role of blink adaptation in the pathophysiology of benign essential blepharospasm","volume":"115","author":"Hasan","year":"1997","journal-title":"Arch. Ophthalmol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.jns.2017.07.035","article-title":"Measurement of spontaneous blinks in patients with Parkinson\u2019s disease using a new high-speed blink analysis system","volume":"380","author":"Kimura","year":"2017","journal-title":"J. Neurol. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1097\/SCS.0000000000008326","article-title":"Comparative blink analysis in patients with established facial paralysis using high-speed video analysis","volume":"33","author":"Ogawa","year":"2022","journal-title":"J. Craniofacial Surg."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1001\/archpsyc.1980.01780250036004","article-title":"Clonidine ameliorates Gilles de la Tourette syndrome","volume":"37","author":"Cohen","year":"1980","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1007\/s00417-019-04567-w","article-title":"Analysis of blink activity and anomalous eyelid movements in patients with hemifacial spasm","volume":"258","author":"Osaki","year":"2020","journal-title":"Graefe\u2019s Arch. Clin. Exp. Ophthalmol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.jtos.2017.05.008","article-title":"TFOS DEWS II definition and classification report","volume":"15","author":"Craig","year":"2017","journal-title":"Ocul. Surf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1176\/ajp.135.2.223","article-title":"Eye blink and schizophrenia: Psychosis or tardive dyskinesia?","volume":"135","author":"Stevens","year":"1978","journal-title":"Am. J. Psychiatry"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.smrv.2013.03.003","article-title":"Look before you (s) leep: Evaluating the use of fatigue detection technologies within a fatigue risk management system for the road transport industry","volume":"18","author":"Dawson","year":"2014","journal-title":"Sleep Med. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ezzat, M., Maged, M., Gamal, Y., Adel, M., Alrahmawy, M., and El-Metwally, S. (2023). Blink-To-Live eye-based communication system for users with speech impairments. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-34310-9"},{"key":"ref_15","unstructured":"Soukupova, T., and Cech, J. (2016, January 3\u20135). Eye blink detection using facial landmarks. Proceedings of the 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia."},{"key":"ref_16","unstructured":"Ge, Z. (2021). Yolox: Exceeding yolo series in 2021. arXiv."},{"key":"ref_17","unstructured":"Kartynnik, Y., Ablavatski, A., Grishchenko, I., and Grundmann, M. (2019). Real-time facial surface geometry from monocular video on mobile GPUs. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1023\/B:VISI.0000013087.49260.fb","article-title":"Robust real-time face detection","volume":"57","author":"Viola","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_19","first-page":"2852","article-title":"Eye blink detection based on multiple Gabor response waves","volume":"Volume 5","author":"Li","year":"2008","journal-title":"Proceedings of the 2008 International Conference on Machine Learning and Cybernetics"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Pauly, L., and Sankar, D. (2015, January 2\u20133). A novel method for eye tracking and blink detection in video frames. Proceedings of the 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS), Bhubaneshwar, India.","DOI":"10.1109\/CGVIS.2015.7449931"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1007\/s10209-011-0256-6","article-title":"Eye-blink detection system for human\u2013computer interaction","volume":"11","year":"2012","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Choi, I., Han, S., and Kim, D. (August, January 31). Eye detection and eye blink detection using adaboost learning and grouping. Proceedings of the 2011 20th International Conference on Computer Communications and Networks (ICCCN), Maui, HI, USA.","DOI":"10.1109\/ICCCN.2011.6005896"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Al-gawwam, S., and Benaissa, M. (2018). Robust eye blink detection based on eye landmarks and Savitzky\u2013Golay filtering. Information, 9.","DOI":"10.3390\/info9040093"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Drutarovsky, T., and Fogelton, A. (2014, January 6\u201312). Eye blink detection using variance of motion vectors. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-16199-0_31"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.cviu.2016.03.011","article-title":"Eye blink detection based on motion vectors analysis","volume":"148","author":"Fogelton","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_26","unstructured":"Farneb\u00e4ck, G. (July, January 29). Two-frame motion estimation based on polynomial expansion. Proceedings of the Image Analysis: 13th Scandinavian Conference, SCIA 2003, Halmstad, Sweden. Proceedings 13."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.cviu.2018.09.006","article-title":"Eye blink completeness detection","volume":"176","author":"Fogelton","year":"2018","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_28","first-page":"5130","article-title":"Eye-lrcn: A long-term recurrent convolutional network for eye blink completeness detection","volume":"35","author":"Lira","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3284","DOI":"10.1109\/JBHI.2022.3153407","article-title":"Video-based eye blink identification and classification","volume":"26","author":"Nousias","year":"2022","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"109871","DOI":"10.1016\/j.patcog.2023.109871","article-title":"Two phase cooperative learning for supervised dimensionality reduction","volume":"144","author":"Nellas","year":"2023","journal-title":"Pattern Recognit."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Beucher, S. (1994). Watershed, hierarchical segmentation and waterfall algorithm. Mathematical Morphology and Its Applications to Image Processing, Springer.","DOI":"10.1007\/978-94-011-1040-2_10"},{"key":"ref_34","unstructured":"(2021, March 31). Talking Face Video. Available online: https:\/\/personalpages.manchester.ac.uk\/staff\/Timothy.F.Cootes\/data\/talking_face\/talking_face.html."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/1\/27\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:31:39Z","timestamp":1759919499000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/1\/27"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,19]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["jimaging11010027"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11010027","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,19]]}}}