{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T11:50:14Z","timestamp":1774612214611,"version":"3.50.1"},"reference-count":61,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T00:00:00Z","timestamp":1680048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software"],"abstract":"<jats:p>The last two years have seen a rapid rise in the duration of time that both adults and children spend on screens, driven by the recent COVID-19 health pandemic. A key adverse effect is digital eye strain (DES). Recent trends in human-computer interaction and user experience have proposed voice or gesture-guided designs that present more effective and less intrusive automated solutions. These approaches inspired the design of a solution that uses facial expression recognition (FER) techniques to detect DES and autonomously adapt the application to enhance the user\u2019s experience. This study sourced and adapted popular open FER datasets for DES studies, trained convolutional neural network models for DES expression recognition, and designed a self-adaptive solution as a proof of concept. Initial experimental results yielded a model with an accuracy of 77% and resulted in the adaptation of the user application based on the FER classification results. We also provide the developed application, model source code, and adapted dataset used for further improvements in the area. Future work should focus on detecting posture, ergonomics, or distance from the screen.<\/jats:p>","DOI":"10.3390\/software2020009","type":"journal-article","created":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T04:45:35Z","timestamp":1680151535000},"page":"197-217","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Vision-Autocorrect: A Self-Adapting Approach towards Relieving Eye-Strain Using Facial-Expression Recognition"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5164-6424","authenticated-orcid":false,"given":"Leah","family":"Mutanu","sequence":"first","affiliation":[{"name":"Department of Computing, United States International University Africa, Nairobi P.O. Box 14634-0800, Kenya"}]},{"given":"Jeet","family":"Gohil","sequence":"additional","affiliation":[{"name":"Department of Computing, United States International University Africa, Nairobi P.O. Box 14634-0800, Kenya"}]},{"given":"Khushi","family":"Gupta","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Sam Houston State University, Huntsville, TX 77341, USA"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,29]]},"reference":[{"key":"ref_1","unstructured":"Elsworthy, E. (2023, March 10). Average Adult Will Spend 34 Years of Their Life Looking at Screens, Poll Claims. Independent 2020. Available online: https:\/\/www.independent.co.uk\/life-style\/fashion\/news\/screen-time-average-lifetime-years-phone-laptop-tv-a9508751.html."},{"key":"ref_2","unstructured":"Nugent, A. (2020). UK adults spend 40% of their waking hours in front of a screen. Independent."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1709","DOI":"10.4103\/ijo.IJO_1782_20","article-title":"Digital eye strain in the era of COVID-19 pandemic: An emerging public health threat","volume":"68","author":"Bhattacharya","year":"2020","journal-title":"Indian J. Ophthalmol."},{"key":"ref_4","unstructured":"Siegel, R. (The Washington Post, 2019). Tweens, Teens and Screens: The Average Time Kids Spend Watching Online Videos Has Doubled in 4 Years, The Washington Post."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12193-018-0258-2","article-title":"Model-based adaptive user interface based on context and user experience evaluation","volume":"12","author":"Hussain","year":"2018","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1016\/j.ergon.2012.09.003","article-title":"A Universalist strategy for the design of Assistive Technology","volume":"42","author":"Plos","year":"2012","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_7","unstructured":"Firmenich, S., Garrido, A., Patern\u00f2, F., and Rossi, G. (2019). Web Accessibility, Springer."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sterritt, R., and Hinchey, M. (2010, January 22\u201326). SPAACE IV: Self-properties for an autonomous & autonomic computing environment\u2014Part IV A Newish Hope. Proceedings of the 2010 Seventh IEEE International Conference and Workshops on Engineering of Autonomic and Autonomous Systems, Oxford, UK.","DOI":"10.1109\/EASe.2010.29"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1333","DOI":"10.1080\/09500340600855460","article-title":"The physiology of eyestrain","volume":"54","author":"Sheedy","year":"2007","journal-title":"J. Mod. Opt."},{"key":"ref_10","first-page":"331","article-title":"Evaluation of the factors which contribute to the ocular complaints in computer users","volume":"7","author":"Agarwal","year":"2013","journal-title":"J. Clin. Diagn. Res. JCDR"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e000146","DOI":"10.1136\/bmjophth-2018-000146","article-title":"Digital eye strain: Prevalence, measurement and amelioration","volume":"3","author":"Sheppard","year":"2018","journal-title":"BMJ Open Ophthalmol."},{"key":"ref_12","first-page":"1","article-title":"Computer vision syndrome (aka digital eye strain)","volume":"17","author":"Rosenfield","year":"2016","journal-title":"Optom. Pract."},{"key":"ref_13","unstructured":"Dachapally, P.R. (2017). Facial emotion detection using convolutional neural networks and representational autoencoder units. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/s00371-019-01628-3","article-title":"Facial emotion detection using modified eyemap\u2013mouthmap algorithm on an enhanced image and classification with tensorflow","volume":"36","author":"Joseph","year":"2020","journal-title":"Vis. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s00371-019-01627-4","article-title":"Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy","volume":"36","author":"Li","year":"2020","journal-title":"Vis. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Huang, Y., Chen, F., Lv, S., and Wang, X. (2019). Facial expression recognition: A survey. Symmetry, 11.","DOI":"10.3390\/sym11101189"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"101404","DOI":"10.1016\/j.tele.2020.101404","article-title":"Facial-expression recognition: An emergent approach to the measurement of tourist satisfaction through emotions","volume":"51","year":"2020","journal-title":"Telemat. Inform."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Generosi, A., Ceccacci, S., and Mengoni, M. (2018, January 2\u20135). A deep learning-based system to track and analyze customer behavior in retail store. Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin), Berlin, Germany.","DOI":"10.1109\/ICCE-Berlin.2018.8576169"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bouzakraoui, M.S., Sadiq, A., and Enneya, N. (December, January 29). Towards a framework for customer emotion detection. Proceedings of the 2016 IEEE\/ACS 13th International Conference of Computer Systems and Applications (AICCSA), Agadir, Morocco.","DOI":"10.1109\/AICCSA.2016.7945830"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2121","DOI":"10.1016\/j.neuropsychologia.2012.05.020","article-title":"Structural correlates of facial emotion recognition deficits in Parkinson\u2019s disease patients","volume":"50","author":"Baggio","year":"2012","journal-title":"Neuropsychologia"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1016\/j.biopsych.2009.01.026","article-title":"Association of impaired facial affect recognition with basic facial and visual processing deficits in schizophrenia","volume":"65","author":"Norton","year":"2009","journal-title":"Biol. Psychiatry"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"67459","DOI":"10.1109\/ACCESS.2018.2878601","article-title":"An effective framework for driver fatigue recognition based on intelligent facial expressions analysis","volume":"6","author":"Khan","year":"2018","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1049\/iet-its.2018.5392","article-title":"Fatigue driving recognition network: Fatigue driving recognition via convolutional neural network and long short-term memory units","volume":"13","author":"Xiao","year":"2019","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Munasinghe, M. (2018, January 6\u20138). Facial expression recognition using facial landmarks and random forest classifier. Proceedings of the 2018 IEEE\/ACIS 17th International Conference on Computer and Information Science (ICIS), Singapore.","DOI":"10.1109\/ICIS.2018.8466510"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1080\/13682199.2018.1553343","article-title":"Detection of eye strain through blink rate and sclera area using raspberry-pi","volume":"67","author":"Reddy","year":"2019","journal-title":"Imaging Sci. J."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lim, J.Z., Mountstephens, J., and Teo, J. (2020). Emotion recognition using eye-tracking: Taxonomy, review and current challenges. Sensors, 20.","DOI":"10.3390\/s20082384"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"114037","DOI":"10.1016\/j.eswa.2020.114037","article-title":"Eye tracking algorithms, techniques, tools, and applications with an emphasis on machine learning and Internet of Things technologies","volume":"166","author":"Klaib","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1109\/TAFFC.2020.2981446","article-title":"Deep facial expression recognition: A survey","volume":"13","author":"Li","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, S.Z., and Jain, A.K. (2011). Handbook of Face Recognition, Springer.","DOI":"10.1007\/978-0-85729-932-1"},{"key":"ref_30","unstructured":"Kanade, T., Cohn, J.F., and Tian, Y. (2000, January 28\u201330). Comprehensive database for facial expression analysis. Proceedings of the fourth IEEE International Conference on Automatic Face and Gesture Recognition (cat. No. PR00580), Grenoble, France."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_32","unstructured":"Goodfellow, I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., and Lee, D.H. (2013, January 3\u20137). Challenges in representation learning: A report on three machine learning contests. Proceedings of the Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Korea. Proceedings, Part III 20."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Petrou, M.M., and Petrou, C. (2010). Image Processing: The Fundamentals, John Wiley & Sons.","DOI":"10.1002\/9781119994398"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Russ, J.C. (2006). The Image Processing Handbook, CRC Press.","DOI":"10.1201\/9780203881095"},{"key":"ref_35","unstructured":"Joshi, P. (2015). OpenCV with Python by Example, Packt Publishing Ltd."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e453","DOI":"10.7717\/peerj.453","article-title":"scikit-image: Image processing in Python","volume":"2","author":"Boulogne","year":"2014","journal-title":"PeerJ"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liliana, D.Y., and Basaruddin, T. (2018, January 2\u20134). Review of automatic emotion recognition through facial expression analysis. Proceedings of the 2018 International Conference on Electrical Engineering and Computer Science (ICECOS), Pangkal, Indonesia.","DOI":"10.1109\/ICECOS.2018.8605222"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1016\/j.patcog.2016.07.026","article-title":"Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order","volume":"61","author":"Lopes","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"64827","DOI":"10.1109\/ACCESS.2019.2917266","article-title":"Local learning with deep and handcrafted features for facial expression recognition","volume":"7","author":"Georgescu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Pecoraro, R., Basile, V., and Bono, V. (2022). Local multi-head channel self-attention for facial expression recognition. Information, 13.","DOI":"10.3390\/info13090419"},{"key":"ref_41","unstructured":"Aouayeb, M., Hamidouche, W., Soladie, C., Kpalma, K., and Seguier, R. (2021). Learning vision transformer with squeeze and excitation for facial expression recognition. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Meng, D., Peng, X., Wang, K., and Qiao, Y. (2019, January 22\u201325). Frame attention networks for facial expression recognition in videos. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8803603"},{"key":"ref_43","unstructured":"Pourmirzaei, M., Montazer, G.A., and Esmaili, F. (2021). Using self-supervised auxiliary tasks to improve fine-grained facial representation. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Gacav, C., Benligiray, B., and Topal, C. (2017, January 5\u20139). Greedy search for descriptive spatial face features. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952406"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Antoniadis, P., Filntisis, P.P., and Maragos, P. (2021, January 15\u201318). Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition. Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), Jodhpur, India. Available online: http:\/\/xxx.lanl.gov\/abs\/2106.03487.","DOI":"10.1109\/FG52635.2021.9667014"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.neucom.2022.10.013","article-title":"In search of a robust facial expressions recognition model: A large-scale visual cross-corpus study","volume":"514","author":"Ryumina","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2132","DOI":"10.1109\/TAFFC.2022.3188390","article-title":"Classifying emotions and engagement in online learning based on a single facial expression recognition neural network","volume":"13","author":"Savchenko","year":"2022","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_48","unstructured":"Tan, M., and Le, Q. (2019, January 15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_49","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_50","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Minhas, R.A., Javed, A., Irtaza, A., Mahmood, M.T., and Joo, Y.B. (2019). Shot classification of field sports videos using AlexNet Convolutional Neural Network. Appl. Sci., 9.","DOI":"10.3390\/app9030483"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_53","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Liu, S., and Deng, W. (2015, January 3\u20136). Very deep convolutional neural network based image classification using small training sample size. Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ACPR.2015.7486599"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1007\/s11263-020-01396-x","article-title":"Progressive darts: Bridging the optimization gap for nas in the wild","volume":"129","author":"Chen","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_57","first-page":"171","article-title":"Amended convolutional neural network with global average pooling for image classification","volume":"Volume 1351","author":"Abraham","year":"2020","journal-title":"Intelligent Systems Design and Applications. ISDA 2020. Advances in Intelligent Systems and Computing"},{"key":"ref_58","first-page":"535","article-title":"G-ResNet: Improved ResNet for brain tumor classification","volume":"Volume 11953","author":"Gedeon","year":"2019","journal-title":"Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/TAFFC.2019.2900240","article-title":"Capturing emotion distribution for multimedia emotion tagging","volume":"12","author":"Wang","year":"2019","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_60","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"151","DOI":"10.13005\/ojcst\/10.01.20","article-title":"Comparison of viola-jones and kanade-lucas-tomasi face detection algorithms","volume":"10","author":"Vijayalakshmi","year":"2017","journal-title":"Orient. J. Comput. Sci. Technol."}],"container-title":["Software"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2674-113X\/2\/2\/9\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:06:32Z","timestamp":1760123192000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2674-113X\/2\/2\/9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,29]]},"references-count":61,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["software2020009"],"URL":"https:\/\/doi.org\/10.3390\/software2020009","relation":{},"ISSN":["2674-113X"],"issn-type":[{"value":"2674-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,29]]}}}