{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,26]],"date-time":"2026-07-26T04:00:48Z","timestamp":1785038448122,"version":"3.55.0"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,11]],"date-time":"2024-12-11T00:00:00Z","timestamp":1733875200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Facial emotion recognition (FER) can serve as a valuable tool for assessing emotional states, which are often linked to mental health. However, mental health encompasses a broad range of factors that go beyond facial expressions. While FER provides insights into certain aspects of emotional well-being, it can be used in conjunction with other assessments to form a more comprehensive understanding of an individual's mental health. This research work proposes a framework for human FER using UNet image segmentation and transfer learning with the EfficientNetB4 model (called FacialNet). The proposed model demonstrates promising results, achieving an accuracy of 90% for six emotion classes (happy, sad, fear, pain, anger, and disgust) and 96.39% for binary classification (happy and sad). The significance of FacialNet is judged by extensive experiments conducted against various machine learning and deep learning models, as well as state-of-the-art previous research works in FER. The significance of FacialNet is further validated using a cross-validation technique, ensuring reliable performance across different data splits. The findings highlight the effectiveness of leveraging UNet image segmentation and EfficientNetB4 transfer learning for accurate and efficient human facial emotion recognition, offering promising avenues for real-world applications in emotion-aware systems and effective computing platforms. Experimental findings reveal that the proposed approach performs substantially better than existing works with an improved accuracy of 96.39% compared to existing 94.26%.<\/jats:p>","DOI":"10.3389\/fncom.2024.1485121","type":"journal-article","created":{"date-parts":[[2024,12,11]],"date-time":"2024-12-11T12:42:06Z","timestamp":1733920926000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["FacialNet: facial emotion recognition for mental health analysis using UNet segmentation with transfer learning model"],"prefix":"10.3389","volume":"18","author":[{"given":"In-seop","family":"Na","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asma","family":"Aldrees","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abeer","family":"Hakeem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linda","family":"Mohaisen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammad","family":"Umer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dina Abdulaziz","family":"AlHammadi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shtwai","family":"Alsubai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nisreen","family":"Innab","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Imran","family":"Ashraf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,12,11]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11042-024-18206-y","article-title":"Design an image-based sentiment analysis system using a deep convolutional neural network and hyperparameter optimization","volume":"83","author":"Anilkumar","year":"2024","journal-title":"Multimed. Tools Appl"},{"key":"B2","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1007\/s12530-021-09393-2","article-title":"A novel approach for facial expression recognition based on gabor filters and genetic algorithm","volume":"13","author":"Boughida","year":"2022","journal-title":"Evol. Syst"},{"key":"B3","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1016\/j.ins.2021.10.005","article-title":"A survey on facial emotion recognition techniques: a state-of-the-art literature review","volume":"582","author":"Canal","year":"2022","journal-title":"Inf. Sci"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1017\/S0954579424000725","article-title":"Childhood maltreatment and resting-state network connectivity: the risk-buffering role of positive parenting","author":"Cao","year":"2024","journal-title":"Dev. Psychopathol"},{"key":"B5","doi-asserted-by":"publisher","first-page":"20481","DOI":"10.1007\/s11227-023-05439-1","article-title":"Dialogueinab: an interaction neural network based on attitudes and behaviors of interlocutors for dialogue emotion recognition","volume":"79","author":"Ding","year":"2023","journal-title":"J. Supercomput"},{"key":"B6","doi-asserted-by":"publisher","first-page":"102422","DOI":"10.1016\/j.mex.2023.102422","article-title":"Adaboost based random forest model for emotion classification of facial images","volume":"11","author":"Gubbala","year":"2023","journal-title":"MethodsX"},{"key":"B7","doi-asserted-by":"publisher","first-page":"012053","DOI":"10.1088\/1742-6596\/1717\/1\/012053","article-title":"Feeling recognition by facial expression using deep learning","volume":"1717","author":"Gupta","year":"2021","journal-title":"J. Phys. Conf. Ser"},{"key":"B8","doi-asserted-by":"publisher","first-page":"4770","DOI":"10.3390\/s23104770","article-title":"Robust human face emotion classification using triplet-loss-based deep cnn features and SVM","volume":"23","author":"Haider","year":"2023","journal-title":"Sensors"},{"key":"B9","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/5254.708428","article-title":"Support vector machines","volume":"13","author":"Hearst","year":"1998","journal-title":"IEEE Intell. Syst. Appl"},{"key":"B10","first-page":"1","article-title":"\u201cHuman face sentiment classification using synthetic sentiment images with deep convolutional neural networks,\u201d","volume-title":"2019 International Conference on Machine Learning and Cybernetics (ICMLC)","author":"Huang","year":"2019"},{"key":"B11","doi-asserted-by":"publisher","first-page":"8425","DOI":"10.1038\/s41598-023-35446-4","article-title":"A study on computer vision for facial emotion recognition","volume":"13","author":"Huang","year":"2023","journal-title":"Sci. Rep"},{"key":"B12","doi-asserted-by":"publisher","first-page":"3064","DOI":"10.3390\/diagnostics12123064","article-title":"Review of semantic segmentation of medical images using modified architectures of unet","volume":"12","author":"Krithika Alias AnbuDevi","year":"2022","journal-title":"Diagnostics"},{"key":"B13","doi-asserted-by":"publisher","first-page":"669","DOI":"10.1038\/s44159-022-00105-4","article-title":"The cultural evolution of emotion","volume":"1","author":"Lindquist","year":"2022","journal-title":"Nat. Rev. Psychol"},{"key":"B14","doi-asserted-by":"publisher","first-page":"707","DOI":"10.3390\/agriculture11080707","article-title":"Review on convolutional neural network (cnn) applied to plant leaf disease classification","volume":"11","author":"Lu","year":"2021","journal-title":"Agriculture"},{"key":"B15","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW.2010.5543262","article-title":"\u201cThe extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression,\u201d","volume-title":"Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops","author":"Lucey","year":"2010"},{"key":"B16","doi-asserted-by":"publisher","first-page":"128359","DOI":"10.1109\/ACCESS.2021.3112546","article-title":"Rfcnn: traffic accident severity prediction based on decision level fusion of machine and deep learning model","volume":"9","author":"Manzoor","year":"2021","journal-title":"IEEE Access"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1640","DOI":"10.1016\/j.procs.2023.01.142","article-title":"Sentiment analysis on images using different transfer learning models","volume":"218","author":"Meena","year":"2023","journal-title":"Procedia Comput. Sci"},{"key":"B18","unstructured":"MohamedY.\n          6 Human Emotions for image classification2024"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1280","DOI":"10.3390\/diagnostics12051280","article-title":"Pneumonia classification from X-ray images with inception-v3 and convolutional neural network","volume":"12","author":"Mujahid","year":"2022","journal-title":"Diagnostics"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.4850\/arXiv.2206.09509","article-title":"Hybrid facial expression recognition (fer2013) model for real-time emotion classification and prediction","author":"Oguine","year":"2022","journal-title":"arXiv"},{"key":"B21","doi-asserted-by":"publisher","first-page":"742","DOI":"10.3390\/diagnostics12030742","article-title":"Deep learning model based on 3d optical coherence tomography images for the automated detection of pathologic myopia","volume":"12","author":"Park","year":"2022","journal-title":"Diagnostics"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.10818","article-title":"Facial emotion recognition using cnn in pytorch","author":"Qu","year":"2023","journal-title":"arXiv"},{"key":"B23","doi-asserted-by":"publisher","first-page":"3429","DOI":"10.3390\/app10103429","article-title":"A customized vgg19 network with concatenation of deep and handcrafted features for brain tumor detection","volume":"10","author":"Rajinikanth","year":"2020","journal-title":"Appl. Sci"},{"key":"B24","doi-asserted-by":"publisher","first-page":"3400","DOI":"10.3390\/s19153400","article-title":"Logistic regression for machine learning in process tomography","volume":"19","author":"Rymarczyk","year":"2019","journal-title":"Sensors"},{"key":"B25","doi-asserted-by":"publisher","first-page":"3560","DOI":"10.1016\/j.matpr.2021.07.297","article-title":"Facial emotion recognition using convolutional neural networks","volume":"80","author":"Sarvakar","year":"2023","journal-title":"Mater. Today Proc"},{"key":"B26","doi-asserted-by":"publisher","first-page":"7397","DOI":"10.1109\/TVCG.2024.3456213","article-title":"Expressive 3d facial animation generation based on local-to-global latent diffusion","volume":"30","author":"Song","year":"2024","journal-title":"IEEE Trans. Vis. Comput. Graph"},{"key":"B27","doi-asserted-by":"publisher","first-page":"2852","DOI":"10.3390\/s21082852","article-title":"Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm","volume":"21","author":"Srinivasu","year":"2021","journal-title":"Sensors"},{"key":"B28","doi-asserted-by":"publisher","first-page":"12717","DOI":"10.1007\/s00521-023-08372-9","article-title":"Real-time facial emotion recognition system among children with autism based on deep learning and iot","volume":"35","author":"Talaat","year":"2023","journal-title":"Neural Comput. Appl"},{"key":"B29","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1016\/j.patrec.2022.11.012","article-title":"ETCNN: extra tree and convolutional neural network-based ensemble model for covid-19 tweets sentiment classification","volume":"164","author":"Umer","year":"2022","journal-title":"Pattern Recognit. Lett"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1777","DOI":"10.1007\/s41870-023-01184-z","article-title":"A novel facial emotion recognition model using segmentation vgg-19 architecture","volume":"15","author":"Vignesh","year":"2023","journal-title":"Int. J. Inf. Technol"},{"key":"B31","doi-asserted-by":"publisher","first-page":"3883","DOI":"10.3390\/s21113883","article-title":"Resnet based deep features and random forest classifier for diabetic retinopathy detection","volume":"21","author":"Yaqoob","year":"2021","journal-title":"Sensors"},{"key":"B32","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.inffus.2020.01.011","article-title":"Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review","volume":"59","author":"Zhang","year":"2020","journal-title":"Inf. Fusion"},{"key":"B33","doi-asserted-by":"publisher","first-page":"227","DOI":"10.33168\/JSMS.2023.0515","article-title":"Research on emotion recognition-based smart assistant system: Emotional intelligence and personalized services","volume":"13","author":"Zhu","year":"2023","journal-title":"J. Syst. Manag. Sci"},{"key":"B34","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1186\/s13677-024-00675-z","article-title":"Computational intelligence-based classification system for the diagnosis of memory impairment in psychoactive substance users","volume":"13","author":"Zhu","year":"2024","journal-title":"J. Cloud Comput"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1485121\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T13:33:01Z","timestamp":1734355981000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1485121\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,11]]},"references-count":34,"alternative-id":["10.3389\/fncom.2024.1485121"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1485121","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,11]]},"article-number":"1485121"}}