{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:51:40Z","timestamp":1777697500453,"version":"3.51.4"},"reference-count":36,"publisher":"SAGE Publications","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDT"],"published-print":{"date-parts":[[2022,6,17]]},"abstract":"<jats:p>A cognitive-analysis of facial features can make facial expression recognition system more robust and efficient for Human-Machine Interaction (HMI) applications. Through this work, we propose a new methodology to improve accuracy of facial expression recognition system even with the constraints like partial hidden faces or occlusions for real time applications. As a first step, seven independent facial segments: Full-Face, half-face (left\/right), upper half face, lower half face, eyes, mouth and nose are considered to recognize facial expression. Unlike the work reported in literature, where arbitrarily generated patch type occlusions on facial regions are used, in this work a detailed analysis of each facial feature is explored. Using the results thus obtained, these seven sub models are combined using a Stacked Generalized ensemble method with deep neural network as meta-learner to improve accuracy of facial expression recognition system even in occluded state. The accuracy of the proposed model improved up to 30% compared to individual model accuracies for cross-corpus seven model datasets. The proposed system uses CNN with RPA compliance and is also configured on Raspberry Pi, which can be used for HRI and Industry 4.0 applications which involve face occlusion and partially hidden face challenges.<\/jats:p>","DOI":"10.3233\/idt-210249","type":"journal-article","created":{"date-parts":[[2022,5,24]],"date-time":"2022-05-24T11:51:16Z","timestamp":1653393076000},"page":"457-473","source":"Crossref","is-referenced-by-count":0,"title":["Facial expression recognition under constrained conditions using stacked generalized convolution neural network"],"prefix":"10.1177","volume":"16","author":[{"given":"Suchitra","family":"Saxena","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shikha","family":"Tripathi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sudarshan","family":"T S B","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/IDT-210249_ref1","doi-asserted-by":"publisher","DOI":"10.1109\/T4E.2013.19"},{"key":"10.3233\/IDT-210249_ref2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-642-42051-1_16","article-title":"Challenges in Representation Learning: A Report on Three Machine Learning Contests","author":"Goodfellow","year":"2013","journal-title":"Workshop Challenges in Representation Learning (ICM12013)"},{"key":"10.3233\/IDT-210249_ref3","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00052-1"},{"issue":"4","key":"10.3233\/IDT-210249_ref4","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.neunet.2005.03.006","article-title":"Emotion Recognition in Human\u2013Computer Interaction","volume":"18","author":"Fragopanagos","year":"2005","journal-title":"Neural Networks, Elsevier"},{"issue":"3","key":"10.3233\/IDT-210249_ref5","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1016\/jchb.2012.10.016","article-title":"Using non-verbal cues to (automatically) assess children\u2019s performance difficulties with arithmetic problems","volume":"29","author":"Amelsvoort","year":"2013","journal-title":"Computers in Human Behavior"},{"issue":"4","key":"10.3233\/IDT-210249_ref6","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1109\/TCDS.2017.2783684","article-title":"Study of mechanisms of social interaction stimulation in autism spectrum disorder by assisted humanoid robot","volume":"10","author":"Coco","year":"2018","journal-title":"IEEE Transactions on Cognitive and Developmental Systems"},{"issue":"3","key":"10.3233\/IDT-210249_ref7","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1109\/TCDS.2017.2721552","article-title":"Artificial intelligent system for automatic depression level analysis through visual and vocal expressions","volume":"10","author":"Jan","year":"2018","journal-title":"IEEE Transactions on Cognitive and Developmental Systems"},{"key":"10.3233\/IDT-210249_ref8","first-page":"1","article-title":"Tsihrintzis: Visual affect recognition","author":"Ioanna-Ourania","journal-title":"Frontiers in Artificial Intelligence and Applications 214"},{"issue":"3","key":"10.3233\/IDT-210249_ref9","first-page":"350","article-title":"Tsihrintzis, Maria Virvou: On assisting a visual-facial affect recognition system with keyboard-stroke pattern information","volume":"23","author":"Ioanna-Ourania","year":"2010","journal-title":"Knowl. Based Syst."},{"key":"10.3233\/IDT-210249_ref10","doi-asserted-by":"crossref","unstructured":"Darwin C. The expression of emotions in man and animals. John Murray, reprinted by Universityof Chicago Press, 1965; 1872.","DOI":"10.1037\/10001-000"},{"key":"10.3233\/IDT-210249_ref12","doi-asserted-by":"publisher","first-page":"4150","DOI":"10.1007\/s10489-019-01500-w","article-title":"Detecting facial emotions using normalized minimal feature vectors and semi-supervised twin support vector machines classifier","volume":"49","author":"Kumar","year":"2019","journal-title":"Appl Intell."},{"key":"10.3233\/IDT-210249_ref13","first-page":"5039","article-title":"Lips Don\u2019t Lie: A Generalisable and Robust Approach To Face Forgery Detection","author":"Alexandros","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"10.3233\/IDT-210249_ref14","unstructured":"Alexandros H, Rodrigo M, Stavros P, Maja P. Leveraging real talking faces via self-supervision for robust forgery detection, arXiv preprint arXiv: 220107131, 2022."},{"key":"10.3233\/IDT-210249_ref15","doi-asserted-by":"crossref","unstructured":"Gross R, Matthews I, Cohn J, Kanade T, Baker S. Multi-PIE. Proc Int Conf Autom Face Gesture Recognit. 2010; 28(5): 807-813.","DOI":"10.1016\/j.imavis.2009.08.002"},{"key":"10.3233\/IDT-210249_ref17","doi-asserted-by":"crossref","unstructured":"Ekman P. and Friesen W. The Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, San Francisco, 1978.","DOI":"10.1037\/t27734-000"},{"issue":"2","key":"10.3233\/IDT-210249_ref18","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1109\/TSMCB.2005.859075","article-title":"Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences","volume":"36","author":"Pantic","year":"2006","journal-title":"in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)"},{"issue":"7","key":"10.3233\/IDT-210249_ref19","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1016\/j.imavis.2007.11.004","article-title":"An analysis of facial expression recognition under partial facial image oc-clusion","volume":"26","author":"Kotsia","year":"2008","journal-title":"Image and Vision Computing."},{"key":"10.3233\/IDT-210249_ref20","doi-asserted-by":"crossref","unstructured":"Cotter SF. Recognition of occluded facial expressions using a Fusion of Localized Sparse Representation Classifiers. IEEE Digital Signal Processing Workshop and IEEE Signal Processing Education Workshop, 2011; pp.\u00a0437-442.","DOI":"10.1109\/DSP-SPE.2011.5739254"},{"key":"10.3233\/IDT-210249_ref21","doi-asserted-by":"crossref","unstructured":"Cheng Y, et al. A Deep Structure for Facial Expression Recognition under Partial Occlusion. Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2014; pp.\u00a0211-214.","DOI":"10.1109\/IIH-MSP.2014.59"},{"key":"10.3233\/IDT-210249_ref22","doi-asserted-by":"crossref","unstructured":"Liu S, et al. Facial expression recognition under partial occlusion based on Weber Local Descriptor histogram and decision fusion. 33rd Chinese Control Conference, 2014, pp.\u00a04664-4668.","DOI":"10.1109\/ChiCC.2014.6895725"},{"issue":"2","key":"10.3233\/IDT-210249_ref23","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1109\/TCDS.2019.2917711","article-title":"Facial Expression Recognition via Deep Action Units Graph Network Based on Psychological Mechanism","volume":"12","author":"Liu","year":"2020","journal-title":"IEEE Transactions on Cognitive and Developmental Systems"},{"key":"10.3233\/IDT-210249_ref24","doi-asserted-by":"publisher","DOI":"10.1109\/NCC48643.2020.9056011"},{"key":"10.3233\/IDT-210249_ref25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/FCV.2015.7103729","article-title":"Development of Deep Learning-based Facial Expression Recognition System","author":"Jung","journal-title":"21st Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV)"},{"key":"10.3233\/IDT-210249_ref26","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2015.7457876"},{"key":"10.3233\/IDT-210249_ref27","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.07.233"},{"key":"10.3233\/IDT-210249_ref28","doi-asserted-by":"publisher","first-page":"4630","DOI":"10.1109\/ACCESS.2017.2784096","article-title":"Facial expression recognition using weighted mixture deep neural network based on double-channel facial images","author":"Yang","journal-title":"IEEE Access"},{"key":"10.3233\/IDT-210249_ref29","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594276"},{"key":"10.3233\/IDT-210249_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8967874"},{"key":"10.3233\/IDT-210249_ref31","doi-asserted-by":"publisher","first-page":"7183","DOI":"10.1109\/ACCESS.2020.2964298","article-title":"ROI-Attention Vectorized CNN Model for Static Facial Expression Recognition","author":"Sun","year":"2020","journal-title":"in IEEE Access"},{"key":"10.3233\/IDT-210249_ref32","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-020-01855-5"},{"key":"10.3233\/IDT-210249_ref33","doi-asserted-by":"publisher","first-page":"106172","DOI":"10.1016\/j.knosys.2020.106172","article-title":"Deep reinforcement learning for robust emotional classification in facial expression recognition","author":"Li","journal-title":"Knowledge-Based Systems"},{"key":"10.3233\/IDT-210249_ref34","doi-asserted-by":"publisher","first-page":"5543","DOI":"10.1007\/s10489-020-02125-0","article-title":"EmNet: a deep integrated convolutional neural network for facial emotion recognition in the wild","volume":"51","author":"Saurav","year":"2021","journal-title":"Appl Intell."},{"issue":"2","key":"10.3233\/IDT-210249_ref35","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1007\/s00426-016-0829-3","article-title":"What makes a smiling face look happy? Visual saliency, distinctiveness, and affect","volume":"82","author":"Calvo","year":"2018","journal-title":"Psychol Res."},{"key":"10.3233\/IDT-210249_ref36","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-27257-7_12"},{"key":"10.3233\/IDT-210249_ref37","unstructured":"Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift, in International conference on machine learning, 2015, pp.\u00a0448-456."},{"issue":"2","key":"10.3233\/IDT-210249_ref38","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"David","year":"1992","journal-title":"Neural Networks"}],"container-title":["Intelligent Decision Technologies"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDT-210249","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:23:21Z","timestamp":1777454601000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDT-210249"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,17]]},"references-count":36,"journal-issue":{"issue":"2"},"URL":"https:\/\/doi.org\/10.3233\/idt-210249","relation":{},"ISSN":["1872-4981","1875-8843"],"issn-type":[{"value":"1872-4981","type":"print"},{"value":"1875-8843","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,17]]}}}