{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T07:23:39Z","timestamp":1782890619102,"version":"3.54.5"},"reference-count":50,"publisher":"World Scientific Pub Co Pte Ltd","issue":"08","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Patt. Recogn. Artif. Intell."],"published-print":{"date-parts":[[2023,6,30]]},"abstract":"<jats:p> Facial Expression Recognition (FER) is a prominent research area in Computer Vision and Artificial Intelligence that has been playing a crucial role in human\u2013computer interaction. The existing FER system focuses on spatial features for identifying the emotion, which suffers when recognizing emotions from a dynamic sequence of facial expressions in real time. Deep learning techniques based on the fusion of convolutional neural networks (CNN) and long short-term memory (LSTM) are presented in this paper for recognizing emotion and identifying the relationship between the sequence of facial expressions. In this approach, a hyperparameter tweaked VGG-19 skeleton is employed to extract the spatial features automatically from a sequence of images, which avoids the shortcoming of the conventional feature extraction methods. Second, these features are given into bidirectional LSTM (Bi-LSTM) for extracting spatiotemporal features of time series in two directions, which recognize emotion from a sequence of expressions. The proposed method\u2019s performance is evaluated using the CK+ benchmark as well as an in-house dataset captured from the designed IoT kit. Finally, this approach has been verified through hold-out cross-validation techniques. The proposed techniques show an accuracy of 0.92% on CK+, and 0.84% on the in-house dataset. The experimental results reveal that the proposed method outperforms compared to baseline methods and state-of-the-art approaches. Furthermore, precision, recall, F1-score, and ROC curve metrics have been used to evaluate the performance of the proposed system. <\/jats:p>","DOI":"10.1142\/s0218001423560086","type":"journal-article","created":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T06:13:23Z","timestamp":1685081603000},"source":"Crossref","is-referenced-by-count":21,"title":["Emotion Recognition from Facial Expression Using Hybrid CNN\u2013LSTM Network"],"prefix":"10.1142","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0382-3013","authenticated-orcid":false,"given":"M.","family":"Mohana","sequence":"first","affiliation":[{"name":"Department of Computer Science, Centre for Machine Learning and Intelligence (CMLI), Avinashilingam Institute (Deemed University), Coimbatore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"P.","family":"Subashini","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Centre for Machine Learning and Intelligence (CMLI), Avinashilingam Institute (Deemed University), Coimbatore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Krishnaveni","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Centre for Machine Learning and Intelligence (CMLI), Avinashilingam Institute (Deemed University), Coimbatore, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"219","published-online":{"date-parts":[[2023,7,7]]},"reference":[{"key":"S0218001423560086BIB001","first-page":"234","volume-title":"2017 2nd Int. Conf. Image, Vision, and Computing (ICIVC)","author":"Asim M.","year":"2017"},{"key":"S0218001423560086BIB002","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/978-981-16-4115-2_8","volume-title":"Human-Centered Technology for a Better Tomorrow","author":"Azizan M. A.","year":"2022"},{"key":"S0218001423560086BIB003","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1145\/3395035.3425248","volume-title":"Companion Publication of the 2020 Int. Conf. Multimodal Interaction","author":"Bai M.","year":"2020"},{"key":"S0218001423560086BIB004","first-page":"433","volume-title":"Proc. 18th ACM Int. Conf. Multimodal Interaction","author":"Bargal S. A.","year":"2016"},{"key":"S0218001423560086BIB005","first-page":"585","volume-title":"Computational Intelligence: Theories, Applications and Future Directions - Volume II","author":"Bilkhu M. S.","year":"2019"},{"key":"S0218001423560086BIB006","first-page":"288","volume-title":"Proc. 17th Int. Conf. Pattern Recognition, 2004. ICPR 2004","volume":"1","author":"Buciu I.","year":"2004"},{"key":"S0218001423560086BIB007","first-page":"67","volume-title":"2018 13th IEEE Int. Conf. Automatic Face & Gesture Recognition (FG 2018)","author":"Cao Q.","year":"2018"},{"key":"S0218001423560086BIB008","first-page":"1","author":"Chowdary M. K.","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"S0218001423560086BIB010","first-page":"2625","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"Donahue J.","year":"2015"},{"issue":"2","key":"S0218001423560086BIB011","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1037\/h0030377","volume":"17","author":"Ekman P.","year":"1971","journal-title":"J. Pers. Soc. Psychol."},{"key":"S0218001423560086BIB012","author":"Ekman P.","year":"1978","journal-title":"Environ. Psychol. Nonverbal Behav."},{"key":"S0218001423560086BIB013","first-page":"445","volume-title":"Proc. 18th ACM Int. Conf. Multimodal Interaction","author":"Fan Y.","year":"2016"},{"issue":"4","key":"S0218001423560086BIB014","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1134\/S1054661807040190","volume":"17","author":"Feng X.","year":"2007","journal-title":"Pattern Recognit. Image Anal."},{"key":"S0218001423560086BIB015","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1007\/978-3-030-64559-5_23","volume-title":"Int. Symp. Visual Computing","author":"Haddad J.","year":"2020"},{"key":"S0218001423560086BIB016","first-page":"770","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"He K.","year":"2016"},{"issue":"8","key":"S0218001423560086BIB017","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"Hochreiter S.","year":"1997","journal-title":"Neural Comput."},{"key":"S0218001423560086BIB018","first-page":"448","volume-title":"Int. Conf. Machine Learning","author":"Ioffe S.","year":"2015"},{"key":"S0218001423560086BIB019","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.patrec.2018.04.010","volume":"115","author":"Jain N.","year":"2018","journal-title":"Pattern Recognit. Lett."},{"key":"S0218001423560086BIB020","first-page":"1","volume-title":"2016 IEEE Winter Conf. Applications of Computer Vision (WACV)","author":"Jaiswal S.","year":"2016"},{"key":"S0218001423560086BIB022","doi-asserted-by":"crossref","first-page":"93998","DOI":"10.1109\/ACCESS.2019.2928364","volume":"7","author":"Li T. H. S.","year":"2019","journal-title":"IEEE Access"},{"issue":"3","key":"S0218001423560086BIB023","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1007\/s00371-019-01636-3","volume":"36","author":"Liang D.","year":"2020","journal-title":"Vis. Comput."},{"key":"S0218001423560086BIB024","first-page":"337","volume-title":"Proc. Mobile Multimedia\/Image Processing, Security, and Applications","volume":"7708","author":"Lu L.","year":"2010"},{"key":"S0218001423560086BIB025","first-page":"94","volume-title":"2010 IEEE Computer Society Conf. Computer Vision and Pattern Recognition \u2014 Workshops","author":"Lucey P.","year":"2010"},{"key":"S0218001423560086BIB026","first-page":"63","volume-title":"Advances in Face Detection and Facial Image Analysis","author":"Martinez B.","year":"2016"},{"key":"S0218001423560086BIB027","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1016\/j.procs.2016.07.233","volume":"93","author":"Mayya V.","year":"2016","journal-title":"Procedia Comput. Sci."},{"issue":"3","key":"S0218001423560086BIB028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s42452-020-2234-1","volume":"2","author":"Mehendale N.","year":"2020","journal-title":"SN Appl. Sci."},{"key":"S0218001423560086BIB029","doi-asserted-by":"crossref","first-page":"193","DOI":"10.4324\/9781315080918-15","volume-title":"Communication Theory","author":"Mehrabian A.","year":"2017"},{"key":"S0218001423560086BIB030","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1145\/958432.958479","volume-title":"Proc. 5th Int. Conf. Multimodal Interfaces","author":"Michel P.","year":"2003"},{"key":"S0218001423560086BIB031","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1109\/IIPHDW.2018.8388338","volume-title":"2018 Int. Interdisciplinary PhD Workshop (IIPhDW)","author":"Miko\u0142ajczyk A.","year":"2018"},{"key":"S0218001423560086BIB033","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/978-981-13-2622-6_8","volume-title":"Computational Science and Technology","author":"Peter M.","year":"2019"},{"key":"S0218001423560086BIB034","doi-asserted-by":"crossref","first-page":"26633","DOI":"10.1007\/s11042-020-10133-y","volume":"81","author":"Pise A.","year":"2022","journal-title":"Multimedia Tools Appl."},{"key":"S0218001423560086BIB035","first-page":"317","volume-title":"2020 6th Int. Conf. Advanced Computing and Communication Systems (ICACCS)","author":"Pranav E.","year":"2020"},{"key":"S0218001423560086BIB036","first-page":"1","volume-title":"Proc. HRI","author":"Ratliff M. S.","year":"2008"},{"issue":"6","key":"S0218001423560086BIB037","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1002\/tee.22151","volume":"10","author":"Ren F.","year":"2015","journal-title":"IEEJ Trans. Electr. Electron. Eng."},{"issue":"7","key":"S0218001423560086BIB038","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1007\/s00521-018-3358-8","volume":"29","author":"Ruiz-Garcia A.","year":"2018","journal-title":"Neural Comput. Appl."},{"key":"S0218001423560086BIB039","doi-asserted-by":"crossref","first-page":"3367","DOI":"10.1109\/ICASSP40776.2020.9053919","volume-title":"2020 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP)","author":"Sepas-Moghaddam A.","year":"2020"},{"key":"S0218001423560086BIB040","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1145\/3005467.3005471","volume-title":"Proc. 2nd Workshop Advancements in Social Signal Processing for Multimodal Interaction","author":"Sert M.","year":"2016"},{"key":"S0218001423560086BIB041","first-page":"123","volume-title":"2017 IEEE 15th Int. Conf. Software Engineering Research, Management, and Applications (SERA)","author":"Shan K.","year":"2017"},{"issue":"1","key":"S0218001423560086BIB042","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","volume":"6","author":"Shorten C.","year":"2019","journal-title":"J. Big Data"},{"issue":"1","key":"S0218001423560086BIB044","first-page":"1929","volume":"15","author":"Srivastava N.","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"S0218001423560086BIB046","first-page":"2818","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"Szegedy C.","year":"2016"},{"issue":"1","key":"S0218001423560086BIB047","doi-asserted-by":"crossref","first-page":"118","DOI":"10.3969\/j.issn.1004-4132.2010.01.019","volume":"21","author":"Tan H.","year":"2010","journal-title":"J. Syst. Eng. Electron."},{"issue":"2","key":"S0218001423560086BIB048","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1023\/B:VISI.0000013087.49260.fb","volume":"57","author":"Viola P.","year":"2004","journal-title":"Int. J. Comput. Vis."},{"issue":"1","key":"S0218001423560086BIB049","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.cviu.2006.10.011","volume":"108","author":"Wang J.","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"S0218001423560086BIB050","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.jpdc.2020.03.012","volume":"141","author":"Xiao G.","year":"2020","journal-title":"J. Parallel Distrib. Comput."},{"key":"S0218001423560086BIB051","doi-asserted-by":"crossref","first-page":"2117","DOI":"10.1109\/ICMA.2015.7237813","volume-title":"2015 IEEE Int. Conf. Mechatronics and Automation (ICMA)","author":"Xu X.","year":"2015"},{"issue":"2","key":"S0218001423560086BIB052","doi-asserted-by":"crossref","first-page":"022022","DOI":"10.1088\/1742-6596\/1168\/2\/022022","volume":"1168","author":"Ying X.","year":"2019","journal-title":"J. Phys., Conf. Ser."},{"key":"S0218001423560086BIB053","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1109\/KBEI.2019.8734943","volume-title":"2019 5th Conf. Knowledge Based Engineering and Innovation (KBEI)","author":"Zadeh M. M. T.","year":"2019"},{"key":"S0218001423560086BIB054","first-page":"1","volume-title":"2020 Fifth Int. Conf. Informatics and Computing (ICIC)","author":"Zahara L.","year":"2020"},{"issue":"12","key":"S0218001423560086BIB055","doi-asserted-by":"crossref","first-page":"2324","DOI":"10.1109\/TIP.2008.2006658","volume":"17","author":"Zhang M.","year":"2008","journal-title":"IEEE Trans. Image Process."}],"container-title":["International Journal of Pattern Recognition and Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218001423560086","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T06:40:22Z","timestamp":1690353622000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0218001423560086"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,30]]},"references-count":50,"journal-issue":{"issue":"08","published-print":{"date-parts":[[2023,6,30]]}},"alternative-id":["10.1142\/S0218001423560086"],"URL":"https:\/\/doi.org\/10.1142\/s0218001423560086","relation":{},"ISSN":["0218-0014","1793-6381"],"issn-type":[{"value":"0218-0014","type":"print"},{"value":"1793-6381","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,30]]},"article-number":"2356008"}}