{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T17:01:27Z","timestamp":1770397287996,"version":"3.49.0"},"reference-count":68,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T00:00:00Z","timestamp":1770336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>\n                    Emotion estimation is a field that has been studied for a long time, and several approaches using machine learning models exist. This article presents BlendFER-Lite, an LSTM model that uses Blendshapes from the MediaPipe library to analyze facial expressions detected from a live-streamed camera feed. This model is trained on the FER2013 dataset and achieves 71% accuracy and an F1-score of 62%, meeting the accuracy benchmark for the FER2013 dataset while significantly reducing computational costs compared to current methods. For the sake of reproducibility, the code repository, datasets, and models proposed in this paper, in addition to the preprint, can be found on Hugging Face at:\n                    <jats:ext-link>https:\/\/huggingface.co\/papers\/2501.13432<\/jats:ext-link>\n                    .\n                  <\/jats:p>\n                  <jats:sec>\n                    <jats:title>JEL Classification<\/jats:title>\n                    <jats:p>D8, H51<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>MSC Classification<\/jats:title>\n                    <jats:p>35A01, 65L10, 65L12, 65L20, 65L70<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fnbot.2025.1678984","type":"journal-article","created":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T06:36:27Z","timestamp":1770359787000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Emotion estimation from video footage with LSTM"],"prefix":"10.3389","volume":"19","author":[{"given":"Samer","family":"Attrah","sequence":"first","affiliation":[{"name":"Hogeschool van Arnhem en Nijmegen, Automotive and Engineering Academy","place":["Arnhem, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,2,6]]},"reference":[{"key":"B1","article-title":"\u201cTensorflow: a system for large-scale machine learning,\u201d","author":"Abadi","year":"2016","journal-title":"12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)"},{"key":"B2","doi-asserted-by":"publisher","first-page":"39460","DOI":"10.32604\/cmc.2023.039460","article-title":"Deep facial emotion recognition using local features based on facial landmarks for security system","volume":"76","author":"An","year":"2023","journal-title":"Comput. Mater. Continua"},{"key":"B3","unstructured":"Attrah\n              S.\n            \n          \n          Kaggle\n          BlendFER-Lite"},{"key":"B4","unstructured":"Attrah\n              S.\n            \n          \n          FER2013 Blendshapes Dataset Example (Partial)"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477553","article-title":"\u201cOpenface: an open source facial behavior analysis toolkit,\u201d","author":"Baltru\u0161aitis","year":"2016","journal-title":"2016 IEEE Winter Conference on Applications of Computer Vision (WACV)"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1267","DOI":"10.1109\/TAFFC.2021.3124142","article-title":"Impact of facial landmark localization on facial expression recognition","volume":"14","author":"Belmonte","year":"2021","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B7","doi-asserted-by":"publisher","first-page":"104724","DOI":"10.1016\/j.imavis.2023.104724","article-title":"Emotion recognition at a distance: the robustness of machine learning based on hand-crafted facial features vs. deep learning models","volume":"136","author":"Bisogni","year":"2023","journal-title":"Image Vis. Comput"},{"key":"B8","article-title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","author":"Chung","year":"2014","journal-title":"arXiv preprint arXiv:1412.3555"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Computation","year":"2016","journal-title":"Neural Comput"},{"key":"B10","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1023\/A:1022627411411","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn"},{"key":"B11","article-title":"The mcc approaches the geometric mean of precision and recall as true negatives approach infinity","author":"Crall","year":"2023","journal-title":"arXiv preprint arXiv:2305.00594"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3143004","article-title":"\u201cFrom individual to group-level emotion recognition: Emotiw 5.0,\u201d","author":"Dhall","year":"2017","journal-title":"Proceedings of the 19th ACM International Conference on Multimodal Interaction"},{"key":"B13","doi-asserted-by":"publisher","first-page":"E1454","DOI":"10.1073\/pnas.1322355111","article-title":"Compound facial expressions of emotion","volume":"111","author":"Du","year":"2014","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B14","doi-asserted-by":"publisher","first-page":"2279","DOI":"10.1016\/S0031-3203(01)00178-9","article-title":"Image processing with neural networks\u2014a review","volume":"35","author":"Egmont-Petersen","year":"2002","journal-title":"Pattern Recognit"},{"key":"B15","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1037\/\/0003-066X.48.4.384","article-title":"Facial expression and emotion","volume":"48","author":"Ekman","year":"1993","journal-title":"Am. Psychol"},{"key":"B16","author":"Ekman","year":"2003","journal-title":"Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues, volume 10"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780195104462.001.0001","author":"Ekman","year":"1997","journal-title":"What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS)"},{"key":"B18","volume-title":"Mean Squared Error","author":"Error","year":"2008"},{"key":"B19","doi-asserted-by":"publisher","first-page":"26756","DOI":"10.1109\/ACCESS.2022.3156598","article-title":"AD-corre: adaptive correlation-based loss for facial expression recognition in the wild","volume":"10","author":"Fard","year":"2022","journal-title":"IEEE Access"},{"key":"B20","doi-asserted-by":"publisher","first-page":"8704","DOI":"10.3390\/s22228704","article-title":"Development of real-time landmark-based emotion recognition cnn for masked faces","volume":"22","author":"Farkhod","year":"2022","journal-title":"Sensors"},{"key":"B21","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.procs.2022.12.109","article-title":"Facial expression recognition using bidirectional lstm-cnn","volume":"216","author":"Febrian","year":"2023","journal-title":"Procedia Comput. Sci"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1295","DOI":"10.1016\/j.procs.2023.01.108","article-title":"Facial emotion recognition using handcrafted features and cnn","volume":"218","author":"Gautam","year":"2023","journal-title":"Procedia Comput. Sci"},{"key":"B23","author":"Goodfellow","year":"2016","journal-title":"Deep Learning"},{"key":"B24","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/978-3-642-42051-1_16","article-title":"\u201cChallenges in representation learning: a report on three machine learning contests,\u201d","volume-title":"Neural information processing: 20th international conference, ICONIP 2013, Daegu, Korea, november 3\u20137, 2013. Proceedings, Part III 20","author":"Goodfellow","year":"2013"},{"key":"B25","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1016\/j.imavis.2009.08.002","article-title":"Multi-pie","volume":"28","author":"Gross","year":"2010","journal-title":"Image Vis. Comput"},{"key":"B26","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B27","doi-asserted-by":"publisher","first-page":"7132","DOI":"10.1109\/CVPR.2018.00745","article-title":"\u201cSqueeze-and-excitation networks,\u201d","author":"Hu","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B28","doi-asserted-by":"publisher","first-page":"8425","DOI":"10.1038\/s41598-023-35446-4","article-title":"A study on computer vision for facial emotion recognition","volume":"13","author":"Huang","year":"2023","journal-title":"Sci. Rep"},{"key":"B29","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.patrec.2018.04.010","article-title":"Hybrid deep neural networks for face emotion recognition","volume":"115","author":"Jain","year":"2018","journal-title":"Pattern Recognit. Lett"},{"key":"B30","article-title":"Facial emotion recognition: state of the art performance on fer2013","author":"Khaireddin","year":"2021","journal-title":"arXiv preprint arXiv:2105.03588"},{"key":"B31","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"B32","article-title":"Facedig: automated tool for placing landmarks on facial portraits for geometric morphometrics users","author":"Kleisner","year":"2024","journal-title":"arXiv preprint arXiv:2411.01508"},{"key":"B33","doi-asserted-by":"publisher","first-page":"2328","DOI":"10.1109\/CVPRW56347.2022.00259","article-title":"\u201cAbaw: valence-arousal estimation, expression recognition, action unit detection &multi-task learning challenges,\u201d","author":"Kollias","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B34","first-page":"157","article-title":"\u201cAbaw: learning from synthetic data &multi-task learning challenges,\u201d","volume-title":"European Conference on Computer Vision","author":"Kollias","year":"2023"},{"key":"B35","doi-asserted-by":"publisher","first-page":"794","DOI":"10.1109\/FG47880.2020.00126","article-title":"\u201cAnalysing affective behavior in the first abaw 2020 competition,\u201d","author":"Kollias","year":"2020","journal-title":"2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)(FG)"},{"key":"B36","article-title":"Face behavior a la carte: Expressions, affect and action units in a single network","author":"Kollias","year":"","journal-title":"arXiv preprint arXiv:1910.11111"},{"key":"B37","article-title":"Distribution matching for heterogeneous multi-task learning: a large-scale face study","author":"Kollias","year":"2021","journal-title":"arXiv preprint arXiv:2105.03790"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW59228.2023.00626","article-title":"\u201cAbaw: valence-arousal estimation, expression recognition, action unit detection &emotional reaction intensity estimation challenges,\u201d","author":"Kollias","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B39","doi-asserted-by":"publisher","first-page":"907","DOI":"10.1007\/s11263-019-01158-4","article-title":"Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond","volume":"127","author":"Kollias","year":"","journal-title":"Int. J. Comput. Vision"},{"key":"B40","article-title":"Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface","author":"Kollias","year":"2019","journal-title":"arXiv preprint arXiv:1910.04855"},{"key":"B41","article-title":"Affect analysis in-the-wild: Valence-arousal, expressions, action units and a unified framework","author":"Kollias","year":"","journal-title":"arXiv preprint arXiv:2103.15792"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00408","article-title":"\u201cAnalysing affective behavior in the second abaw2 competition,\u201d","author":"Kollias","year":"","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42979-025-03688-w","article-title":"Machine learning-driven emotion recognition through facial landmark analysis","volume":"6","author":"Kumar","year":"2025","journal-title":"SN Comput. Sci"},{"key":"B44","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1109\/CAST.2016.7914982","article-title":"\u201cA real-time robust facial expression recognition system using hog features,\u201d","volume-title":"2016 International Conference on Computing, Analytics and Security Trends (CAST)","author":"Kumar","year":"2016"},{"key":"B45","doi-asserted-by":"publisher","first-page":"1377","DOI":"10.1080\/02699930903485076","article-title":"Presentation and validation of the radboud faces database","volume":"24","author":"Langner","year":"2010","journal-title":"Cogn. Emot"},{"key":"B46","doi-asserted-by":"publisher","first-page":"100545","DOI":"10.1016\/j.cosrev.2023.100545","article-title":"Facial expression and body gesture emotion recognition: a systematic review on the use of visual data in affective computing","volume":"48","author":"Leong","year":"2023","journal-title":"Comput. Sci. Rev"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.2312\/egst.20141042","article-title":"\u201cPractice and theory of blendshape facial models,\u201d","author":"Lewis","year":"2014","journal-title":"Eurographics 2014 - State of the Art Reports"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1195","DOI":"10.1109\/TAFFC.2020.2981446","article-title":"Deep facial expression recognition: a survey","volume":"13","author":"Li","year":"2020","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B49","article-title":"Focal loss for dense object detection","author":"Lin","year":"2017","journal-title":"arXiv preprint arXiv:1708.02002"},{"key":"B50","article-title":"Decoupled weight decay regularization","author":"Loshchilov","year":"2017","journal-title":"arXiv preprint arXiv:1711.05101"},{"key":"B51","article-title":"\u201cMediapipe: a framework for perceiving and processing reality,\u201d","author":"Lugaresi","year":"","journal-title":"Third Workshop on Computer Vision for AR\/VR at IEEE Computer Vision and Pattern Recognition (CVPR)"},{"key":"B52","article-title":"Mediapipe: A framework for building perception pipelines","author":"Lugaresi","year":"","journal-title":"arXiv preprint arXiv:1906.08172"},{"key":"B53","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1007\/978-3-030-16272-6_11","article-title":"Survey on ai-based multimodal methods for emotion detection","volume":"11400","author":"Marechal","year":"2019","journal-title":"High-Perform. Model. Simul. Big Data Applic"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1201\/9781003040620","author":"Medsker","year":"2001","journal-title":"Recurrent Neural Networks: Design and Applications, 1st Edn."},{"key":"B55","article-title":"Emotion recognition using transformers with masked learning","author":"Min","year":"2024","journal-title":"arXiv preprint arXiv:2403.13731"},{"key":"B56","doi-asserted-by":"publisher","first-page":"3046","DOI":"10.3390\/s21093046","article-title":"Deep-emotion: facial expression recognition using attentional convolutional network","volume":"21","author":"Minaee","year":"2021","journal-title":"Sensors"},{"key":"B57","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"Affectnet: a database for facial expression, valence, and arousal computing in the wild","volume":"10","author":"Mollahosseini","year":"2017","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B58","article-title":"Survey of resampling techniques for improving classification performance in unbalanced datasets","author":"More","year":"2016","journal-title":"arXiv preprint arXiv:1608.06048"},{"key":"B59","unstructured":"O'Malley\n              T.\n            \n            \n              Bursztein\n              E.\n            \n            \n              Long\n              J.\n            \n            \n              Chollet\n              F.\n            \n            \n              Jin\n              H.\n            \n            \n              Invernizzi\n              L.\n            \n          \n          Kerastuner\n          \n          2019"},{"key":"B60","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/ICME.2005.1521424","article-title":"\u201cWeb-based database for facial expression analysis,\u201d","volume-title":"2005 IEEE International Conference on Multimedia and Expo","author":"Pantic","year":"2005"},{"key":"B61","article-title":"On the convergence of adam and beyond","author":"Reddi","year":"2019","journal-title":"arXiv preprint arXiv:1904.09237"},{"key":"B62","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/MECO52532.2021.9460191","article-title":"\u201cComparison of facial landmark detection methods for micro-expressions analysis,\u201d","volume-title":"2021 10th Mediterranean Conference on Embedded Computing (MECO)","author":"Savin","year":"2021"},{"key":"B63","doi-asserted-by":"publisher","first-page":"42569","DOI":"10.1007\/s11042-023-14682-w","article-title":"Emotion estimation from nose feature using pyramid structure","volume":"82","author":"Shaila","year":"2023","journal-title":"Multimed. Tools Appl"},{"key":"B64","first-page":"65","article-title":"\u201cInduced disgust, happiness and surprise: an addition to the mmi facial expression database,\u201d","volume-title":"Proceedings of the 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect","author":"Valstar","year":"2010"},{"key":"B65","doi-asserted-by":"publisher","DOI":"10.1109\/ATEE52255.2021.9425073","article-title":"\u201cConvolutional neural network hyperparameters optimization for facial emotion recognition,\u201d","author":"Vulpe-Grigorasi","year":"2021","journal-title":"2021 12th International Symposium on Advanced Topics in Electrical Engineering (ATEE)"},{"key":"B66","unstructured":"Yuan\n              X.\n            \n          \n          Fer2013-facial-emotion-recognition-pytorch\n          \n          2021"},{"key":"B67","doi-asserted-by":"crossref","first-page":"1980","DOI":"10.1109\/CVPRW.2017.248","article-title":"\u201cAff-wild: valence and arousal \u2018in-the-wild' challenge,\u201d","volume-title":"Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on","author":"Zafeiriou","year":"2017"},{"key":"B68","doi-asserted-by":"publisher","first-page":"5756","DOI":"10.1109\/CVPRW59228.2023.00610","article-title":"\u201cLeveraging TCN and transformer for effective visual-audio fusion in continuous emotion recognition,\u201d","author":"Zhou","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1678984\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T06:36:32Z","timestamp":1770359792000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1678984\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,6]]},"references-count":68,"alternative-id":["10.3389\/fnbot.2025.1678984"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2025.1678984","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,6]]},"article-number":"1678984"}}