{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T17:12:53Z","timestamp":1786122773742,"version":"build-2736575974"},"reference-count":35,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T00:00:00Z","timestamp":1786060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Facial expression recognition technology is vital for security, verification, and personalization, but it faces challenges due to variations in scale, illumination, occlusion, and facial expressions. This paper presents a hybrid architecture that combines Vision Transformers (ViTs) to capture global context with EfficientNet-B3 for multi-scale feature extraction. Unlike simple concatenation, our approach projects the ViT\u2019s [CLS] token and the EfficientNet\u2019s global pooling features into a shared 512-dimensional space before merging, enabling better alignment of global and local features. When tested on the FERPlus dataset, it reaches an accuracy of 94.4 \u00b1 0.3%, surpassing several recent methods, notably existing transformer- and CNN-based methods. Ablation studies show each component\u2019s contribution, with the full model outperforming the no-fusion version by 2.6%. With around 98 million parameters and an inference time of ~23 ms per image, it balances efficiency and high performance, suitable for real-time use on suitable hardware. Evaluation via confusion matrix, t-SNE visualization, and comparisons with recent techniques such as HLA-ViT (90.13%), AU-ViT (90.15%), and CCFER (91.24%) demonstrates its robustness and discriminative feature learning. This work highlights the promise of hybrid deep learning architectures in tackling real-world facial expression recognition challenges.<\/jats:p>","DOI":"10.3390\/jimaging12080360","type":"journal-article","created":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T11:44:53Z","timestamp":1786103093000},"page":"360","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Hybrid Vision Transformer and EfficientNet-B3 Framework for Facial Expression Recognition"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9445-8044","authenticated-orcid":false,"given":"Sasan","family":"Karamizadeh","sequence":"first","affiliation":[{"name":"Department of Computer, Ershad Damavand Institute of Higher Education, Tehran 1416834311, Iran"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2958-6901","authenticated-orcid":false,"given":"Saman","family":"Shojae Chaeikar","sequence":"additional","affiliation":[{"name":"Department of Cybersecurity, Sydney International School of Technology and Commerce, Sydney, NSW 2000, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mazdak","family":"Zamani","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, College of Arts and Sciences, State University of New York\u2013Empire State University, New York, NY 10016, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1729881420972278","DOI":"10.1177\/1729881420972278","article-title":"A review of algorithms and techniques for image-based recognition and inference in mobile robotic systems","volume":"17","author":"Tawiah","year":"2020","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_2","unstructured":"Sun, Z., and Tzimiropoulos, G. (2022). Part-based face recognition with vision transformers. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"25152","DOI":"10.1109\/ACCESS.2023.3255110","article-title":"On the role of thermal imaging in automotive applications: A critical review","volume":"11","author":"Farooq","year":"2023","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3505244","article-title":"Transformers in vision: A survey","volume":"54","author":"Khan","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hariharan, G.K., Rohith, R., Maheswari, B., Harini, N., Raghul, J., and Saraswathi, U. (2025). Deep Learning Approach for Automated Tridosha Classification Based on Ayurvedic Facial Features. Proceedings of the 2025 Fourth International Conference on Smart Technologies and Systems for Next Generation Computing (ICSTSN), IEEE.","DOI":"10.1109\/ICSTSN67075.2025.11398045"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shih, P., and Liu, C. (2005). Comparative assessment of content-based face image retrieval in different color spaces. Audio-and Video-Based Biometric Person Authentication, Springer.","DOI":"10.1007\/11527923_108"},{"key":"ref_7","first-page":"1","article-title":"A Video Face Recognition Leveraging Temporal Information Based on Vision Transformer","volume":"Volume 27","author":"Zhang","year":"2023","journal-title":"Proceedings of the 2023 International Conference on Artificial Intelligence and Image Processing"},{"key":"ref_8","unstructured":"Xiangwei, F. (2022, January 22\u201324). Facial Expression Recognition Based on Semi-supervised Vision Transformer. Proceedings of the 2022 International Symposium on Artificial Intelligence and Energy Engineering, Belfort, France."},{"key":"ref_9","unstructured":"Tan, M., and Le, Q. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, PMLR."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.ins.2021.08.043","article-title":"Facial expression recognition with grid-wise attention and visual transformer","volume":"580","author":"Huang","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"104688","DOI":"10.1016\/j.imavis.2023.104688","article-title":"Synthetic data for face recognition: Current state and future prospects","volume":"135","author":"Boutros","year":"2023","journal-title":"Image Vis. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"He, L., He, L., and Peng, L. (2023). CFormerFaceNet: Efficient lightweight network merging a CNN and transformer for face recognition. Appl. Sci., 13.","DOI":"10.3390\/app13116506"},{"key":"ref_13","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J\u00e9gou, H. (2021). Training data-efficient image transformers & distillation through attention. International Conference on Machine Learning, PMLR."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kim, S., Nam, J., and Ko, B.C. (2022). Facial expression recognition based on squeeze vision transformer. Sensors, 22.","DOI":"10.3390\/s22103729"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, X. (2025, January 22\u201323). Convolutional Based Vision Transformer Networks for an Efficient Face Recognition. Proceedings of the 2025 2nd International Conference on Intelligent Algorithms for Computational Intelligence Systems, Hassan, India.","DOI":"10.1109\/IACIS65746.2025.11211302"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Galv\u00e1n, A., Higuero, M., Sanz, A., Atutxa, A., Jacob, E., and Saavedra, M. (2025). Comparing CNN and ViT for open-set face recognition. Electronics, 14.","DOI":"10.3390\/electronics14193840"},{"key":"ref_17","unstructured":"Khawar, I., Muhammad, Z.Z., and Arif, M. (2022). Face Pyramid Vision Transformer. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, G., Park, G., Kang, S., and Woo, S.S. (2023, January 27\u201331). S-ViT: Sparse vision transformer for accurate face recognition. Proceedings of the 38th ACM\/SIGAPP Symposium on Applied Computing, Tallinn, Estonia.","DOI":"10.1145\/3555776.3577640"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1109\/TCSVT.2022.3212434","article-title":"Hybrid CNN-transformer features for visual place recognition","volume":"33","author":"Wang","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"214","DOI":"10.17694\/bajece.1486140","article-title":"Face Expression Recognition via transformer-based classification models","volume":"12","author":"Acar","year":"2024","journal-title":"Balk. J. Electr. Comput. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, R., and Sun, X. (2023, January 1\u20134). Dynamic Facial Expression Recognition Based on Vision Transformer with Deformable Module. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Oahu, HI, USA.","DOI":"10.1109\/SMC53992.2023.10394270"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2299","DOI":"10.1007\/s40747-021-00399-6","article-title":"Transferable face image privacy protection based on federated learning and ensemble models","volume":"7","author":"Yang","year":"2021","journal-title":"Complex Intell. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6103","DOI":"10.1007\/s40747-024-01456-6","article-title":"Residual serialized cross grouping transformer for small scale sketch face recognition","volume":"10","author":"Du","year":"2024","journal-title":"Complex Intell. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3216","DOI":"10.1109\/TVCG.2018.2866090","article-title":"Deep neural representation guided face sketch synthesis","volume":"25","author":"Sheng","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.vrih.2022.07.006","article-title":"Transformer architecture based on mutual attention for image anomaly detection","volume":"5","author":"Zhang","year":"2023","journal-title":"Virtual Real. Intell. Hardw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e2203","DOI":"10.1002\/cav.2203","article-title":"3D facial attractiveness prediction based on deep feature fusion","volume":"35","author":"Liu","year":"2024","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5545","DOI":"10.1007\/s00371-023-03121-4","article-title":"Two-stage dual-resolution face network for cross-resolution face recognition in surveillance systems","volume":"40","author":"Chen","year":"2024","journal-title":"Vis. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"7701","DOI":"10.1007\/s00371-023-03202-4","article-title":"Disentangled representation transformer network for 3D face reconstruction and robust dense alignment","volume":"40","author":"Li","year":"2024","journal-title":"Vis. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ouannes, L., Khalifa, A.B., and Amara, N.E.B. (2024, January 1\u20134). Enhancing face recognition in degraded conditions via vision transformer. Proceedings of the 2024 10th International Conference on Control, Decision and Information Technologies, Valletta, Malta.","DOI":"10.1109\/CoDIT62066.2024.10708244"},{"key":"ref_30","unstructured":"(2026, April 10). FERPlus Dataset. Available online: https:\/\/gts.ai\/dataset-download\/ferplus-dataset."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"6253","DOI":"10.1109\/TCSVT.2022.3165321","article-title":"Adaptive multilayer perceptual attention network for facial expression recognition","volume":"32","author":"Liu","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Tian, Y., Zhu, J., Yao, H., and Chen, D. (2024). Facial expression recognition based on vision transformer with hybrid local attention. Appl. Sci., 14.","DOI":"10.3390\/app14156471"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"169263","DOI":"10.1109\/ACCESS.2024.3496506","article-title":"Face expression recognition with vision transformer and local mutual information maximization","volume":"12","author":"Dou","year":"2024","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4943761","DOI":"10.1049\/sfw2\/4943761","article-title":"A Novel Facial Expression Recognition Approach Combining Canny Edge Detection and Convolutional Neural Networks","volume":"2025","author":"Ding","year":"2025","journal-title":"IET Softw."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3331","DOI":"10.1109\/TMM.2025.3535327","article-title":"Facial action units as a joint dataset training bridge for facial expression recognition","volume":"27","author":"Mao","year":"2025","journal-title":"IEEE Trans. Multimed."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/8\/360\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T11:57:47Z","timestamp":1786103867000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/8\/360"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,7]]},"references-count":35,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["jimaging12080360"],"URL":"https:\/\/doi.org\/10.3390\/jimaging12080360","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,7]]}}}