{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T07:10:48Z","timestamp":1777705848650,"version":"3.51.4"},"reference-count":46,"publisher":"SAGE Publications","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2022,4,28]]},"abstract":"<jats:p>Facial expression recognition (FER) has been one of the research focuses in recent years due to its significance in human-computer interactions. However, there are still challenges in the field of FER caused by the diversity and variation of facial expressions in real scenes, the singleness of feature type and the lack of enough discriminant features cannot effectively improve the recognition performance. To solve these problems, we propose a Multi-feature Fusion Network (MFNet) with dual-branch based on deep learning. Firstly, the MFNet uses the pyramid parallel multiscale residual network structure with progressive max-pooling of channel attention to extract multi-level facial features and enhance the discrimination of features; In the meantime, a shallow Gabor convolutional network is designed to enhance the adaptation of learned features to the orientation and scale changes and improve the ability to capture local details features; Finally, the maximum expression features obtained by the above two networks are fused to make more effective expression recognition. Experiments on three public large-scale wild FER datasets (RAF-DB, FERPlus, and AffectNet) show that our MFNet has a superior recognition performance than other recognition methods.<\/jats:p>","DOI":"10.3233\/jifs-211021","type":"journal-article","created":{"date-parts":[[2022,1,28]],"date-time":"2022-01-28T12:06:23Z","timestamp":1643371583000},"page":"4999-5011","source":"Crossref","is-referenced-by-count":5,"title":["Multi-feature fusion network for facial expression recognition in the wild"],"prefix":"10.1177","volume":"42","author":[{"given":"Weijun","family":"Gong","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi, China"}]},{"given":"Chaoqing","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Software, Xinjiang University, Urumqi, China"}]},{"given":"Jinlu","family":"Jia","sequence":"additional","affiliation":[{"name":"College of Software, Xinjiang University, Urumqi, China"}]},{"given":"Yurong","family":"Qian","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi, China"},{"name":"College of Software, Xinjiang University, Urumqi, China"},{"name":"Key Laboratory of Signal Detection and Processing in Xinjiang Uygur Autonomous Region, Urumqi, China"}]},{"given":"Yingying","family":"Fan","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi, China"}]}],"member":"179","reference":[{"key":"10.3233\/JIFS-211021_ref1","doi-asserted-by":"crossref","unstructured":"Li S. and Deng W. , Deep Facial Expression Recognition: A Survey, IEEE Transactions on Affective Computing (2020).","DOI":"10.1109\/TAFFC.2020.2981446"},{"issue":"1","key":"10.3233\/JIFS-211021_ref2","doi-asserted-by":"crossref","first-page":"479","DOI":"10.3233\/JIFS-161787","article-title":"Facial expression recognition using weber discrete wavelet transform","volume":"33","author":"Nazir","year":"2017","journal-title":"Journal of Intelligent & Fuzzy Systems"},{"issue":"3","key":"10.3233\/JIFS-211021_ref3","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1007\/s00371-019-01636-3","article-title":"Deep convolutional bilstm fusion network for facial expression recognition","volume":"36","author":"Liang","year":"2020","journal-title":"The Visual Computer"},{"key":"10.3233\/JIFS-211021_ref4","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.patrec.2019.12.013","article-title":"Facial expression recogni tion based on deep convolution long short-term memory networks of double-channel weighted mixture","volume":"131","author":"Zhang","year":"2020","journal-title":"Pattern Recognition Letters"},{"key":"10.3233\/JIFS-211021_ref5","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1109\/AFGR.1998.670949","article-title":"Coding facial expressions with gabor wavelets","author":"Lyons","year":"1998","journal-title":"Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition"},{"key":"10.3233\/JIFS-211021_ref6","first-page":"94","article-title":"The extended cohn-kanade dataset(ck+): A complete dataset for action unit and emotion-specified expression","author":"Lucey","year":"2010","journal-title":"2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops"},{"key":"10.3233\/JIFS-211021_ref7","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.imavis.2011.07.002","article-title":"Facial expression recognition from near-infrared videos","volume":"29","author":"Zhao","year":"2011","journal-title":"Image and Vision Computing"},{"key":"10.3233\/JIFS-211021_ref8","first-page":"2852","article-title":"Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild","author":"Li","year":"2017","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"10.3233\/JIFS-211021_ref9","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1145\/2993148.2993165","article-title":"Training deep networks for facial expression recognition with crowd-sourced label distribution","author":"Barsoum","year":"2016","journal-title":"Proceedings of the 18th ACM International Conference on Multimodal Interaction"},{"issue":"1","key":"10.3233\/JIFS-211021_ref10","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"Affectnet: A database for facial expression, valence, and arousal computing in the wild","volume":"10","author":"Mollahosseini","year":"2017","journal-title":"IEEE Transactions on Affective Computing"},{"key":"10.3233\/JIFS-211021_ref11","first-page":"2927","article-title":"Multimodal 2D and 3D for In-The-Wild Facial Expression Recognition","author":"Ly","year":"2019","journal-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops"},{"key":"10.3233\/JIFS-211021_ref12","doi-asserted-by":"crossref","unstructured":"Li Y. , Lu G. , Li J. , Zhang Z. and Zhang D. , Facial Expression Recognition in the Wild Using Multi-level Features and Attention Mechanisms, IEEE Transactions on Affective Computing (2020).","DOI":"10.1109\/TAFFC.2020.3031602"},{"key":"10.3233\/JIFS-211021_ref13","first-page":"423","article-title":"Hand-Crafted feature guided deep learning for facial expression recognition","author":"Zeng","year":"2018","journal-title":"2018 13th IEEE International Conference on Automatic Face & Gesture Recognition"},{"key":"10.3233\/JIFS-211021_ref14","first-page":"2304","article-title":"A facial expression recognition algorithm based on CNN and LBP feature","author":"Xu","year":"2020","journal-title":"2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference"},{"key":"10.3233\/JIFS-211021_ref15","first-page":"2209","article-title":"Patch-gated CNN for occlusion-aware facial expression recognition","author":"Li","year":"2018","journal-title":"2018 24th International Conference on Pattern Recognition"},{"key":"10.3233\/JIFS-211021_ref16","unstructured":"Amos B. , Ludwiczuk B. and Satyanarayanan M. , Openface: A general-purpose face recognition library with mobile applications, CMU School of Computer Science 6(2) (2016)."},{"issue":"10","key":"10.3233\/JIFS-211021_ref17","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Processing Letters"},{"key":"10.3233\/JIFS-211021_ref18","unstructured":"Minaee S. , Luo P. , Lin Z. and Bowyer K. , Going Deeper Into Face Detection: A Survey, arXiv preprint arXiv:2103.14983, (2021)."},{"key":"10.3233\/JIFS-211021_ref19","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1109\/CVPR.2005.177","article-title":"Histograms of oriented gradients for human detection","author":"Dalal","year":"2005","journal-title":"2005 IEEE computer society conference on computer vision and pattern recognition"},{"issue":"6","key":"10.3233\/JIFS-211021_ref20","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1016\/j.imavis.2008.08.005","article-title":"Facial expression recognition based on Local Binary Patterns: A comprehensive study","volume":"27","author":"Shan","year":"2009","journal-title":"Image and Vision Computing"},{"key":"10.3233\/JIFS-211021_ref21","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1109\/ICCMS.2010.45","article-title":"Automatic Facial Expression Recognition Using Gabor Filter and Expression Analysis","author":"Ou","year":"2010","journal-title":"2010 Second International Conference on Computer Modeling and Simulation"},{"key":"10.3233\/JIFS-211021_ref22","first-page":"117","article-title":"Challenges in representation learning: A report on three machine learning contests","author":"Goodfellow","year":"2013","journal-title":"International conference on neural information processing"},{"key":"10.3233\/JIFS-211021_ref23","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1145\/2818346.2829994","article-title":"Video and image based emotion recognition challenges in the wild Emotiw 2015","author":"Dhall","year":"2015","journal-title":"Proceedings of the 2015 ACM on International Conference on Multimodal Interaction"},{"key":"10.3233\/JIFS-211021_ref24","first-page":"222","article-title":"Facial expression recognition with inconsistently annotated datasets","author":"Zeng","year":"2018","journal-title":"Proceedings of the European conference on computer vision"},{"key":"10.3233\/JIFS-211021_ref25","first-page":"367","article-title":"Covariance pooling for facial expression recognition","author":"Acharya","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops"},{"key":"10.3233\/JIFS-211021_ref26","doi-asserted-by":"crossref","first-page":"5800","DOI":"10.1609\/aaai.v34i04.6037","article-title":"Efficient facial feature learning with wide ensemble-based convolutional neural networks","author":"Siqueira","year":"2020","journal-title":"Proceedings of the AAAI conference on artificial intelligence"},{"key":"10.3233\/JIFS-211021_ref27","first-page":"6897","article-title":"Suppressing uncertainties for large-scale facial expression recognition","author":"Wang","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"10.3233\/JIFS-211021_ref28","first-page":"406","article-title":"Discriminant distribution-agnostic loss for facial expression recognition in the wild","author":"Farzaneh","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops"},{"key":"10.3233\/JIFS-211021_ref29","first-page":"2402","article-title":"Facial expression recognition in the wild via deep attentive center loss","author":"Farzaneh","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision"},{"key":"10.3233\/JIFS-211021_ref30","doi-asserted-by":"crossref","unstructured":"Shi J. and Zhu S. , Learning to Amend Facial Expression Representation via De-albino and Affinity, arXiv preprint arXiv:2103.10189, (2021).","DOI":"10.23919\/CCC55666.2022.9901738"},{"issue":"4","key":"10.3233\/JIFS-211021_ref31","doi-asserted-by":"crossref","first-page":"3510","DOI":"10.1609\/aaai.v35i4.16465","article-title":"Robust lightweight facial expression recognition network with label distribution training","volume":"35","author":"Zhao","year":"2021","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"5","key":"10.3233\/JIFS-211021_ref32","doi-asserted-by":"crossref","first-page":"2439","DOI":"10.1109\/TIP.2018.2886767","article-title":"Occlusion aware facial expression recognition using CNN with attention mechanism","volume":"28","author":"Li","year":"2018","journal-title":"IEEE Transactions on Image Processing"},{"key":"10.3233\/JIFS-211021_ref33","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1145\/3240508.3240578","article-title":"Emotion recognition in speech using cross-modal transfer in the wild","author":"Albanie","year":"2018","journal-title":"Proceedings of the 26th ACM international conference on Multimedia"},{"key":"10.3233\/JIFS-211021_ref34","doi-asserted-by":"crossref","unstructured":"Fan Y. , Li V. and Lam J.C. , Facial expression recognition with deeply-supervised attention network, IEEE Transactions on Affective Computing (2020).","DOI":"10.1109\/TAFFC.2020.2988264"},{"key":"10.3233\/JIFS-211021_ref35","doi-asserted-by":"crossref","first-page":"4057","DOI":"10.1109\/TIP.2019.2956143","article-title":"Region attention networks for pose and occlusion robust facial expression recognition","volume":"29","author":"Wang","year":"2020","journal-title":"IEEE Transactions on Image Processing"},{"key":"10.3233\/JIFS-211021_ref36","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.neucom.2020.12.076","article-title":"LBAN-IL: A novel method of high discriminative representation for facial expression recognition","volume":"432","author":"Li","year":"2021","journal-title":"Neurocomputing"},{"issue":"1","key":"10.3233\/JIFS-211021_ref37","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/TAFFC.2016.2593719","article-title":"Facial expression recognition in video with multiple feature fusion","volume":"9","author":"Chen","year":"2016","journal-title":"IEEE Transactions on Affective Computing"},{"key":"10.3233\/JIFS-211021_ref38","first-page":"1","article-title":"Combining convolutional neural networks for emotion recognition","author":"Huang","year":"2017","journal-title":"2017 IEEE MIT Undergraduate Research Technology Conference"},{"key":"10.3233\/JIFS-211021_ref39","doi-asserted-by":"crossref","first-page":"64827","DOI":"10.1109\/ACCESS.2019.2917266","article-title":"Local learning with deep and handcrafted features for facial expression recognition","volume":"7","author":"Georgescu","year":"2019","journal-title":"IEEE Access"},{"key":"10.3233\/JIFS-211021_ref40","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.neucom.2019.05.005","article-title":"Three convolutional neural network models for facial expression recognition in the wild","volume":"355","author":"Shao","year":"2019","journal-title":"Neurocomputing"},{"key":"10.3233\/JIFS-211021_ref41","doi-asserted-by":"crossref","unstructured":"Ma F. , Sun B. and Li S. , Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion, IEEE Transactions on Affective Computing (2021).","DOI":"10.1109\/TAFFC.2021.3122146"},{"key":"10.3233\/JIFS-211021_ref42","unstructured":"Duta I.C. , Liu L. , Zhu F. and Shao L. , Pyramidal convolution: rethinking convolutional neural networks for visual recognition, arXiv preprint arXiv:2006.11538, (2020)."},{"key":"10.3233\/JIFS-211021_ref43","first-page":"11531","article-title":"ECA-Net: efficient channel attention for deep convolutional neural networks","author":"Wang","year":"2020","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"issue":"9","key":"10.3233\/JIFS-211021_ref44","doi-asserted-by":"crossref","first-page":"4357","DOI":"10.1109\/TIP.2018.2835143","article-title":"Gabor convolutional networks","volume":"27","author":"Luan","year":"2018","journal-title":"IEEE Transactions on Image Processing"},{"issue":"3","key":"10.3233\/JIFS-211021_ref45","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"International Journal of Computer Vision"},{"key":"10.3233\/JIFS-211021_ref46","first-page":"618","article-title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","author":"Selvaraju","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-211021","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:45:23Z","timestamp":1777455923000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-211021"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,28]]},"references-count":46,"journal-issue":{"issue":"6"},"URL":"https:\/\/doi.org\/10.3233\/jifs-211021","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,28]]}}}