{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T14:26:07Z","timestamp":1766067967873,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T00:00:00Z","timestamp":1712880000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Hubei Province Key R&amp;D Program of China","award":["2022BBA0016","22Z299"],"award-info":[{"award-number":["2022BBA0016","22Z299"]}]},{"name":"2022 Special Tasks of Philosophy and Social Science Research of Hubei Provincial Department of Education","award":["2022BBA0016","22Z299"],"award-info":[{"award-number":["2022BBA0016","22Z299"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Occlusion or posture change of the face in natural scenes has typical asymmetry; however, an asymmetric face plays a key part in the lack of information available for facial expression recognition. To solve the problem of low accuracy of asymmetric facial expression recognition, this paper proposes a fusion of channel global features and a spatial local information expression recognition network called the \u201cChannel\u2013Spatial Integration Network\u201d (CSINet). First, to extract the underlying detail information and deepen the network, the attention residual module with a redundant information filtering function is designed, and the backbone feature-extraction network is constituted by module stacking. Second, considering the loss of information in the local key area of face occlusion, the channel\u2013spatial fusion structure is constructed, and the channel features and spatial features are combined to enhance the accuracy of occluded facial recognition. Finally, before the full connection layer, more local spatial information is embedded into the global channel information to capture the relationship between different channel\u2013spatial targets, which improves the accuracy of feature expression. Experimental results on the natural scene facial expression data sets RAF-DB and FERPlus show that the recognition accuracies of the modeling approach proposed in this paper are 89.67% and 90.83%, which are 13.24% and 11.52% higher than that of the baseline network ResNet50, respectively. Compared with the latest facial expression recognition methods such as CVT, PACVT, etc., the method in this paper obtains better evaluation results of masked facial expression recognition, which provides certain theoretical and technical references for daily facial emotion analysis and human\u2013computer interaction applications.<\/jats:p>","DOI":"10.3390\/sym16040471","type":"journal-article","created":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T11:09:20Z","timestamp":1712920160000},"page":"471","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["CSINet: Channel\u2013Spatial Fusion Networks for Asymmetric Facial Expression Recognition"],"prefix":"10.3390","volume":"16","author":[{"given":"Yan","family":"Cheng","sequence":"first","affiliation":[{"name":"College of Food Science and Technology, Huazhong Agricultural University, Wuhan 430070, China"},{"name":"School of Logistics, Wuhan Technical College of Communications, Wuhan 430065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6379-0882","authenticated-orcid":false,"given":"Defeng","family":"Kong","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Huang, X., and Romano, D.M. (2024). Coral Morph: An Artistic Shape-Changing Textile Installation for Mindful Emotion Regulation in the Wild. Int. J. Hum. Comput. Interact., 1\u201317.","DOI":"10.1080\/10447318.2024.2313283"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jeong, M., and Ko, B.C. (2018). Driver\u2019s facial expression recognition in real-time for safe driving. Sensors, 18.","DOI":"10.3390\/s18124270"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3627","DOI":"10.1007\/s40747-021-00419-5","article-title":"License plate identification and recognition in a non-standard environment using neural pattern matching","volume":"8","author":"Shafi","year":"2022","journal-title":"Complex Intell. Syst."},{"key":"ref_4","first-page":"619","article-title":"A survey on human face expression recognition techniques","volume":"33","author":"Revina","year":"2021","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.neucom.2022.04.052","article-title":"Facial expressions recognition with multi-region divided attention networks for smart education cloud applications","volume":"493","author":"Guo","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kortli, Y., Jridi, M., Al Falou, A., and Atri, M. (2020). Face recognition systems: A survey. Sensors, 20.","DOI":"10.3390\/s20020342"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.patrec.2022.11.004","article-title":"Face-mask-aware facial expression recognition based on face parsing and vision transformer","volume":"164","author":"Yang","year":"2022","journal-title":"Pattern Recognit. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"104824","DOI":"10.1016\/j.imavis.2023.104824","article-title":"GFFT: Global-local feature fusion transformers for facial expression recognition in the wild","volume":"139","author":"Xu","year":"2023","journal-title":"Image Vis. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2439","DOI":"10.1109\/TIP.2018.2886767","article-title":"Occlusion aware facial expression recognition using CNN with attention mechanism","volume":"28","author":"Li","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3158369","article-title":"Facial expression analysis under partial occlusion: A survey","volume":"51","author":"Zhang","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.patrec.2019.12.013","article-title":"Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture","volume":"131","author":"Zhang","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pan, B., Wang, S., and Xia, B. (2019, January 21\u201325). Occluded facial expression recognition enhanced through privileged information. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3351049"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Du, H., Chen, Y., and Shu, Z. (2024, January 26\u201328). Facial Expression Recognition Algorithm Based on Local Feature Extraction. Proceedings of the 2024 IEEE 4th International Conference on Power, Electronics and Computer Applications (ICPECA), Shenyang, China.","DOI":"10.1109\/ICPECA60615.2024.10470984"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1177\/001391657400600205","article-title":"A verbal measure of information rate for studies in environmental psychology","volume":"6","author":"Mehrabian","year":"1974","journal-title":"Environ. Behav."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4057","DOI":"10.1109\/TIP.2019.2956143","article-title":"Region attention networks for pose and occlusion robust facial expression recognition","volume":"29","author":"Wang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s11633-023-1417-9","article-title":"Adaptively Enhancing Facial Expression Crucial Regions via a Local Non-local Joint Network","volume":"21","author":"Shi","year":"2024","journal-title":"Mach. Intell. Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.neunet.2023.11.033","article-title":"Hierarchical attention network with progressive feature fusion for facial expression recognition","volume":"170","author":"Tao","year":"2024","journal-title":"Neural Netw."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rizwan, S.A., Jalal, A., and Kim, K. (2020, January 17\u201319). An accurate facial expression detector using multi-landmarks selection and local transform features. Proceedings of the 2020 3rd International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan.","DOI":"10.1109\/ICACS47775.2020.9055954"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1109\/TAI.2022.3172272","article-title":"Landmark-Aware and Part-Based Ensemble Transfer Learning Network for Static Facial Expression Recognition from Images","volume":"4","author":"Wadhawan","year":"2022","journal-title":"IEEE Trans. Artif. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.patrec.2020.01.016","article-title":"Facial expression recognition based on a multi-task global-local network","volume":"131","author":"Yu","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhao, Z., and Liu, Q. (2021, January 20\u201324). Former-dfer: Dynamic facial expression recognition transformer. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.","DOI":"10.1145\/3474085.3475292"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s13042-022-01681-w","article-title":"A descriptive human visual cognitive strategy using graph neural network for facial expression recognition","volume":"15","author":"Liu","year":"2024","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"110157","DOI":"10.1016\/j.patcog.2023.110157","article-title":"Dual subspace manifold learning based on GCN for intensity-invariant facial expression recognition","volume":"148","author":"Chen","year":"2024","journal-title":"Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"106111","DOI":"10.1016\/j.neunet.2024.106111","article-title":"A novel transformer autoencoder for multi-modal emotion recognition with incomplete data","volume":"172","author":"Cheng","year":"2024","journal-title":"Neural Netw."},{"key":"ref_25","first-page":"17616","article-title":"Relative uncertainty learning for facial expression recognition","volume":"34","author":"Zhang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ruan, D., Yan, Y., Lai, S., Chai, Z., Shen, C., and Wang, H. (2021, January 19\u201325). Feature decomposition and reconstruction learning for effective facial expression recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00757"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.neucom.2020.06.014","article-title":"Attention mechanism-based CNN for facial expression recognition","volume":"411","author":"Li","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_28","first-page":"915","article-title":"Printed Circuit Board (PCB) Surface Micro Defect Detection Model Based on Residual Network with Novel Attention Mechanism","volume":"78","author":"Hu","year":"2024","journal-title":"Comput. Mater. Contin."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 11\u201314). Identity mappings in deep residual networks. Proceedings of the Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3911","DOI":"10.1109\/TCSVT.2019.2915238","article-title":"Channel-wise and spatial feature modulation network for single image super-resolution","volume":"30","author":"Hu","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"122562","DOI":"10.1016\/j.eswa.2023.122562","article-title":"CSENMT: A deep image compressed sensing encryption network via multi-color space and texture feature","volume":"241","author":"Chai","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4683","DOI":"10.1109\/TIP.2020.2973812","article-title":"The devil is in the channels: Mutual-channel loss for fine-grained image classification","volume":"29","author":"Chang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","first-page":"7562","article-title":"End-to-End RGB-D Image Compression via Exploiting Channel-Modality Redundancy","volume":"38","author":"Zheng","year":"2024","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"102774","DOI":"10.1016\/j.jvcir.2020.102774","article-title":"Detail retaining convolutional neural network for image denoising","volume":"71","author":"Li","year":"2020","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, J., and Zhu, S. (2024, January 14\u201319). Channel-Spatial Transformer for Efficient Image Super-Resolution. Proceedings of the ICASSP 2024\u20142024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.","DOI":"10.1109\/ICASSP48485.2024.10446047"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hou, Q., Zhou, D., and Feng, J. (2021, January 19\u201325). Coordinate attention for efficient mobile network design. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01350"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"122804","DOI":"10.1016\/j.eswa.2023.122804","article-title":"SAM-Net: Self-Attention based Feature Matching with Spatial Transformers and Knowledge Distillation","volume":"242","author":"Kelenyi","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8425","DOI":"10.1038\/s41598-023-35446-4","article-title":"A study on computer vision for facial emotion recognition","volume":"13","author":"Huang","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"38528","DOI":"10.1109\/ACCESS.2020.2964752","article-title":"Expression recognition method based on a lightweight convolutional neural network","volume":"8","author":"Zhao","year":"2020","journal-title":"IEEE Access"},{"key":"ref_43","first-page":"326","article-title":"Comparison Analysis of CXR Images in Detecting Pneumonia Using VGG16 and ResNet50 Convolution Neural Network Model","volume":"8","author":"Izdihar","year":"2024","journal-title":"JOIV Int. J. Inform. Vis."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1109\/TAFFC.2021.3122146","article-title":"Facial expression recognition with visual transformers and attentional selective fusion","volume":"14","author":"Ma","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"6544","DOI":"10.1109\/TIP.2021.3093397","article-title":"Learning deep global multi-scale and local attention features for facial expression recognition in the wild","volume":"30","author":"Zhao","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"6253","DOI":"10.1109\/TCSVT.2022.3165321","article-title":"Adaptive multilayer perceptual attention network for facial expression recognition","volume":"32","author":"Liu","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Gadekallu, T.R., Khare, N., Bhattacharya, S., Singh, S., Maddikunta, P.K.R., Ra, I.H., and Alazab, M. (2020). Early Detection of Diabetic Retinopathy Using PCA-Firefly Based Deep Learning Model. Electronics, 9.","DOI":"10.3390\/electronics9020274"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.ins.2021.08.043","article-title":"Facial expression recognition with grid-wise attention and visual transformer","volume":"580","author":"Huang","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_49","first-page":"1","article-title":"Fusion of Global Enhancement and Local Attention Features for Expression Recognition Network","volume":"11","author":"Juan","year":"2023","journal-title":"J. Front. Comput. Sci. Technol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1825","DOI":"10.1049\/ipr2.12155","article-title":"Sparse representation for face recognition: A review paper","volume":"15","author":"Madarkar","year":"2021","journal-title":"IET Image Process."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"19083","DOI":"10.1109\/ACCESS.2022.3151048","article-title":"MLCM: Multi-label confusion matrix","volume":"10","author":"Heydarian","year":"2022","journal-title":"IEEE Access"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/4\/471\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:27:17Z","timestamp":1760106437000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/4\/471"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,12]]},"references-count":52,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["sym16040471"],"URL":"https:\/\/doi.org\/10.3390\/sym16040471","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2024,4,12]]}}}