{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T19:25:04Z","timestamp":1793129104118,"version":"4.5.0"},"reference-count":43,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T00:00:00Z","timestamp":1733788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Emotion recognition using electroencephalography (EEG) is a key aspect of brain-computer interface research. Achieving precision requires effectively extracting and integrating both spatial and temporal features. However, many studies focus on a single dimension, neglecting the interplay and complementarity of multi-feature information, and the importance of fully integrating spatial and temporal dynamics to enhance performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We propose the Spatiotemporal Adaptive Fusion Network (STAFNet), a novel framework combining adaptive graph convolution and temporal transformers to enhance the accuracy and robustness of EEG-based emotion recognition. The model includes an adaptive graph convolutional module to capture brain connectivity patterns through spatial dynamic evolution and a multi-structured transformer fusion module to integrate latent correlations between spatial and temporal features for emotion classification.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Extensive experiments were conducted on the SEED and SEED-IV datasets to evaluate the performance of STAFNet. The model achieved accuracies of 97.89% and 93.64%, respectively, outperforming state-of-the-art methods. Interpretability analyses, including confusion matrices and t-SNE visualizations, were employed to examine the influence of different emotions on the model's recognition performance. Furthermore, an investigation of varying GCN layer depths demonstrated that STAFNet effectively mitigates the over-smoothing issue in deeper GCN architectures.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>In summary, the findings validate the effectiveness of STAFNet in EEG-based emotion recognition. The results emphasize the critical role of spatiotemporal feature extraction and introduce an innovative framework for feature fusion, advancing the state of the art in emotion recognition.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnins.2024.1519970","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T06:34:23Z","timestamp":1733812463000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["STAFNet: an adaptive multi-feature learning network via spatiotemporal fusion for EEG-based emotion recognition"],"prefix":"10.3389","volume":"18","author":[{"given":"Fo","family":"Hu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kailun","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mengyuan","family":"Qian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaofeng","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zukang","family":"Qiao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lekai","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junlong","family":"Xiong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,12,10]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"2976","DOI":"10.3390\/s22082976","article-title":"Deep learning-based approach for emotion recognition using electroencephalography (EEG) signals using bi-directional long short-term memory (BI-LSTM)","volume":"22","author":"Algarni","year":"2022","journal-title":"Sensors"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11571-021-09756-0","article-title":"Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals","volume":"16","author":"Bagherzadeh","year":"2022","journal-title":"Cogn. Neurodyn"},{"key":"B3","doi-asserted-by":"publisher","first-page":"834952","DOI":"10.3389\/fnbot.2022.834952","article-title":"Linking multi-layer dynamical gcn with style-based recalibration cnn for EEG-based emotion recognition","volume":"16","author":"Bao","year":"2022","journal-title":"Front. Neurorobot"},{"key":"B4","doi-asserted-by":"publisher","first-page":"107767","DOI":"10.1016\/j.neuropsychologia.2021.107767","article-title":"Amygdala-prefrontal connectivity during emotion regulation: a meta-analysis of psychophysiological interactions","volume":"153","author":"Berboth","year":"2021","journal-title":"Neuropsychologia"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TPAMI.2021.3137605","article-title":"A comprehensive survey of scene graphs: Generation and application","volume":"45","author":"Chang","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B6","doi-asserted-by":"publisher","first-page":"3592","DOI":"10.1109\/TASLP.2021.3129331","article-title":"Multimodal emotion recognition with temporal and semantic consistency","volume":"29","author":"Chen","year":"2021","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Proc"},{"key":"B7","doi-asserted-by":"publisher","first-page":"118530","DOI":"10.1109\/ACCESS.2019.2936817","article-title":"A hierarchical bidirectional gru model with attention for EEG-based emotion classification","volume":"7","author":"Chen","year":"2019","journal-title":"IEEE Access"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2023.3319315","article-title":"Hybrid network using dynamic graph convolution and temporal self-attention for EEG-based emotion recognition","author":"Cheng","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B9","doi-asserted-by":"publisher","first-page":"4551","DOI":"10.3390\/s20164551","article-title":"CNN and LSTM-based emotion charting using physiological signals","volume":"20","author":"Dar","year":"2020","journal-title":"Sensors"},{"key":"B10","first-page":"81","article-title":"\u201cDifferential entropy feature for eeg-based emotion classification,\u201d","volume-title":"International IEEE\/EMBS Conference on Neural Engineering","author":"Duan","year":"2013"},{"key":"B11","doi-asserted-by":"publisher","first-page":"3878","DOI":"10.1109\/TNSRE.2024.3487216","article-title":"A novel multi-feature fusion network with spatial partitioning strategy and cross-attention for armband-based gesture recognition","volume":"32","author":"Hu","year":"","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B12","doi-asserted-by":"publisher","first-page":"9618","DOI":"10.1109\/TITS.2023.3348517","article-title":"EEG-based driver fatigue detection using spatio-temporal fusion network with brain region partitioning strategy","volume":"25","author":"Hu","year":"","journal-title":"IEEE Trans. Intellig. Transp. Syst"},{"key":"B13","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/TAFFC.2017.2713359","article-title":"Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition","volume":"10","author":"Huang","year":"2019","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1200656","DOI":"10.3389\/fphys.2023.1200656","article-title":"CDBA: a novel multi-branch feature fusion model for eeg-based emotion recognition","volume":"14","author":"Huang","year":"2023","journal-title":"Front. Physiol"},{"key":"B15","doi-asserted-by":"publisher","first-page":"106241","DOI":"10.1016\/j.bspc.2024.106241","article-title":"Enhanced multimodal emotion recognition in healthcare analytics: a deep learning based model-level fusion approach","volume":"94","author":"Islam","year":"2024","journal-title":"Biomed. Signal Process. Control"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2339834","DOI":"10.1109\/TAFFC.2014.2339834","article-title":"Feature extraction and selection for emotion recognition from EEG","volume":"5","author":"Jenke","year":"2014","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2024.3405663","article-title":"An efficient graph learning system for emotion recognition inspired by the cognitive prior graph of eeg brain network","author":"Li","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1169949","DOI":"10.3389\/fnhum.2023.1169949","article-title":"Stgate: spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition","volume":"17","author":"Li","year":"2023","journal-title":"Front. Hum. Neurosci"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1404","DOI":"10.3390\/s23031404","article-title":"Fusion graph representation of EEG for emotion recognition","volume":"23","author":"Li","year":"2023","journal-title":"Sensors"},{"key":"B20","doi-asserted-by":"publisher","first-page":"108747","DOI":"10.1016\/j.measurement.2020.108747","article-title":"Physiological-signal-based emotion recognition: an odyssey from methodology to philosophy","volume":"172","author":"Li","year":"2020","journal-title":"Measurement"},{"key":"B21","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1109\/TAFFC.2017.2714671","article-title":"Emotions recognition using EEG signals: a survey","volume":"10","author":"Meneses Alarc\u00e3o","year":"2017","journal-title":"IEEE Trans. Affective Comp"},{"key":"B22","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.inffus.2021.07.007","article-title":"Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sources","volume":"77","author":"Ngai","year":"2021","journal-title":"Inform. Fusion"},{"key":"B23","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1109\/TNSRE.2023.3236434","article-title":"A brain network analysis-based double way deep neural network for emotion recognition","volume":"31","author":"Niu","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehab. Eng"},{"key":"B24","doi-asserted-by":"publisher","first-page":"107095","DOI":"10.1016\/j.engappai.2023.107095","article-title":"Research on decision-level fusion method based on structural causal model in system-level fault detection and diagnosis","volume":"126","author":"Pu","year":"2023","journal-title":"Eng. Appl. Artif. Intell"},{"key":"B25","doi-asserted-by":"publisher","first-page":"100072","DOI":"10.1016\/j.array.2021.100072","article-title":"Emotion recognition from EEG-based relative power spectral topography using convolutional neural network","volume":"11","author":"Rahman","year":"2021","journal-title":"Array"},{"key":"B26","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1109\/TNSRE.2022.3230250","article-title":"Eeg conformer: convolutional transformer for EEG decoding and visualization","volume":"31","author":"Song","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehab. Eng"},{"key":"B27","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1109\/TNSRE.2023.3342331","article-title":"ADFCNN: attention-based dual-scale fusion convolutional neural network for motor imagery brain-computer interface","volume":"32","author":"Tao","year":"2024","journal-title":"IEEE Trans. Neural Syst. Rehab. Eng"},{"key":"B28","doi-asserted-by":"publisher","first-page":"8079","DOI":"10.1109\/JSEN.2024.3354553","article-title":"Nonlinear signal processing methods for automatic emotion recognition using electrodermal activity","volume":"24","author":"Veeranki","year":"","journal-title":"IEEE Sens. J"},{"key":"B29","doi-asserted-by":"publisher","first-page":"19952","DOI":"10.1109\/ACCESS.2024.3361832","article-title":"Comparison of electrodermal activity signal decomposition techniques for emotion recognition","volume":"12","author":"Veeranki","year":"","journal-title":"IEEE Access"},{"key":"B30","doi-asserted-by":"publisher","first-page":"78123","DOI":"10.1109\/ACCESS.2022.3194039","article-title":"Investigation of the effect of increased dimension levels in speech emotion recognition","volume":"10","author":"Wang","year":"2022","journal-title":"IEEE Access"},{"key":"B31","doi-asserted-by":"publisher","first-page":"93711","DOI":"10.1109\/ACCESS.2019.2927768","article-title":"Phase-locking value based graph convolutional neural networks for emotion recognition","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"B32","doi-asserted-by":"publisher","first-page":"101756","DOI":"10.1016\/j.bspc.2019.101756","article-title":"Eeg-based emotion recognition using simple recurrent units network and ensemble learning","volume":"58","author":"Wei","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"B33","doi-asserted-by":"publisher","first-page":"5383","DOI":"10.1109\/TII.2023.3333914","article-title":"Privacy-preserving federated learning for power transformer fault diagnosis with unbalanced data","volume":"20","author":"Wu","year":"2024","journal-title":"IEEE Trans. Indust. Inform"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11571-021-09751-5","article-title":"4D attention-based neural network for EEG emotion recognition","volume":"16","author":"Xiao","year":"2022","journal-title":"Cogn. Neurodyn"},{"key":"B35","doi-asserted-by":"publisher","first-page":"9733","DOI":"10.1109\/TPAMI.2021.3127346","article-title":"ZeroNAS: differentiable generative adversarial networks search for zero-shot learning","volume":"44","author":"Yan","year":"2022","journal-title":"IEEE Trans. Pattern Analy. Mach. Intellig"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3216829","article-title":"Siam-GCAN: a siamese graph convolutional attention network for EEG emotion recognition","volume":"71","author":"Zeng","year":"2022","journal-title":"IEEE Trans. Instrum. Meas"},{"key":"B37","doi-asserted-by":"publisher","first-page":"934","DOI":"10.1109\/TAES.2023.3331339","article-title":"Feature-level fusion recognition of space targets with composite micromotion","volume":"60","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Aerosp. Electron. Syst"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1110","DOI":"10.1109\/TCYB.2018.2797176","article-title":"Emotionmeter: a multimodal framework for recognizing human emotions","volume":"49","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Cybern"},{"key":"B39","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1109\/TAMD.2015.2431497","article-title":"Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks","volume":"7","author":"Zheng","year":"2015","journal-title":"IEEE Trans. Auton. Ment. Dev"},{"key":"B40","doi-asserted-by":"publisher","first-page":"10389","DOI":"10.1109\/TPAMI.2024.3442811","article-title":"A new brain network construction paradigm for brain disorder via diffusion-based graph contrastive learning","volume":"46","author":"Zong","year":"2024","journal-title":"IEEE Trans. Pattern Analy. Mach. Intellig"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2024.3425949","article-title":"BDHT: generative AI enables causality analysis for mild cognitive impairment","author":"Zuo","year":"","journal-title":"IEEE Trans. Automat. Sci. Eng"},{"key":"B42","doi-asserted-by":"publisher","first-page":"3652","DOI":"10.1109\/TCYB.2023.3344641","article-title":"Prior-guided adversarial learning with hypergraph for predicting abnormal connections in Alzheimer's Disease","volume":"54","author":"Zuo","year":"","journal-title":"IEEE Trans. Cybernet"},{"key":"B43","doi-asserted-by":"publisher","first-page":"4017","DOI":"10.1109\/TNSRE.2023.3323432","article-title":"Brain structure-function fusing representation learning using adversarial decomposed-VAE for analyzing MCI","volume":"31","author":"Zuo","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehab. Eng"}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2024.1519970\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T06:34:27Z","timestamp":1733812467000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2024.1519970\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,10]]},"references-count":43,"alternative-id":["10.3389\/fnins.2024.1519970"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2024.1519970","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,10]]},"article-number":"1519970"}}