{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T05:02:34Z","timestamp":1789707754319,"version":"4.0.1"},"reference-count":41,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T00:00:00Z","timestamp":1789084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62271342"],"award-info":[{"award-number":["62271342"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62201377"],"award-info":[{"award-number":["62201377"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["62271342"],"award-info":[{"award-number":["62271342"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"award":["62201377"],"award-info":[{"award-number":["62201377"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Brain Sci.","Brain Sciences"],"abstract":"<jats:p>Background\/Objectives: Hybrid EEG-fNIRS emotion recognition aims to accurately identify an individual\u2019s emotional state by analyzing neurophysiological signals and constitutes an important research direction in affective brain\u2013computer interfaces and human\u2013computer interaction. In recent years, EEG-fNIRS emotion recognition has advanced from handcrafted feature extraction and shallow fusion to deep learning and graph-based modeling. However, most existing methods rely on predefined fixed frequency-band partitioning and second-order graph structures that only support pairwise connections, making it difficult to accommodate inter-subject frequency variability and to characterize high-order brain network relationships such as multi-channel synergistic activation within a frequency band and cross-frequency coupling. Methods: To address these issues, this paper proposes an EEG-fNIRS emotion recognition framework based on a Spatial-Frequency Hypergraph Neural Network (SF-HGNN). First, a Dynamic Frequency Band Decomposition module is designed to achieve adaptive optimization of the EEG and fNIRS frequency bands; second, a Multi-scale Temporal Convolution module extracts temporal features at different time scales; third, a Spatial-Frequency Adaptive Hypergraph Convolution module is constructed to model intra-band cross-channel spatial synergy and channel-wise cross-frequency coupling; and finally, a Cross-Modal Attention Fusion mechanism achieves high-order interaction between the complementary information of the two modalities. The proposed method was validated on the public ENTER dataset comprising 50 participants and four emotion categories (sadness, happiness, fear, and calm). Results: Experimental results show that SF-HGNN achieves accuracies of 82.94% and 68.85% in subject-dependent and subject-independent experiments, respectively; ablation studies and visualization analyses further verify the effectiveness of each module and the interpretability of the model. Conclusions: Future work will focus on validation with larger-scale data and improving cross-subject domain generalization.<\/jats:p>","DOI":"10.3390\/brainsci16090962","type":"journal-article","created":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T14:39:24Z","timestamp":1789137564000},"page":"962","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Spatial-Frequency Hypergraph Neural Network for EEG-fNIRS Emotion Recognition"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7203-3894","authenticated-orcid":false,"given":"Haifeng","family":"Li","sequence":"first","affiliation":[{"name":"College of Integrated Circuits, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xueying","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6063-9127","authenticated-orcid":false,"given":"Guijun","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaru","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Integrated Circuits, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lixia","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030024, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1314","DOI":"10.1109\/JPROC.2023.3277471","article-title":"Affective brain\u2013computer interfaces (aBCIs): A tutorial","volume":"111","author":"Wu","year":"2023","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/TPAMI.2008.52","article-title":"A survey of affect recognition methods: Audio, visual, and spontaneous expressions","volume":"31","author":"Zeng","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1109\/TAMD.2015.2431497","article-title":"Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks","volume":"7","author":"Zheng","year":"2015","journal-title":"IEEE Trans. Auton. Ment. Dev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1109\/TNSRE.2022.3149899","article-title":"FGANet: FNIRS-guided attention network for hybrid EEG-fNIRS brain-computer interfaces","volume":"30","author":"Kwak","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1016\/j.neuroimage.2013.05.004","article-title":"A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology","volume":"85","author":"Scholkmann","year":"2014","journal-title":"NeuroImage"},{"key":"ref_6","first-page":"1","article-title":"EEG based emotion recognition: A tutorial and review","volume":"55","author":"Li","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"126081","DOI":"10.1016\/j.eswa.2024.126081","article-title":"A bidirectional cross-modal transformer representation learning model for EEG-fNIRS multimodal affective BCI","volume":"266","author":"Si","year":"2025","journal-title":"Expert Syst. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, J., Wang, A., Li, S., Zhang, D., and Li, X. (2025). MFA-CNN: An Emotion Recognition Network Integrating 1D\u20132D Convolutional Neural Network and Cross-Modal Causal Features. Brain Sci., 15.","DOI":"10.3390\/brainsci15111165"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"15632","DOI":"10.1109\/TPAMI.2023.3299568","article-title":"Sparse Bayesian learning for end-to-end EEG decoding","volume":"45","author":"Wang","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19489","DOI":"10.1109\/TNNLS.2022.3225855","article-title":"Hierarchical dynamic graph convolutional network with interpretability for EEG-based emotion recognition","volume":"36","author":"Ye","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zheng, W.L., Zhu, J.Y., Peng, Y., and Lu, B.-L. (2014). EEG-based Emotion Classification Using Deep Belief Networks. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China, 14\u201318 July 2014, IEEE.","DOI":"10.1109\/ICME.2014.6890166"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1109\/TAFFC.2014.2339834","article-title":"Feature extraction and selection for emotion recognition from EEG","volume":"5","author":"Jenke","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_13","first-page":"139","article-title":"Investigate the Frequency Spectrum of Noise Signals Affecting Synchronous Signals of fMRI and fNIRS in Resting and Moving Wrists","volume":"6","author":"Far","year":"2019","journal-title":"Front. Biomed. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wang, L., Zhu, J., Du, L., Jin, B., and Wei, X. (2024). FreAML: A Frequency-Domain Adaptive Meta-Learning Framework for EEG-Based Emotion Recognition. Proceedings of the 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbon, Portugal, 3\u20136 December 2024, IEEE.","DOI":"10.1109\/BIBM62325.2024.10822213"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1007\/s11571-022-09890-3","article-title":"Feature hypergraph representation learning on spatial-temporal correlations for EEG emotion recognition","volume":"17","author":"Li","year":"2023","journal-title":"Cogn. Neurodyn.}"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1574","DOI":"10.1109\/LSP.2022.3179946","article-title":"EEG-GCN: Spatio-temporal and self-adaptive graph convolutional networks for single and multi-view EEG-based emotion recognition","volume":"29","author":"Gao","year":"2022","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"102595","DOI":"10.1016\/j.bspc.2021.102595","article-title":"A systematic review on hybrid EEG\/fNIRS in brain-computer interface","volume":"68","author":"Liu","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1109\/TAFFC.2023.3288118","article-title":"PR-PL: A novel prototypical representation based pairwise learning framework for emotion recognition using EEG signals","volume":"15","author":"Zhou","year":"2023","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1109\/TAFFC.2017.2712143","article-title":"Identifying stable patterns over time for emotion recognition from EEG","volume":"10","author":"Zheng","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_20","first-page":"2506312","article-title":"Fusion analysis of EEG-fNIRS multimodal brain signals: A multitask classification algorithm incorporating spatial-temporal convolution and dual attention mechanisms","volume":"74","author":"Shi","year":"2025","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: A compact convolutional neural network for EEG-based brain\u2013computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1109\/TNSRE.2022.3230250","article-title":"EEG conformer: Convolutional transformer for EEG decoding and visualization","volume":"31","author":"Song","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1109\/TNSRE.2024.3355750","article-title":"Graph neural network-based eeg classification: A survey","volume":"32","author":"Klepl","year":"2024","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"126901","DOI":"10.1016\/j.neucom.2023.126901","article-title":"A review of Graph Neural Networks for Electroencephalography data analysis","volume":"562","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1290","DOI":"10.1109\/TAFFC.2020.2994159","article-title":"EEG-based emotion recognition using regularized graph neural networks","volume":"13","author":"Zhong","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"016046","DOI":"10.1088\/1741-2552\/acb79e","article-title":"Emotion recognition using spatial-temporal EEG features through convolutional graph attention network","volume":"20","author":"Li","year":"2023","journal-title":"J. Neural Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TAFFC.2018.2817622","article-title":"EEG emotion recognition using dynamical graph convolutional neural networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"21546","DOI":"10.1109\/ACCESS.2023.3247820","article-title":"Spatial\u2013temporal dynamic graph attention network for skeleton-based action recognition","volume":"11","author":"Rahevar","year":"2023","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kim, S., Lee, S.Y., Gao, Y., Antelmi, A., Polato, M., and Shin, K. (2024, January 25\u201329). A survey on hypergraph neural networks: An in-depth and step-by-step guide. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.","DOI":"10.1145\/3637528.3671457"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3558","DOI":"10.1609\/aaai.v33i01.33013558","article-title":"Hypergraph neural networks","volume":"Volume 33","author":"Feng","year":"2019","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"109057","DOI":"10.1016\/j.neunet.2026.109057","article-title":"Hypergraph multi-modal learning for EEG-based emotion recognition in conversation","volume":"202","author":"Kang","year":"2026","journal-title":"Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, G., Liu, Y., and Zhang, X. (2024). EEG\u2013fNIRS-Based emotion recognition using graph convolution and capsule attention network. Brain Sci., 14.","DOI":"10.3390\/brainsci14080820"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.neucli.2016.07.002","article-title":"Methods for artifact detection and removal from scalp EEG: A review","volume":"46","author":"Islam","year":"2016","journal-title":"Neurophysiol. Clin. Neurophysiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"100901","DOI":"10.1117\/1.JBO.26.10.100901","article-title":"Beer\u2013Lambert law for optical tissue diagnostics: Current state of the art and the main limitations","volume":"26","author":"Oshina","year":"2021","journal-title":"J. Biomed. Opt."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1038\/nn1057","article-title":"Distinct spatial frequency sensitivities for processing faces and emotional expressions","volume":"6","author":"Vuilleumier","year":"2003","journal-title":"Nat. Neurosci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1109\/TAFFC.2017.2714671","article-title":"Emotions recognition using EEG signals: A survey","volume":"10","author":"Alarcao","year":"2019","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Chen, J., Yu, K., Wang, F., Zhou, Z., Bi, Y., Zhuang, S., and Zhang, D. (2024). Temporal convolutional network-enhanced real-time implicit emotion recognition with an innovative wearable fNIRS-EEG dual-modal system. Electronics, 13.","DOI":"10.3390\/electronics13071310"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Arif, A., Wang, Y., Yin, R., Zhang, X., and Helmy, A. (2024). EF-Net: Mental state recognition by analyzing multimodal EEG-fNIRS via CNN. Sensors, 24.","DOI":"10.3390\/s24061889"},{"key":"ref_39","unstructured":"Qiu, L., Feng, W., Ying, Z., and Pan, J. (2024, January 24\u201327). EFMLNet: Fusion model based on end-to-end mutual information learning for hybrid EEG-fNIRS brain-computer interface applications. Proceedings of the Annual Meeting of the Cognitive Science Society, Rotterdam, The Netherlands."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"106943","DOI":"10.1016\/j.bspc.2024.106943","article-title":"E-FNet: A EEG-fNIRS dual-stream model for Brain\u2013Computer Interfaces","volume":"100","author":"Yu","year":"2025","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"103023","DOI":"10.1016\/j.inffus.2025.103023","article-title":"STA-Net: Spatial\u2013temporal alignment network for hybrid EEG-fNIRS decoding","volume":"119","author":"Liu","year":"2025","journal-title":"Inf. Fusion"}],"container-title":["Brain Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3425\/16\/9\/962\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T04:16:36Z","timestamp":1789704996000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3425\/16\/9\/962"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,11]]},"references-count":41,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["brainsci16090962"],"URL":"https:\/\/doi.org\/10.3390\/brainsci16090962","relation":{},"ISSN":["2076-3425"],"issn-type":[{"value":"2076-3425","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,11]]}}}