{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T21:51:32Z","timestamp":1783979492553,"version":"3.55.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,7,23]],"date-time":"2024-07-23T00:00:00Z","timestamp":1721692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202367"],"award-info":[{"award-number":["62202367"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Project of China Knowledge Centre for Engineering Science and Technology"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Emotion recognition based on electroencephalography (EEG) is becoming a growing concern of researchers due to its various applications and portable devices. Existing methods are mainly dedicated to EEG feature representation and have made impressive progress. However, the problem of scarce labels restricts their further promotion. In light of this, we propose a self-supervised framework with contrastive learning for robust EEG-based emotion recognition, which can effectively leverage both readily available unlabeled EEG signals and labeled ones to learn highly discriminative EEG features. Firstly, we construct a specific pretext task according to the sequential non-stationarity of emotional EEG signals for contrastive learning, which aims at extracting pseudo-label information from all EEG data. Meanwhile, we propose a novel negative segment selection algorithm to reduce the noise of unlabeled data during the contrastive learning process. Secondly, to mitigate the overfitting issue induced by a small number of labeled samples during learning, we originate a loss function with label smoothing regularization that can guide the model to learn generalizable features. Extensive experiments over three benchmark datasets demonstrate the effectiveness and superiority of our model on EEG-based emotion recognition task. Besides, the generalization and robustness of the model have also been proved through sufficient experiments.<\/jats:p>","DOI":"10.1145\/3674975","type":"journal-article","created":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T11:13:48Z","timestamp":1720178028000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Self-Supervised EEG Representation Learning for Robust Emotion Recognition"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7863-3751","authenticated-orcid":false,"given":"Huan","family":"Liu","sequence":"first","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7110-7396","authenticated-orcid":false,"given":"Yuzhe","family":"Zhang","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1809-3405","authenticated-orcid":false,"given":"Xuxu","family":"Chen","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5869-6544","authenticated-orcid":false,"given":"Dalin","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5453-4660","authenticated-orcid":false,"given":"Rui","family":"Li","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7674-698X","authenticated-orcid":false,"given":"Tao","family":"Qin","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,7,23]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519304"},{"issue":"10","key":"e_1_3_1_3_2","first-page":"355","article-title":"Emotion recognition based on EEG using LSTM recurrent neural network","volume":"8","author":"Alhagry Salma","year":"2017","unstructured":"Salma Alhagry, Aly Aly Fahmy, and Reda A. El-Khoribi. 2017. Emotion recognition based on EEG using LSTM recurrent neural network. Emotion 8, 10 (2017), 355\u2013358.","journal-title":"Emotion"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/abca18"},{"key":"e_1_3_1_5_2","volume-title":"Proceedings of the ICLR","author":"Bashivan Pouya","year":"2016","unstructured":"Pouya Bashivan, Irina Rish, Mohammed Yeasin, and Noel Codella. 2016. Learning representations from EEG with deep recurrent-convolutional neural networks. In Proceedings of the ICLR."},{"key":"e_1_3_1_6_2","first-page":"5050","volume-title":"Proceedings of the NIPS","author":"Berthelot David","year":"2019","unstructured":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A. Raffel. 2019. Mixmatch: A holistic approach to semi-supervised learning. In Proceedings of the NIPS. 5050\u20135060."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3560821"},{"key":"e_1_3_1_9_2","unstructured":"Nicholas Carlini Anish Athalye Nicolas Papernot Wieland Brendel Jonas Rauber Dimitris Tsipras Ian Goodfellow Aleksander Madry and Alexey Kurakin. 2019. On evaluating adversarial robustness. arXiv:1902.06705. Retrieved from https:\/\/arxiv.org\/abs\/1902.06705"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3392049"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3565973"},{"key":"e_1_3_1_12_2","first-page":"8270","article-title":"Adversarial distributional training for robust deep learning","volume":"33","author":"Dong Yinpeng","year":"2020","unstructured":"Yinpeng Dong, Zhijie Deng, Tianyu Pang, Jun Zhu, and Hang Su. 2020. Adversarial distributional training for robust deep learning. Advances in Neural Information Processing Systems 33 (2020), 8270\u20138283.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_13_2","article-title":"An efficient LSTM network for emotion recognition from multichannel EEG signals","author":"Du Xiaobing","year":"2020","unstructured":"Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng, Yong-Jin Liu, and Hongan Wang. 2020. An efficient LSTM network for emotion recognition from multichannel EEG signals. IEEE Transactions on Affective Computing 13, 3 (2020), 1528\u20131540.","journal-title":"IEEE Transactions on Affective Computing"},{"key":"e_1_3_1_14_2","first-page":"81","volume-title":"Proceedings of the NER","author":"Duan Ruo-Nan","year":"2013","unstructured":"Ruo-Nan Duan, Jia-Yi Zhu, and Bao-Liang Lu. 2013. Differential entropy feature for EEG-based emotion classification. In Proceedings of the NER. 81\u201384."},{"issue":"6","key":"e_1_3_1_15_2","doi-asserted-by":"crossref","first-page":"2266","DOI":"10.1109\/JSEN.2018.2883497","article-title":"Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals","volume":"19","author":"Gupta Vipin","year":"2018","unstructured":"Vipin Gupta, Mayur Dahyabhai Chopda, and Ram Bilas Pachori. 2018. Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals. IEEE Sensors Journal 19, 6 (2018), 2266\u20132274.","journal-title":"IEEE Sensors Journal"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1162\/0899766042321814"},{"key":"e_1_3_1_17_2","first-page":"16000","volume-title":"Proceedings of the CVPR","author":"He Kaiming","year":"2022","unstructured":"Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll\u00e1r, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the CVPR. 16000\u201316009."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_19_2","first-page":"3326","volume-title":"Proceedings of the ICCV","author":"Huang Jinchi","year":"2019","unstructured":"Jinchi Huang, Lie Qu, Rongfei Jia, and Binqiang Zhao. 2019. O2u-net: A simple noisy label detection approach for deep neural networks. In Proceedings of the ICCV. 3326\u20133334."},{"key":"e_1_3_1_20_2","first-page":"3765","volume-title":"Proceedings of the NIPS","author":"Hyvarinen Aapo","year":"2016","unstructured":"Aapo Hyvarinen and Hiroshi Morioka. 2016. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. In Proceedings of the NIPS. 3765\u20133773."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3281630"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.15"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475697"},{"key":"e_1_3_1_24_2","unstructured":"Wei-Hong Li Chuan-Sheng Foo and Hakan Bilen. 2019. Learning to impute: A general framework for semi-supervised learning. arXiv:1912.10364. Retrieved from https:\/\/arxiv.org\/abs\/1912.10364"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2885474"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.04.003"},{"key":"e_1_3_1_27_2","first-page":"1913\u2013 1925","article-title":"EEGFuseNet: Hybrid unsupervised deep feature characterization and fusion for high-dimensional EEG with an application to emotion recognition","volume":"29","author":"Liang Zhen","year":"2021","unstructured":"Zhen Liang, Rushuang Zhou, Li Zhang, Linling Li, Gan Huang, Zhiguo Zhang, and Shin Ishii. 2021. EEGFuseNet: Hybrid unsupervised deep feature characterization and fusion for high-dimensional EEG with an application to emotion recognition. IEEE Transactions on Neural Systems and Rehabilitation Engineering 29 (2021), 1913\u2013 1925.","journal-title":"IEEE Transactions on Neural Systems and Rehabilitation Engineering"},{"key":"e_1_3_1_28_2","article-title":"EEG-based multimodal emotion recognition: A machine learning perspective","author":"Liu Huan","year":"2024","unstructured":"Huan Liu, Tianyu Lou, Yuzhe Zhang, Yixiao Wu, Yang Xiao, Christian S. Jensen, and Dalin Zhang. 2024. EEG-based multimodal emotion recognition: A machine learning perspective. IEEE Transactions on Instrumentation and Measurement 73 (2024), 1\u201329.","journal-title":"IEEE Transactions on Instrumentation and Measurement"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1080\/02699930802204677"},{"key":"e_1_3_1_30_2","first-page":"238","volume-title":"Proceedings of the Machine Learning for Health","author":"Mohsenvand Mostafa Neo","year":"2020","unstructured":"Mostafa Neo Mohsenvand, Mohammad Rasool Izadi, and Pattie Maes. 2020. Contrastive representation learning for electroencephalogram classification. In Proceedings of the Machine Learning for Health. PMLR, 238\u2013253."},{"key":"e_1_3_1_31_2","article-title":"When does label smoothing help?","volume":"32","author":"M\u00fcller Rafael","year":"2019","unstructured":"Rafael M\u00fcller, Simon Kornblith, and Geoffrey E. Hinton. 2019. When does label smoothing help? Advances in Neural Information Processing Systems 32 (2019), 4696\u20134705.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2021.3124056","article-title":"Self-weighted semi-supervised classification for joint EEG-based emotion recognition and affective activation patterns mining","volume":"70","author":"Peng Yong","year":"2021","unstructured":"Yong Peng, Wanzeng Kong, Feiwei Qin, Feiping Nie, Jinglong Fang, Bao-Liang Lu, and Andrzej Cichocki. 2021. Self-weighted semi-supervised classification for joint EEG-based emotion recognition and affective activation patterns mining. IEEE Transactions on Instrumentation and Measurement 70 (2021), 1\u201311.","journal-title":"IEEE Transactions on Instrumentation and Measurement"},{"key":"e_1_3_1_33_2","volume-title":"Emotions and Life: Perspectives from Psychology, Biology, and Evolution.","author":"Plutchik Robert","year":"2003","unstructured":"Robert Plutchik. 2003. Emotions and Life: Perspectives from Psychology, Biology, and Evolution.American Psychological Association."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2018.090843"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2022.3147876"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3164516"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.25"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2817622"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018628609742"},{"key":"e_1_3_1_40_2","first-page":"268","volume-title":"Proceedings of the ECCV","author":"Gansbeke Wouter Van","year":"2020","unstructured":"Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. 2020. Scan: Learning to classify images without labels. In Proceedings of the ECCV. 268\u2013285."},{"key":"e_1_3_1_41_2","article-title":"UltraCLR: Contrastive representation learning framework for ultrasound-based sensing","author":"Wang Xun","year":"2024","unstructured":"Xun Wang, Zhizheng Yang, Wei Wang, Haipeng Dai, ShuYu Shi, and Qing Gu. 2024. UltraCLR: Contrastive representation learning framework for ultrasound-based sensing. ACM Transactions on Sensor Networks 20, 4 (2024), 1\u201323.","journal-title":"ACM Transactions on Sensor Networks"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aced22"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3407899"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3089942"},{"key":"e_1_3_1_45_2","article-title":"Cross-modal credibility modelling for EEG-based multimodal emotion recognition","author":"Zhang Yuzhe","year":"2024","unstructured":"Yuzhe Zhang, Huan Liu, Di Wang, Dalin Zhang, Tianyu Lou, Qinghua Zheng, and Chai Quek. 2024. Cross-modal credibility modelling for EEG-based multimodal emotion recognition. Journal of Neural Engineering 21, 2 (2024), 026040.","journal-title":"Journal of Neural Engineering"},{"key":"e_1_3_1_46_2","article-title":"EEG-based emotion recognition with emotion localization via hierarchical self-attention","author":"Zhang Yuzhe","year":"2022","unstructured":"Yuzhe Zhang, Huan Liu, Dalin Zhang, Xuxu Chen, Tao Qin, and Qinghua Zheng. 2022. EEG-based emotion recognition with emotion localization via hierarchical self-attention. IEEE Transactions on Affective Computing 14, 3 (2023), 2458\u20132469.","journal-title":"IEEE Transactions on Affective Computing"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2015.2431497"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.2994159"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3674975","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3674975","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:56Z","timestamp":1750291556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3674975"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,23]]},"references-count":47,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3674975"],"URL":"https:\/\/doi.org\/10.1145\/3674975","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,23]]},"assertion":[{"value":"2024-03-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}