{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T05:27:07Z","timestamp":1774934827445,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":38,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,5,25]],"date-time":"2025-05-25T00:00:00Z","timestamp":1748131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Outstanding Award for Talent Project of the Chinese Academy of Sciences","award":["29J20-052-III"],"award-info":[{"award-number":["29J20-052-III"]}]},{"name":"The Key Laboratory of Biomedical Spectroscopy of Xi'an","award":["201805050ZD1CG34"],"award-info":[{"award-number":["201805050ZD1CG34"]}]},{"name":"Shaanxi Province Technological Innovation Guidance Special Project: Regional Science and Technology Innovation Center, Strategic Scientific and Technological Strength Category","award":["2024QY-SZX-26"],"award-info":[{"award-number":["2024QY-SZX-26"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,5,26]]},"DOI":"10.1145\/3715669.3723106","type":"proceedings-article","created":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T06:57:59Z","timestamp":1748069879000},"page":"1-6","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["MSNet: Multimodal Self-attention Network for Depression Detection via Fusion of Eye Tracking and EEG"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-1669-9789","authenticated-orcid":false,"given":"Feiyu","family":"Zhu","sequence":"first","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China and University of Chinese Academy of Sciences, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8735-946X","authenticated-orcid":false,"given":"Bingbing","family":"Wu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China and University of Chinese Academy of Sciences, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5035-483X","authenticated-orcid":false,"given":"Yongsheng","family":"Huo","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China and University of Chinese Academy of Sciences, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0221-7767","authenticated-orcid":false,"given":"Ruochen","family":"Dang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3216-5013","authenticated-orcid":false,"given":"Bingliang","family":"Hu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4826-2408","authenticated-orcid":false,"given":"Quan","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics (XIOPM), Chinese Academy of Sciences, Xi An, China"}]}],"member":"320","published-online":{"date-parts":[[2025,5,25]]},"reference":[{"key":"e_1_3_3_1_2_1","doi-asserted-by":"crossref","unstructured":"Thomas Armstrong and Bunmi\u00a0O Olatunji. 2012. Eye tracking of attention in the affective disorders: A meta-analytic review and synthesis. Clinical psychology review 32 8 (2012) 704\u2013723.","DOI":"10.1016\/j.cpr.2012.09.004"},{"key":"e_1_3_3_1_3_1","doi-asserted-by":"crossref","unstructured":"Said\u00a0Yacine Boulahia Abdenour Amamra Mohamed\u00a0Ridha Madi and Said Daikh. 2021. Early intermediate and late fusion strategies for robust deep learning-based multimodal action recognition. Machine Vision and Applications 32 6 (2021) 121.","DOI":"10.1007\/s00138-021-01249-8"},{"key":"e_1_3_3_1_4_1","doi-asserted-by":"crossref","unstructured":"Nicola\u00a0Luigi Bragazzi and Giovanni Del\u00a0Puente. 2014. A proposal for including nomophobia in the new DSM-V. Psychology research and behavior management (2014) 155\u2013160.","DOI":"10.2147\/PRBM.S41386"},{"key":"e_1_3_3_1_5_1","doi-asserted-by":"crossref","unstructured":"Alexander Craik Yongtian He and Jose\u00a0L Contreras-Vidal. 2019. Deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of neural engineering 16 3 (2019) 031001.","DOI":"10.1088\/1741-2552\/ab0ab5"},{"key":"e_1_3_3_1_6_1","doi-asserted-by":"crossref","unstructured":"Arnaud Delorme and Scott Makeig. 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of neuroscience methods 134 1 (2004) 9\u201321.","DOI":"10.1016\/j.jneumeth.2003.10.009"},{"key":"e_1_3_3_1_7_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et\u00a0al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2010.11929 (2020)."},{"key":"e_1_3_3_1_8_1","doi-asserted-by":"crossref","unstructured":"Xianliang Ge Yunxian Pan Sujie Wang Linze Qian Jingjia Yuan Jie Xu Nitish Thakor and Yu Sun. 2022. Improving intention detection in single-trial classification through fusion of EEG and eye-tracker data. IEEE Transactions on Human-Machine Systems 53 1 (2022) 132\u2013141.","DOI":"10.1109\/THMS.2022.3225633"},{"key":"e_1_3_3_1_9_1","doi-asserted-by":"crossref","unstructured":"Manuel Gra\u00f1a and Igone Morais-Quilez. 2023. A review of Graph Neural Networks for Electroencephalography data analysis. Neurocomputing 562 (2023) 126901.","DOI":"10.1016\/j.neucom.2023.126901"},{"key":"e_1_3_3_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2019.8856563"},{"key":"e_1_3_3_1_11_1","doi-asserted-by":"crossref","unstructured":"Benjamin\u00a0L Hankin. 2006. Adolescent depression: Description causes and interventions. Epilepsy & behavior 8 1 (2006) 102\u2013114.","DOI":"10.1016\/j.yebeh.2005.10.012"},{"key":"e_1_3_3_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_3_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00745"},{"key":"e_1_3_3_1_14_1","doi-asserted-by":"crossref","unstructured":"Suhaima Jamal Meenalosini\u00a0Vimal Cruz Sibi Chakravarthy Camden Wahl and Hayden Wimmer. 2023. Integration of EEG and eye tracking technology: a systematic review. SoutheastCon 2023 (2023) 209\u2013216.","DOI":"10.1109\/SoutheastCon51012.2023.10115167"},{"key":"e_1_3_3_1_15_1","doi-asserted-by":"crossref","unstructured":"Hee\u00a0E Kim Alejandro Cosa-Linan Nandhini Santhanam Mahboubeh Jannesari Mate\u00a0E Maros and Thomas Ganslandt. 2022. Transfer learning for medical image classification: a literature review. BMC medical imaging 22 1 (2022) 69.","DOI":"10.1186\/s12880-022-00793-7"},{"key":"e_1_3_3_1_16_1","unstructured":"Thomas\u00a0N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1609.02907 (2016)."},{"key":"e_1_3_3_1_17_1","doi-asserted-by":"crossref","unstructured":"Sander Koelstra and Ioannis Patras. 2013. Fusion of facial expressions and EEG for implicit affective tagging. Image and Vision Computing 31 2 (2013) 164\u2013174.","DOI":"10.1016\/j.imavis.2012.10.002"},{"key":"e_1_3_3_1_18_1","doi-asserted-by":"crossref","unstructured":"Dana Lahat T\u00fclay Adali and Christian Jutten. 2015. Multimodal data fusion: an overview of methods challenges and prospects. Proc. IEEE 103 9 (2015) 1449\u20131477.","DOI":"10.1109\/JPROC.2015.2460697"},{"key":"e_1_3_3_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP42928.2021.9506702"},{"key":"e_1_3_3_1_20_1","doi-asserted-by":"crossref","unstructured":"Lei LIU and Renlai ZHOU. 2015. An important neural indicator of measuring depression: The asymmetry of resting frontal activity. Advances in Psychological Science 23 6 (2015) 1000.","DOI":"10.3724\/SP.J.1042.2015.01000"},{"key":"e_1_3_3_1_21_1","unstructured":"Diyuan Lu and Jochen Triesch. 2019. Residual deep convolutional neural network for EEG signal classification in epilepsy. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1903.08100 (2019)."},{"key":"e_1_3_3_1_22_1","unstructured":"Ravikiran Mane Effie Chew Karen Chua Kai\u00a0Keng Ang Neethu Robinson A\u00a0Prasad Vinod Seong-Whan Lee and Cuntai Guan. 2021. FBCNet: A multi-view convolutional neural network for brain-computer interface. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2104.01233 (2021)."},{"key":"e_1_3_3_1_23_1","doi-asserted-by":"crossref","unstructured":"D Merlin\u00a0Praveena D Angelin\u00a0Sarah and S Thomas\u00a0George. 2022. Deep learning techniques for EEG signal applications\u2013a review. IETE journal of Research 68 4 (2022) 3030\u20133037.","DOI":"10.1080\/03772063.2020.1749143"},{"key":"e_1_3_3_1_24_1","doi-asserted-by":"crossref","unstructured":"Ghulam Muhammad Fatima Alshehri Fakhri Karray Abdulmotaleb El\u00a0Saddik Mansour Alsulaiman and Tiago\u00a0H Falk. 2021. A comprehensive survey on multimodal medical signals fusion for smart healthcare systems. Information Fusion 76 (2021) 355\u2013375.","DOI":"10.1016\/j.inffus.2021.06.007"},{"key":"e_1_3_3_1_25_1","unstructured":"World\u00a0Health Organization. 2001. The World Health Report 2001: Mental health: new understanding new hope. (2001)."},{"key":"e_1_3_3_1_26_1","doi-asserted-by":"crossref","unstructured":"Eelco\u00a0AB Over Ignace\u00a0TC Hooge and Casper\u00a0J Erkelens. 2006. A quantitative measure for the uniformity of fixation density: The Voronoi method. Behavior research methods 38 (2006) 251\u2013261.","DOI":"10.3758\/BF03192777"},{"key":"e_1_3_3_1_27_1","doi-asserted-by":"crossref","unstructured":"Jiahui Pan Jianhao Zhang Fei Wang Wuhan Liu Haiyun Huang Weishun Tang Huijian Liao Man Li Jianhui Wu Xueli Li et\u00a0al. 2021. Automatic Sleep Staging Based on EEG-EOG Signals for Depression Detection.Intelligent Automation & Soft Computing 28 1 (2021).","DOI":"10.32604\/iasc.2021.015970"},{"key":"e_1_3_3_1_28_1","doi-asserted-by":"crossref","unstructured":"Vasileios Skaramagkas Giorgos Giannakakis Emmanouil Ktistakis Dimitris Manousos Ioannis Karatzanis Nikolaos\u00a0S Tachos Evanthia Tripoliti Kostas Marias Dimitrios\u00a0I Fotiadis and Manolis Tsiknakis. 2021. Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Reviews in Biomedical Engineering 16 (2021) 260\u2013277.","DOI":"10.1109\/RBME.2021.3066072"},{"key":"e_1_3_3_1_29_1","doi-asserted-by":"crossref","unstructured":"Yonghao Song Qingqing Zheng Bingchuan Liu and Xiaorong Gao. 2022. EEG conformer: Convolutional transformer for EEG decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering 31 (2022) 710\u2013719.","DOI":"10.1109\/TNSRE.2022.3230250"},{"key":"e_1_3_3_1_30_1","doi-asserted-by":"crossref","unstructured":"Abhishek Srivastava Debesh Jha Sukalpa Chanda Umapada Pal H\u00e5vard\u00a0D Johansen Dag Johansen Michael\u00a0A Riegler Sharib Ali and P\u00e5l Halvorsen. 2021. MSRF-Net: a multi-scale residual fusion network for biomedical image segmentation. IEEE Journal of Biomedical and Health Informatics 26 5 (2021) 2252\u20132263.","DOI":"10.1109\/JBHI.2021.3138024"},{"key":"e_1_3_3_1_31_1","doi-asserted-by":"crossref","unstructured":"S\u00f6ren\u00a0Richard Stahlschmidt Benjamin Ulfenborg and Jane Synnergren. 2022. Multimodal deep learning for biomedical data fusion: a review. Briefings in bioinformatics 23 2 (2022) bbab569.","DOI":"10.1093\/bib\/bbab569"},{"key":"e_1_3_3_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_3_1_33_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_3_1_34_1","doi-asserted-by":"crossref","unstructured":"Enqiang Wang Qing Yu Yelin Chen Wushouer Slamu and Xukang Luo. 2022. Multi-modal knowledge graphs representation learning via multi-headed self-attention. Information Fusion 88 (2022) 78\u201385.","DOI":"10.1016\/j.inffus.2022.07.008"},{"key":"e_1_3_3_1_35_1","doi-asserted-by":"crossref","unstructured":"Quan Wang Feiyu Zhu Ruochen Dang Xiaojie Wei Gongen Han Jinhua Huang and Bingliang Hu. 2023. An eye tracking investigation of attention mechanism in driving behavior under emotional issues and cognitive load. Scientific Reports 13 1 (2023) 16963.","DOI":"10.1038\/s41598-023-43693-8"},{"key":"e_1_3_3_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM52615.2021.9669556"},{"key":"e_1_3_3_1_37_1","doi-asserted-by":"crossref","unstructured":"Sana Yasin Syed\u00a0Asad Hussain Sinem Aslan Imran Raza Muhammad Muzammel and Alice Othmani. 2021. EEG based Major Depressive disorder and Bipolar disorder detection using Neural Networks: A review. Computer Methods and Programs in Biomedicine 202 (2021) 106007.","DOI":"10.1016\/j.cmpb.2021.106007"},{"key":"e_1_3_3_1_38_1","doi-asserted-by":"crossref","unstructured":"Sana Yasin Alice Othmani Imran Raza and Syed\u00a0Asad Hussain. 2023. Machine learning based approaches for clinical and non-clinical depression recognition and depression relapse prediction using audiovisual and EEG modalities: A comprehensive review. Computers in Biology and Medicine 159 (2023) 106741.","DOI":"10.1016\/j.compbiomed.2023.106741"},{"key":"e_1_3_3_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM55620.2022.9995663"}],"event":{"name":"ETRA '25: 2025 Symposium on Eye Tracking Research and Applications","location":"Tokyo Japan","acronym":"ETRA '25","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGGRAPH ACM Special Interest Group on Computer Graphics and Interactive Techniques"]},"container-title":["Proceedings of the 2025 Symposium on Eye Tracking Research and Applications"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715669.3723106","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:18Z","timestamp":1750295898000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715669.3723106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,25]]},"references-count":38,"alternative-id":["10.1145\/3715669.3723106","10.1145\/3715669"],"URL":"https:\/\/doi.org\/10.1145\/3715669.3723106","relation":{},"subject":[],"published":{"date-parts":[[2025,5,25]]},"assertion":[{"value":"2025-05-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}