{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T05:24:12Z","timestamp":1780637052942,"version":"3.54.1"},"reference-count":32,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["No. 2017YFB1002505"],"award-info":[{"award-number":["No. 2017YFB1002505"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61601028"],"award-info":[{"award-number":["No. 61601028"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61431007"],"award-info":[{"award-number":["No. 61431007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U2241208"],"award-info":[{"award-number":["U2241208"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>At present, single-modal brain-computer interface (BCI) still has limitations in practical application, such as low flexibility, poor autonomy, and easy fatigue for subjects. This study developed an asynchronous robotic arm control system based on steady-state visual evoked potentials (SSVEP) and eye-tracking in virtual reality (VR) environment, including simultaneous and sequential modes. For simultaneous mode, target classification was realized by decision-level fusion of electroencephalography (EEG) and eye-gaze. The stimulus duration for each subject was non-fixed, which was determined by an adjustable window method. Subjects could autonomously control the start and stop of the system using triple blink and eye closure, respectively. For sequential mode, no calibration was conducted before operation. First, subjects\u2019 gaze area was obtained through eye-gaze, and then only few stimulus blocks began to flicker. Next, target classification was determined using EEG. Additionally, subjects could reject false triggering commands using eye closure. In this study, the system effectiveness was verified through offline experiment and online robotic-arm grasping experiment. Twenty subjects participated in offline experiment. For simultaneous mode, average ACC and ITR at the stimulus duration of 0.9 s were 90.50% and 60.02 bits\/min, respectively. For sequential mode, average ACC and ITR at the stimulus duration of 1.4 s were 90.47% and 45.38 bits\/min, respectively. Fifteen subjects successfully completed the online tasks of grabbing balls in both modes, and most subjects preferred the sequential mode. The proposed hybrid brain-computer interface (h-BCI) system could increase autonomy, reduce visual fatigue, meet individual needs, and improve the efficiency of the system.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1146415","type":"journal-article","created":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T05:02:11Z","timestamp":1679893331000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["A robotic arm control system with simultaneous and sequential modes combining eye-tracking with steady-state visual evoked potential in virtual reality environment"],"prefix":"10.3389","volume":"17","author":[{"given":"Rongxiao","family":"Guo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanfei","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xi","family":"Luo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaorong","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shangen","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,3,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/ac3044","article-title":"Adaptive asynchronous control system of robotic arm based on augmented reality-assisted brain\u2013computer interface.","volume":"18","author":"Chen","year":"2021","journal-title":"J. Neural Eng."},{"key":"B2","doi-asserted-by":"publisher","first-page":"3140","DOI":"10.1109\/TNSRE.2020.3038209","article-title":"Combination of augmented reality based brain-computer interface and computer vision for high-level control of a robotic arm.","volume":"28","author":"Chen","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabilitation Eng."},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/12\/4\/046008","article-title":"Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain\u2013computer interface.","volume":"12","author":"Chen","year":"","journal-title":"J. Neural Eng."},{"key":"B4","doi-asserted-by":"publisher","first-page":"E6058","DOI":"10.1073\/pnas.1508080112","article-title":"High-speed spelling with a noninvasive brain\u2013computer interface.","volume":"112","author":"Chen","year":"","journal-title":"Proc. Natl. Acad. Sci. U S A."},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aaf594","article-title":"Combination of high-frequency SSVEP-based BCI and computer vision for controlling a robotic arm.","volume":"16","author":"Chen","year":"2019","journal-title":"J. Neural Eng."},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065718500181","article-title":"Control of a 7-DOF robotic arm system with an SSVEP-based BCI.","volume":"28","author":"Chen","year":"2018","journal-title":"Int. J. Neural Syst."},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-8-39","article-title":"Asynchronous BCI control using high-frequency SSVEP.","volume":"8","author":"Diez","year":"2011","journal-title":"J. Neuroeng. Rehabilitation"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aaw6844","article-title":"Noninvasive neuroimaging enhances continuous neural tracking for robotic device control.","volume":"4","author":"Edelman","year":"2019","journal-title":"Sci. Robot."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1155\/2017\/8316485","article-title":"Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system.","volume":"2017","author":"Gao","year":"2017","journal-title":"BioMed Res. Int."},{"key":"B10","doi-asserted-by":"publisher","first-page":"1226","DOI":"10.1109\/TNSRE.2020.2981659","article-title":"Brain-controlled robotic arm system based on multi\u2013directional CNN\u2013BiLSTM network using EEG signals.","volume":"28","author":"Jeong","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabilitation Eng."},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/ab4dc6","article-title":"An online SSVEP-BCI system in an optical see-through augmented reality environment.","volume":"17","author":"Ke","year":"2020","journal-title":"J. Neural Eng."},{"key":"B12","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1109\/TCIAIG.2012.2230003","article-title":"Evaluation and comparison of a multimodal combination of BCI paradigms and eye tracking with affordable consumer-grade hardware in a gaming context.","volume":"5","author":"Kos\u2019 Myna","year":"2013","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"B13","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.bspc.2015.05.012","article-title":"Development of a hybrid mental spelling system combining SSVEP-based brain\u2013computer interface and webcam-based eye tracking.","volume":"21","author":"Lim","year":"2015","journal-title":"Biomed. Signal Process. Control"},{"key":"B14","doi-asserted-by":"publisher","first-page":"2610","DOI":"10.1109\/tbme.2006.889197","article-title":"Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs.","volume":"53","author":"Lin","year":"2006","journal-title":"IEEE. Trans. Biomed. Eng."},{"key":"B15","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1109\/TCDS.2018.2875052","article-title":"Motor-Imagery-Based teleoperation of a dual-arm robot performing manipulation tasks.","volume":"11","author":"Liu","year":"2018","journal-title":"IEEE Trans. Cogn. Develop. Syst."},{"key":"B16","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1145\/3172944.3172988","article-title":"Combining brain-computer interface and eye tracking for high-speed text entry in virtual reality","author":"Ma","year":"2018","journal-title":"Proceedings of the 23rd International Conference on Intelligent User Interfaces"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.3390\/s20030891","article-title":"A hybrid speller design using eye tracking and SSVEP brain\u2013computer interface.","volume":"20","author":"Mannan","year":"2020","journal-title":"Sensors"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1038\/srep38565","article-title":"Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks.","volume":"6","author":"Meng","year":"2016","journal-title":"Sci. Rep."},{"key":"B19","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1109\/TBME.2017.2694818","article-title":"Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis.","volume":"65","author":"Nakanishi","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.3389\/fnpro.2010.00003","article-title":"The hybrid BCI.","volume":"4","author":"Pfurtscheller","year":"","journal-title":"Front. Neurosci."},{"key":"B21","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1109\/TNSRE.2010.2040837","article-title":"Self-paced operation of an SSVEP-Based orthosis with and without an imagery-based \u201cbrain switch:\u201d a feasibility study towards a hybrid BCI.","volume":"18","author":"Pfurtscheller","year":"","journal-title":"IEEE Trans. Neural Syst. Rehabilitation Eng."},{"key":"B22","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1109\/FIT.2018.00025","article-title":"Mesh of SSVEP-based BCI and eye-tracker for use of higher frequency stimuli and lower number of EEG channels","author":"Saboor","year":"2018","journal-title":"Proceedings of the 2018 International Conference on Frontiers of Information Technology"},{"key":"B23","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1016\/j.neucom.2019.08.037","article-title":"Classification of multiclass motor imagery EEG signal using sparsity approach.","volume":"368","author":"Sreeja","year":"2019","journal-title":"Neurocomputing"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.3390\/brainsci7040035","article-title":"A novel hybrid mental spelling application based on eye tracking and SSVEP-based BCI.","volume":"7","author":"Stawicki","year":"2017","journal-title":"Brain Sci."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jneumeth.2021.109442","article-title":"An autonomous hybrid brain-computer interface system combined with eye-tracking in virtual environment.","volume":"368","author":"Tan","year":"2022","journal-title":"J. Neurosci. Methods"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1746","DOI":"10.1109\/TNSRE.2016.2627556","article-title":"A benchmark dataset for SSVEP-based brain\u2013computer interfaces.","volume":"25","author":"Wang","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabilitation Eng."},{"key":"B27","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1016\/s1388-2457(02)00057-3","article-title":"Brain\u2013computer interfaces for communication and control.","volume":"113","author":"Wolpaw","year":"2002","journal-title":"Clin. Neurophysiol."},{"key":"B28","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.robot.2019.02.014","article-title":"Shared control of a robotic arm using non-invasive brain\u2013computer interface and computer vision guidance.","volume":"115","author":"Xu","year":"2019","journal-title":"Rob. Auton. Syst."},{"key":"B29","doi-asserted-by":"publisher","first-page":"2859","DOI":"10.1587\/transinf.2018EDL8122","article-title":"High-speed spelling in virtual reality with sequential hybrid BCIs.","volume":"101","author":"Yao","year":"2018","journal-title":"IEICE Trans. Inf. Syst."},{"key":"B30","doi-asserted-by":"publisher","first-page":"345","DOI":"10.3233\/ICA-180586","article-title":"An online brain-computer interface in mobile virtual reality environments.","volume":"26","author":"Yao","year":"2019","journal-title":"Integr. Comput. Aided Eng."},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2019.00111","article-title":"Semi-Autonomous robotic arm reaching with hybrid gaze-brain machine interface.","volume":"13","author":"Zeng","year":"2020","journal-title":"Front. Neurorobot."},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2020.583641","article-title":"A hybrid BCI based on SSVEP and EOG for robotic arm control.","volume":"14","author":"Zhu","year":"2020","journal-title":"Front. Neurorobot."}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1146415\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T05:02:15Z","timestamp":1679893335000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1146415\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,27]]},"references-count":32,"alternative-id":["10.3389\/fnbot.2023.1146415"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1146415","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,27]]},"article-number":"1146415"}}