{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T22:26:15Z","timestamp":1766787975074,"version":"3.41.2"},"reference-count":54,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,9]],"date-time":"2024-05-09T00:00:00Z","timestamp":1715212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJMS2012"],"award-info":[{"award-number":["JPMJMS2012"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Shared autonomy holds promise for assistive robotics, whereby physically-impaired people can direct robots to perform various tasks for them. However, a robot that is capable of many tasks also introduces many choices for the user, such as which object or location should be the target of interaction. In the context of non-invasive brain-computer interfaces for shared autonomy\u2014most commonly electroencephalography-based\u2014the two most common choices are to provide either auditory or visual stimuli to the user\u2014each with their respective pros and cons. Using the oddball paradigm, we designed comparable auditory and visual interfaces to speak\/display the choices to the user, and had users complete a multi-stage robotic manipulation task involving location and object selection. Users displayed differing competencies\u2014and preferences\u2014for the different interfaces, highlighting the importance of considering modalities outside of vision when constructing human-robot interfaces.<\/jats:p>","DOI":"10.3389\/frobt.2024.1329270","type":"journal-article","created":{"date-parts":[[2024,5,9]],"date-time":"2024-05-09T04:37:04Z","timestamp":1715229424000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A comparison of visual and auditory EEG interfaces for robot multi-stage task control"],"prefix":"10.3389","volume":"11","author":[{"given":"Kai","family":"Arulkumaran","sequence":"first","affiliation":[]},{"given":"Marina","family":"Di Vincenzo","sequence":"additional","affiliation":[]},{"given":"Rousslan Fernand Julien","family":"Dossa","sequence":"additional","affiliation":[]},{"given":"Shogo","family":"Akiyama","sequence":"additional","affiliation":[]},{"given":"Dan","family":"Ogawa Lillrank","sequence":"additional","affiliation":[]},{"given":"Motoshige","family":"Sato","sequence":"additional","affiliation":[]},{"given":"Kenichi","family":"Tomeoka","sequence":"additional","affiliation":[]},{"given":"Shuntaro","family":"Sasai","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,5,9]]},"reference":[{"key":"B1","unstructured":"Do as i can, not as i say: grounding language in robotic affordances\n            AhnM.\n            BrohanA.\n            BrownN.\n            ChebotarY.\n            CortesO.\n            DavidB.\n          2022"},{"key":"B2","first-page":"219","article-title":"Task level hierarchical system for bci-enabled shared autonomy","author":"Akinola","year":"2017"},{"article-title":"Fine-grained object detection and manipulation with segmentation-conditioned perceiver-actor","year":"2023","author":"Akiyama","key":"B3"},{"key":"B4","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1007\/s11370-020-00328-5","article-title":"Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals","volume":"13","author":"Aljalal","year":"2020","journal-title":"Intell. Serv. Robot."},{"key":"B5","first-page":"5723","article-title":"Speecht5: unified-modal encoder-decoder pre-training for spoken language processing","author":"Ao","year":"2022"},{"key":"B6","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.biopsycho.2005.05.002","article-title":"Electrophysiological correlates of categorization: P300 amplitude as index of target similarity","volume":"71","author":"Azizian","year":"2006","journal-title":"Biol. Psychol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"025022","DOI":"10.1088\/1741-2560\/8\/2\/025022","article-title":"P300 audio-visual speller","volume":"8","author":"Belitski","year":"2011","journal-title":"J. neural Eng."},{"key":"B8","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1109\/tsmcc.2012.2219046","article-title":"Eeg-based brain-controlled mobile robots: a survey","volume":"43","author":"Bi","year":"2013","journal-title":"IEEE Trans. human-machine Syst."},{"key":"B9","unstructured":"Rt-1: robotics transformer for real-world control at scale\n            BrohanA.\n            BrownN.\n            CarbajalJ.\n            ChebotarY.\n            DabisJ.\n            FinnC.\n          2022"},{"key":"B10","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1097\/phm.0b013e3181cf569b","article-title":"The role of assistive robotics in the lives of persons with disability","volume":"89","author":"Brose","year":"2010","journal-title":"Am. J. Phys. Med. Rehabilitation"},{"key":"B11","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/mra.2011.2181749","article-title":"MoveIt! [ROS topics]","volume":"19","author":"Chitta","year":"2012","journal-title":"IEEE Robotics Automation Mag."},{"key":"B12","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1111\/j.1469-8986.1981.tb01815.x","article-title":"Surprise!\u2026surprise?","volume":"18","author":"Donchin","year":"1981","journal-title":"Psychophysiology"},{"key":"B13","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1016\/s0079-6123(08)61692-1","article-title":"Event-related potentials and psychological theory","volume":"54","author":"Donchin","year":"1980","journal-title":"Prog. Brain Res."},{"key":"B14","unstructured":"Palm-e: an embodied multimodal language model\n            DriessD.\n            XiaF.\n            SajjadiM. S.\n            LynchC.\n            ChowdheryA.\n            IchterB.\n          2023"},{"key":"B15","doi-asserted-by":"publisher","first-page":"510","DOI":"10.1016\/0013-4694(88)90149-6","article-title":"Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials","volume":"70","author":"Farwell","year":"1988","journal-title":"Electroencephalogr. Clin. Neurophysiology"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.18502\/fbt.v7i4.5322","article-title":"The comparison between visually and auditory oddball tasks in the eeg experiment with healthy subjects","author":"Fayaz","year":"2020","journal-title":"Front. Biomed. Technol."},{"key":"B17","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511815867","volume-title":"Statistical models: theory and practice","author":"Freedman","year":"2009"},{"key":"B18","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1111\/j.1469-8986.2008.00783.x","article-title":"An auditory oddball (p300) spelling system for brain-computer interfaces","volume":"46","author":"Furdea","year":"2009","journal-title":"Psychophysiology"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1436","DOI":"10.1109\/tbme.2014.2300164","article-title":"Visual and auditory brain\u2013computer interfaces","volume":"61","author":"Gao","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B20","first-page":"1","article-title":"Workload: an examination of the concept","volume-title":"Handbook of perception and human performance","author":"Gopher","year":"1986"},{"key":"B21","unstructured":"Rvt: robotic view transformer for 3d object manipulation\n            GoyalA.\n            XuJ.\n            GuoY.\n            BlukisV.\n            ChaoY.-W.\n            FoxD.\n          2023"},{"key":"B22","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1038\/nature11076","article-title":"Reach and grasp by people with tetraplegia using a neurally controlled robotic arm","volume":"485","author":"Hochberg","year":"2012","journal-title":"Nature"},{"key":"B23","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/0301-0511(95)05109-0","article-title":"Effects of attention and time-pressure on p300 subcomponents and implications for mental workload research","volume":"40","author":"Hohnsbein","year":"1995","journal-title":"Biol. Psychol."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.physrep.2021.03.002","article-title":"Physical principles of brain\u2013computer interfaces and their applications for rehabilitation, robotics and control of human brain states","volume":"918","author":"Hramov","year":"2021","journal-title":"Phys. Rep."},{"key":"B25","article-title":"Perceiver IO: a general architecture for structured inputs & outputs","author":"Jaegle","year":"2022","journal-title":"ICLR"},{"key":"B26","unstructured":"Coarse-to-fine q-attention with learned path ranking\n            JamesS.\n            AbbeelP.\n          2022"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1612","DOI":"10.1109\/lra.2022.3140817","article-title":"Q-attention: enabling efficient learning for vision-based robotic manipulation","volume":"7","author":"James","year":"2022","journal-title":"IEEE Robotics Automation Lett."},{"key":"B28","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1016\/j.clinph.2012.08.006","article-title":"A portable auditory p300 brain\u2013computer interface with directional cues","volume":"124","author":"K\u00e4thner","year":"2013","journal-title":"Clin. Neurophysiol."},{"key":"B29","doi-asserted-by":"publisher","first-page":"1252","DOI":"10.1016\/j.clinph.2009.04.019","article-title":"Toward a high-throughput auditory p300-based brain\u2013computer interface","volume":"120","author":"Klobassa","year":"2009","journal-title":"Clin. Neurophysiol."},{"key":"B30","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1111\/j.1469-8986.1986.tb00590.x","article-title":"The effects of practice and task structure on components of the event-related brain potential","volume":"23","author":"Kramer","year":"1986","journal-title":"Psychophysiology"},{"key":"B31","doi-asserted-by":"publisher","first-page":"012017","DOI":"10.1088\/1757-899x\/121\/1\/012017","article-title":"Electroencephalography (eeg) based control in assistive mobile robots: a review","volume":"121","author":"Krishnan","year":"2016","journal-title":"IOP Conf. Ser. Mater. Sci. Eng. IOP Publ."},{"key":"B32","doi-asserted-by":"publisher","first-page":"995","DOI":"10.1109\/ROBOT.2000.844730","article-title":"Rrt-connect: an efficient approach to single-query path planning","volume":"2","author":"Kuffner","year":"2000","journal-title":"ICRA"},{"key":"B33","doi-asserted-by":"publisher","first-page":"792","DOI":"10.1126\/science.887923","article-title":"Augmenting mental chronometry: the p300 as a measure of stimulus evaluation time","volume":"197","author":"Kutas","year":"1977","journal-title":"Science"},{"key":"B34","doi-asserted-by":"publisher","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"Eegnet: a compact convolutional neural network for eeg-based brain\u2013computer interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. neural Eng."},{"key":"B35","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"nature"},{"key":"B36","article-title":"Noir: neural signal operated intelligent robot for everyday activities","volume-title":"CoRL","author":"Lee","year":"2023"},{"key":"B37","unstructured":"Grounding dino: marrying dino with grounded pre-training for open-set object detection\n            LiuS.\n            ZengZ.\n            RenT.\n            LiF.\n            ZhangH.\n            YangJ.\n          2023"},{"key":"B38","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1080\/00140130010011369","article-title":"Comparative study of the effects of auditory, visual and multimodality displays on drivers\u2019 performance in advanced traveller information systems","volume":"44","author":"Liu","year":"2001","journal-title":"Ergonomics"},{"key":"B39","first-page":"523","article-title":"Event-related potentials","volume-title":"APA handbook of research methods in psychology","author":"Luck","year":"2012"},{"key":"B40","doi-asserted-by":"publisher","first-page":"46","DOI":"10.3390\/jpm13010046","article-title":"Personalized brain\u2013computer interface and its applications","volume":"13","author":"Ma","year":"2022","journal-title":"J. Personalized Med."},{"key":"B41","doi-asserted-by":"publisher","first-page":"38565","DOI":"10.1038\/srep38565","article-title":"Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks","volume":"6","author":"Meng","year":"2016","journal-title":"Sci. Rep."},{"key":"B42","doi-asserted-by":"publisher","first-page":"1211","DOI":"10.3390\/s120201211","article-title":"Brain computer interfaces, a review","volume":"12","author":"Nicolas-Alonso","year":"2012","journal-title":"sensors"},{"volume-title":"The design of everyday things","year":"2013","author":"Norman","key":"B43"},{"key":"B44","doi-asserted-by":"publisher","first-page":"106618","DOI":"10.1109\/access.2019.2933049","article-title":"A new paradigm for region-based p300 speller in brain computer interface","volume":"7","author":"Oralhan","year":"2019","journal-title":"Ieee Access"},{"key":"B45","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"JMLR"},{"key":"B46","doi-asserted-by":"publisher","first-page":"2128","DOI":"10.1016\/j.clinph.2007.04.019","article-title":"Updating p300: an integrative theory of p3a and p3b","volume":"118","author":"Polich","year":"2007","journal-title":"Clin. Neurophysiol."},{"key":"B47","first-page":"8748","article-title":"Learning transferable visual models from natural language supervision","author":"Radford","year":"2021"},{"key":"B48","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1017\/s0048577202010223","article-title":"The effect of background noise on p300 to suprathreshold stimuli","volume":"39","author":"Salisbury","year":"2002","journal-title":"Psychophysiology"},{"key":"B49","first-page":"894","article-title":"Cliport: what and where pathways for robotic manipulation","author":"Shridhar","year":""},{"key":"B50","first-page":"785","article-title":"Perceiver-actor: a multi-task transformer for robotic manipulation","volume-title":"CoRL","author":"Shridhar","year":""},{"key":"B51","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/0013-4694(75)90263-1","article-title":"Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man","volume":"38","author":"Squires","year":"1975","journal-title":"Electroencephalogr. Clin. neurophysiology"},{"key":"B52","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.neucom.2020.09.017","article-title":"A review on transfer learning in eeg signal analysis","volume":"421","author":"Wan","year":"2021","journal-title":"Neurocomputing"},{"key":"B53","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1016\/s0278-2626(01)80090-7","article-title":"Task complexity and habituation effects on frontal p300 topography","volume":"46","author":"Wintink","year":"2001","journal-title":"Brain Cognition"},{"key":"B54","doi-asserted-by":"publisher","first-page":"583641","DOI":"10.3389\/fnbot.2020.583641","article-title":"A hybrid bci based on ssvep and eog for robotic arm control","volume":"14","author":"Zhu","year":"2020","journal-title":"Front. neurorobotics"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1329270\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,9]],"date-time":"2024-05-09T04:37:12Z","timestamp":1715229432000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1329270\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,9]]},"references-count":54,"alternative-id":["10.3389\/frobt.2024.1329270"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1329270","relation":{},"ISSN":["2296-9144"],"issn-type":[{"type":"electronic","value":"2296-9144"}],"subject":[],"published":{"date-parts":[[2024,5,9]]},"article-number":"1329270"}}