{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T07:49:36Z","timestamp":1791532176280,"version":"4.3.4"},"reference-count":44,"publisher":"IOP Publishing","issue":"5","license":[{"start":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T00:00:00Z","timestamp":1791504000000},"content-version":"vor","delay-in-days":8,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T00:00:00Z","timestamp":1791504000000},"content-version":"tdm","delay-in-days":8,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["J. Neural Eng."],"published-print":{"date-parts":[[2026,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    <jats:italic>Objective<\/jats:italic>\n                    . Machine learning classifiers used in Brain-Computer Interfaces (BCIs) make classifications based on the distribution of examples on which they were trained. When exposed to electroencephalography (EEG) from unfamiliar classes they can only make blind guesses. Instead of allowing such guesses, these out-of-distribution (OOD) samples should be detected and rejected to improve the robustness of BCIs against unfamiliar cognitive states.\n                    <jats:italic>Approach<\/jats:italic>\n                    . We study OOD detection in Motor Imagery BCIs by training a model on some classes and observing whether an unfamiliar movement class can be detected based on increased uncertainty. We tested seven different OOD detection methods and one more method that has been claimed to boost the quality of OOD detection.\n                    <jats:italic>Main results<\/jats:italic>\n                    . For many BCI users, the uncertainty for the familiar in-distribution classes can still be higher than for the OOD classes, due to the high intrinsic variability inherent in EEG signals. As a result, many OOD detection methods that have shown good performance in other machine learning domains prove to be ineffective at identifying unfamiliar motor imagery patterns in BCIs. However, we also found that OOD detection performance is correlated with on-task performance, and that Deep Ensemble models and Monte Carlo-Dropout models were able to achieve on-task Area Under the Receiver-Operator Characteristic curve\n                    <jats:inline-formula>\n                      <jats:tex-math>\n                        \n                      <\/jats:tex-math>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\">\n                        <mml:mrow>\n                          <mml:mo>&gt;<\/mml:mo>\n                        <\/mml:mrow>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    0.9, and OOD detection ability up to 0.7. This shows that for models and subjects where task performance is high, rejecting unfamiliar cognitive states becomes feasible.\n                    <jats:italic>Significance<\/jats:italic>\n                    . Our research demonstrates a Leave-One-Class-Out OOD detection setup as an experimental paradigm for studying whether models are robust against unfamiliar motor imagery actions that were not seen in the training data. This is the first benchmark in Motor Imagery BCI that evaluates this type of OOD data. Our results demonstrate how to improve the overall safety and reliability of BCIs by preventing erroneous actions.\n                  <\/jats:p>","DOI":"10.1088\/1741-2552\/aeac87","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T22:51:01Z","timestamp":1790376661000},"page":"056046","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["The challenge of out-of-distribution detection in motor imagery BCIs"],"prefix":"10.1088","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-0880-1059","authenticated-orcid":true,"given":"Merlijn","family":"Quincent Mulder","sequence":"first","affiliation":[{"name":"University of Groningen"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5793-9498","authenticated-orcid":false,"given":"Matias","family":"Valdenegro-Toro","sequence":"additional","affiliation":[{"name":"University of Groningen"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6766-3464","authenticated-orcid":true,"given":"Andreea","family":"Ioana Sburlea","sequence":"additional","affiliation":[{"name":"University of Groningen"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1497-8013","authenticated-orcid":true,"given":"Ivo Pascal","family":"de Jong","sequence":"additional","affiliation":[{"name":"University of Groningen"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2026,10,9]]},"reference":[{"key":"jneaeac87bib1","doi-asserted-by":"crossref","DOI":"10.1145\/3292500.3330701","type":"conference-proceedings","article-title":"Optuna: a next-generation hyperparameter optimization framework","author":"Akiba","year":"2019"},{"key":"jneaeac87bib2","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.neuropsychologia.2015.09.012","type":"journal-article","article-title":"Motor imagery based brain\u2013computer interfaces: an emerging technology to rehabilitate motor deficits","volume":"79","author":"Alonso-Valerdi","year":"2015","journal-title":"Neuropsychologia"},{"key":"jneaeac87bib3","article-title":"Mother of all BCI benchmarks","author":"Aristimunha","year":"2023","type":"other"},{"key":"jneaeac87bib4","article-title":"Ood detection for EEG-based machine learning in high-risk environments","author":"Bomatter","year":"2026","type":"preprint"},{"key":"jneaeac87bib5","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1016\/j.brainres.2008.05.089","type":"journal-article","article-title":"Equivalent is not equal: primary motor cortex (mi) activation during motor imagery and execution of sequential movements","volume":"1226","author":"Carrillo-de-la Pe\u00f1a","year":"2008","journal-title":"Brain Res."},{"key":"jneaeac87bib6","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2023.105475","type":"journal-article","article-title":"Toward reliable signals decoding for electroencephalogram: a benchmark study to EEGNeX","volume":"87","author":"Chen","year":"2024","journal-title":"Biomed. Signal Process. Control"},{"key":"jneaeac87bib7","doi-asserted-by":"publisher","first-page":"429","DOI":"10.3390\/a16090429","type":"journal-article","article-title":"Bayesian opportunities for brain\u2013computer interfaces: enhancement of the existing classification algorithms and out-of-domain detection","volume":"16","author":"Chetkin","year":"2023","journal-title":"Algorithms"},{"key":"jneaeac87bib8","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.03.071","type":"journal-article","article-title":"Uncer: a framework for uncertainty estimation and reduction in neural decoding of eeg signals","volume":"538","author":"Duan","year":"2023","journal-title":"Neurocomputing"},{"key":"jneaeac87bib9","first-page":"1050","type":"conference-proceedings","article-title":"Dropout as a bayesian approximation: representing model uncertainty in deep learning","volume":"vol 48","author":"Gal","year":"2016"},{"key":"jneaeac87bib10","article-title":"A comprehensive review of trends, applications and challenges in out-of-distribution detection","author":"Ghassemi","year":"2022","type":"preprint"},{"key":"jneaeac87bib11","doi-asserted-by":"publisher","DOI":"10.1007\/s41666-026-00249-5","type":"journal-article","article-title":"Uncertainty quantification in machine learning for biosignal applications \u2013 a review","author":"de Jong","year":"2026","journal-title":"J. Healthc. Inform. Res."},{"key":"jneaeac87bib12","article-title":"How disentangled are your classification uncertainties?","author":"Jong","year":"2024","type":"preprint"},{"key":"jneaeac87bib13","first-page":"1","type":"conference-proceedings","article-title":"Confidence-aware neural decoding of overt speech from EEG: toward robust brain\u2013computer interfaces","author":"Kim","year":"2026"},{"key":"jneaeac87bib14","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2017","type":"preprint"},{"key":"jneaeac87bib15","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.brainres.2014.09.001","type":"journal-article","article-title":"Motor imagery-based brain activity parallels that of motor execution: evidence from magnetic source imaging of cortical oscillations","volume":"1588","author":"Kraeutner","year":"2014","journal-title":"Brain Res."},{"key":"jneaeac87bib16","type":"conference-proceedings","article-title":"Simple and scalable predictive uncertainty estimation using deep ensembles","volume":"vol 30","author":"Lakshminarayanan","year":"2017"},{"key":"jneaeac87bib17","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7443.003.0014","type":"book","article-title":"Energy-based models","author":"LeCun","year":"2007"},{"key":"jneaeac87bib18","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-026-44228-7","type":"other","article-title":"Implanted brain-computer interface functionality during nighttime in late-stage amyotrophic lateral sclerosis","author":"Leinders","year":"2026"},{"key":"jneaeac87bib19","doi-asserted-by":"publisher","first-page":"24564","DOI":"10.52202\/075280-1068","type":"conference-proceedings","article-title":"Prototype-based aleatoric uncertainty quantification for cross-modal retrieval","volume":"vol 36","author":"Li","year":"2023"},{"key":"jneaeac87bib20","article-title":"Temporal out-of-distribution detection for asynchronous motor imagery brain-computer interfaces","author":"Liu","year":"2026","type":"preprint"},{"key":"jneaeac87bib21","first-page":"21464","type":"conference-proceedings","article-title":"Energy-based Out-of-distribution Detection","volume":"vol 33","author":"Liu","year":"2020"},{"key":"jneaeac87bib22","article-title":"An orderly approach to EEG analysis: visual inspection of the background and pattern recognition","author":"Louis","year":"2016","type":"book"},{"key":"jneaeac87bib23","article-title":"Uncertainty quantification for cross-subject motor imagery classification","author":"Manivannan","year":"2024","type":"preprint"},{"key":"jneaeac87bib24","doi-asserted-by":"publisher","first-page":"7241","DOI":"10.3390\/s21217241","type":"journal-article","article-title":"Monte carlo dropout for uncertainty estimation and motor imagery classification","volume":"21","author":"Milan\u00e9s-Hermosilla","year":"2021","journal-title":"Sensors"},{"key":"jneaeac87bib25","doi-asserted-by":"publisher","first-page":"703","DOI":"10.3390\/s23020703","type":"journal-article","article-title":"Robust motor imagery tasks classification approach using Bayesian neural network","volume":"23","author":"Milan\u00e9s-Hermosilla","year":"2023","journal-title":"Sensors"},{"key":"jneaeac87bib26","first-page":"50972","type":"conference-proceedings","article-title":"Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks","volume":"vol 37","author":"Mucs\u00e1nyi","year":"2024"},{"key":"jneaeac87bib27","doi-asserted-by":"publisher","first-page":"24384","DOI":"10.1109\/CVPR52729.2023.02336","type":"conference-proceedings","article-title":"Deep deterministic uncertainty: a new simple baseline","author":"Mukhoti","year":"2023"},{"key":"jneaeac87bib28","first-page":"427","type":"conference-proceedings","article-title":"Deep neural networks are easily fooled: high confidence predictions for unrecognizable images","author":"Nguyen","year":"2015"},{"key":"jneaeac87bib29","doi-asserted-by":"publisher","first-page":"5391","DOI":"10.1002\/hbm.23730","type":"journal-article","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"jneaeac87bib30","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1038\/s41597-021-00883-1","type":"journal-article","article-title":"Continuous sensorimotor rhythm based brain computer interface learning in a large population","volume":"8","author":"Stieger","year":"2021","journal-title":"Sci. Data"},{"key":"jneaeac87bib31","first-page":"144","type":"conference-proceedings","article-title":"React: Out-of-distribution detection with rectified activations","volume":"vol 34","author":"Sun","year":"2021"},{"key":"jneaeac87bib32","first-page":"20827","type":"conference-proceedings","article-title":"Out-of-distribution detection with deep nearest neighbors","volume":"vol 162","author":"Sun","year":"2022"},{"key":"jneaeac87bib33","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1080\/2326263X.2023.2233368","type":"journal-article","article-title":"A systematic review on artifact removal and classification techniques for enhanced MEG-based BCI systems","volume":"10","author":"Susan Philip","year":"2023","journal-title":"Brain-Comput. Interfaces"},{"key":"jneaeac87bib34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/MLSP62443.2025.11204214","type":"conference-proceedings","article-title":"Uncertainty quantification for motor imagery BCI \u2013 machine learning vs. deep learning","author":"Suurmeijer","year":"2025"},{"key":"jneaeac87bib35","doi-asserted-by":"publisher","first-page":"55","DOI":"10.3389\/fnins.2012.00055","type":"journal-article","article-title":"Review of the BCI competition IV","volume":"6","author":"Tangermann","year":"2012","journal-title":"Front. Neurosci."},{"key":"jneaeac87bib36","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2026.103374","type":"journal-article","article-title":"Uncertainty in deep learning for EEG under dataset shifts","volume":"174","author":"Tveter","year":"2026","journal-title":"Artif. Intell. Med."},{"key":"jneaeac87bib37","first-page":"9690","type":"conference-proceedings","article-title":"Uncertainty estimation using a single deep deterministic neural network","volume":"vol 119","author":"Van Amersfoort","year":"2020"},{"key":"jneaeac87bib38","doi-asserted-by":"publisher","first-page":"21142","DOI":"10.52202\/068431-1537","type":"conference-proceedings","article-title":"Evaluating latent space robustness and uncertainty of EEG-ML models under realistic distribution shifts","volume":"vol 35","author":"Wagh","year":"2022"},{"key":"jneaeac87bib39","article-title":"From aleatoric to epistemic: exploring uncertainty quantification techniques in artificial intelligence","author":"Wang","year":"2025","type":"preprint"},{"key":"jneaeac87bib40","doi-asserted-by":"publisher","first-page":"1466","DOI":"10.1038\/s41551-025-01536-z","type":"journal-article","article-title":"Long-term unsupervised recalibration of cursor-based intracortical brain\u2013computer interfaces using a hidden markov model","volume":"10","author":"Wilson","year":"2026","journal-title":"Nat. Biomed. Eng."},{"key":"jneaeac87bib41","doi-asserted-by":"publisher","first-page":"8375","DOI":"10.3390\/s23208375","type":"journal-article","article-title":"Estimating patient-level uncertainty in seizure detection using group-specific out-of-distribution detection technique","volume":"23","author":"Wong","year":"2023","journal-title":"Sensors"},{"key":"jneaeac87bib42","doi-asserted-by":"publisher","first-page":"5635","DOI":"10.1007\/s11263-024-02117-4","type":"journal-article","article-title":"Generalized out-of-distribution detection: a survey","volume":"132","author":"Yang","year":"2024","journal-title":"Int. J. Comput. Vis."},{"key":"jneaeac87bib43","article-title":"Beatrix: out-of-distribution generalization of large EEG model via invariant contrastive fine-tuning","author":"Zheng","year":"2025","type":"other"},{"key":"jneaeac87bib44","type":"journal-article","article-title":"OOD-BCI","author":"Mulder","year":"2026","journal-title":"Github"}],"container-title":["Journal of Neural Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T07:02:44Z","timestamp":1791529364000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1741-2552\/aeac87"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,1]]},"references-count":44,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,10,9]]},"published-print":{"date-parts":[[2026,10,1]]}},"URL":"https:\/\/doi.org\/10.1088\/1741-2552\/aeac87","relation":{},"ISSN":["1741-2560","1741-2552"],"issn-type":[{"value":"1741-2560","type":"print"},{"value":"1741-2552","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,1]]},"assertion":[{"value":"The challenge of out-of-distribution detection in motor imagery BCIs","name":"article_title","label":"Article Title"},{"value":"Journal of Neural Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2026 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2026-02-26","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-09-25","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-10-09","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}