{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T11:46:56Z","timestamp":1793360816765,"version":"4.5.0"},"reference-count":51,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2021,10,30]],"date-time":"2021-10-30T00:00:00Z","timestamp":1635552000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Motor Imagery (MI)-based Brain\u2013Computer Interfaces (BCIs) have been widely used as an alternative communication channel to patients with severe motor disabilities, achieving high classification accuracy through machine learning techniques. Recently, deep learning techniques have spotlighted the state-of-the-art of MI-based BCIs. These techniques still lack strategies to quantify predictive uncertainty and may produce overconfident predictions. In this work, methods to enhance the performance of existing MI-based BCIs are proposed in order to obtain a more reliable system for real application scenarios. First, the Monte Carlo dropout (MCD) method is proposed on MI deep neural models to improve classification and provide uncertainty estimation. This approach was implemented using Shallow Convolutional Neural Network (SCNN-MCD) and with an ensemble model (E-SCNN-MCD). As another contribution, to discriminate MI task predictions of high uncertainty, a threshold approach is introduced and tested for both SCNN-MCD and E-SCNN-MCD approaches. The BCI Competition IV Databases 2a and 2b were used to evaluate the proposed methods for both subject-specific and non-subject-specific strategies, obtaining encouraging results for MI recognition.<\/jats:p>","DOI":"10.3390\/s21217241","type":"journal-article","created":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T22:24:22Z","timestamp":1635805462000},"page":"7241","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":104,"title":["Monte Carlo Dropout for Uncertainty Estimation and Motor Imagery Classification"],"prefix":"10.3390","volume":"21","author":[{"given":"Daily","family":"Milan\u00e9s-Hermosilla","sequence":"first","affiliation":[{"name":"Department of Automatic Engineering, Universidad de Oriente, Santiago de Cuba 90500, Cuba"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7449-1939","authenticated-orcid":false,"given":"Rafael","family":"Trujillo Codorni\u00fa","sequence":"additional","affiliation":[{"name":"Serconi, Holgu\u00edn 80100, Cuba"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ren\u00e9","family":"L\u00f3pez-Baracaldo","sequence":"additional","affiliation":[{"name":"Zimtronic, Miami, FL 33222, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5808-1999","authenticated-orcid":false,"given":"Roberto","family":"Sagar\u00f3-Zamora","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Universidad de Oriente, Santiago de Cuba 90500, Cuba"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8937-031X","authenticated-orcid":false,"given":"Denis","family":"Delisle-Rodriguez","sequence":"additional","affiliation":[{"name":"Postgraduate Program in Electrical Engineering, Federal University of Espirito Santo, Vitoria 29075-910, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2325-1518","authenticated-orcid":false,"given":"John Jairo","family":"Villarejo-Mayor","sequence":"additional","affiliation":[{"name":"Department of Physical Education, Federal University of Paran\u00e1, Curitiba 80050-520, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6607-7305","authenticated-orcid":false,"given":"Jos\u00e9 Ricardo","family":"N\u00fa\u00f1ez-\u00c1lvarez","sequence":"additional","affiliation":[{"name":"Department of Energy, Universidad de la Costa, Barranquilla 080002, Colombia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,30]]},"reference":[{"key":"ref_1","first-page":"567","article-title":"Towards a new modality-independent interface for a robotic wheelchair","volume":"22","author":"Cheein","year":"2013","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/s42256-018-0004-1","article-title":"The need for uncertainty quantification in machine-assisted medical decision making","volume":"1","author":"Begoli","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"36218","DOI":"10.1109\/ACCESS.2018.2836917","article-title":"Neural network-based uncertainty quantification: A survey of methodologies and applications","volume":"6","author":"Kabir","year":"2018","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Norouzi, A., Emami, A., Najarian, K., Karimi, N., and Soroushmehr, S.R. (2019, January 12\u201317). Exploiting uncertainty of deep neural networks for improving segmentation accuracy in MRI images. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682530"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neuroimage.2019.03.042","article-title":"Bayesian QuickNAT: Model uncertainty in deep whole-brain segmentation for structure-wise quality control","volume":"195","author":"Roy","year":"2019","journal-title":"NeuroImage"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1111\/coin.12411","article-title":"Estimating uncertainty in deep learning for reporting confidence to clinicians in medical image segmentation and diseases detection","volume":"37","author":"Ghoshal","year":"2021","journal-title":"Comput. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Michelmore, R., Wicker, M., Laurenti, L., Cardelli, L., Gal, Y., and Kwiatkowska, M. (August, January 31). Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196844"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Feng, D., Rosenbaum, L., and Dietmayer, K. (2018, January 4\u20137). Towards safe autonomous driving: Capture uncertainty in the deep neural network for lidar 3d vehicle detection. Proceedings of the 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569814"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1177\/0278364918770733","article-title":"The limits and potentials of deep learning for robotics","volume":"37","author":"Brock","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_10","unstructured":"Malinin, A. (2019). Uncertainty Estimation in Deep Learning with Application to Spoken Language Assessment. [Ph.D. Thesis, University of Cambridge]."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ghoshal, B., and Tucker, A. (2021, January 7\u20139). On Cost-Sensitive Calibrated Uncertainty in Deep Learning: An application on COVID-19 detection. Proceedings of the 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS), Aveiro, Portugal.","DOI":"10.1109\/CBMS52027.2021.00100"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"22812","DOI":"10.1109\/ACCESS.2020.2970023","article-title":"Uncertainty assisted robust tuberculosis identification with bayesian convolutional neural networks","volume":"8","author":"Abideen","year":"2020","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Stoean, C., Stoean, R., Atencia, M., Abdar, M., Vel\u00e1zquez-P\u00e9rez, L., Khosravi, A., Nahavandi, S., Acharya, U.R., and Joya, G. (2020). Automated detection of presymptomatic conditions in Spinocerebellar Ataxia type 2 using Monte Carlo dropout and deep neural network techniques with electrooculogram signals. Sensors, 20.","DOI":"10.3390\/s20113032"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jungo, A., Meier, R., Ermis, E., Blatti-Moreno, M., Herrmann, E., Wiest, R., and Reyes, M. (2018). On the effect of inter-observer variability for a reliable estimation of uncertainty of medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-030-00928-1_77"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"101715","DOI":"10.1016\/j.media.2020.101715","article-title":"DR|GRADUATE: Uncertainty-aware deep learning-based diabetic retinopathy grading in eye fundus images","volume":"63","author":"Aresta","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-017-17876-z","article-title":"Leveraging uncertainty information from deep neural networks for disease detection","volume":"7","author":"Leibig","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gill, R.S., Caldairou, B., Bernasconi, N., and Bernasconi, A. (2019). Uncertainty-Informed Detection of Epileptogenic Brain Malformations Using Bayesian Neural Networks, Springer International Publishing.","DOI":"10.1007\/978-3-030-32251-9_25"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1080\/026990596124566","article-title":"Attention deficits and dual task demands after mild traumatic brain injury","volume":"10","author":"Cicerone","year":"1996","journal-title":"Brain Inj."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/bs.pbr.2016.04.002","article-title":"Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates","volume":"228","author":"Jeunet","year":"2016","journal-title":"Prog. Brain Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1152\/jn.00918.2015","article-title":"Efficient neuroplasticity induction in chronic stroke patients by an associative brain-computer interface","volume":"115","author":"Jiang","year":"2016","journal-title":"J. Neurophysiol."},{"key":"ref_21","unstructured":"Neal, R.M. (1993). Bayesian learning via stochastic dynamics. Advances in Neural Information Processing Systems, Morgan Kaufmann."},{"key":"ref_22","unstructured":"Gal, Y. (2016). Uncertainty in Deep Learning. [Ph.D. Thesis, University of Cambridge]."},{"key":"ref_23","unstructured":"Gal, Y., and Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. International Conference on Machine Learning, PMLR."},{"key":"ref_24","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.artint.2014.02.004","article-title":"The dropout learning algorithm","volume":"210","author":"Baldi","year":"2014","journal-title":"Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ma, L., and Kaewell, J. (2020, January 6\u201311). Fast Monte Carlo Dropout and Error Correction for Radio Transmitter Classification. Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS), New York, NY, USA.","DOI":"10.1109\/WIFS49906.2020.9360887"},{"key":"ref_27","unstructured":"Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017, January 4\u20139). Simple and scalable predictive uncertainty estimation using deep ensembles. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Haas, J., and Rabus, B. (2021). Uncertainty Estimation for Deep Learning-Based Segmentation of Roads in Synthetic Aperture Radar Imagery. Remote Sens., 13.","DOI":"10.3390\/rs13081472"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"98275","DOI":"10.1109\/ACCESS.2021.3091399","article-title":"Shallow Convolutional Network Excel for Classifying Motor Imagery EEG in BCI applications","volume":"9","author":"Hermosilla","year":"2021","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"39","DOI":"10.3389\/fnins.2012.00039","article-title":"Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b","volume":"6","author":"Ang","year":"2012","journal-title":"Front. Neurosci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sakhavi, S., and Guan, C. (2017, January 25\u201328). Convolutional neural network-based transfer learning and knowledge distillation using multi-subject data in motor imagery BCI. Proceedings of the 8th International IEEE\/EMBS Conference on Neural Engineering (NER), Shanghai, China.","DOI":"10.1109\/NER.2017.8008420"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/2326263X.2020.1801112","article-title":"Effects of local and global spatial patterns in EEG motor-imagery classification using convolutional neural network","volume":"7","author":"Liao","year":"2020","journal-title":"Brain-Comput. Interfaces"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1016\/j.future.2019.06.027","article-title":"Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion","volume":"101","author":"Amin","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"016003","DOI":"10.1088\/1741-2560\/14\/1\/016003","article-title":"A novel deep learning approach for classification of EEG motor imagery signals","volume":"14","author":"Tabar","year":"2016","journal-title":"J. Neural Eng."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Li, F., He, F., Wang, F., Zhang, D., Xia, Y., and Li, X. (2020). A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl. Sci., 10.","DOI":"10.3390\/app10051605"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"918","DOI":"10.3389\/fnins.2020.00918","article-title":"Deep learning based inter-subject continuous decoding of motor imagery for practical brain-computer interfaces","volume":"14","author":"Roy","year":"2020","journal-title":"Front. Neurosci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/0013-4694(92)90133-3","article-title":"Event-related synchronization (ERS): An electrophysiological correlate of cortical areas at rest","volume":"83","author":"Pfurtscheller","year":"1992","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/0013-4694(79)90063-4","article-title":"Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement","volume":"46","author":"Pfurtscheller","year":"1979","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.neuroimage.2005.12.003","article-title":"Mu rhythm (de) synchronization and EEG single-trial classification of different motor imagery tasks","volume":"31","author":"Pfurtscheller","year":"2006","journal-title":"NeuroImage"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1842","DOI":"10.1016\/S1388-2457(99)00141-8","article-title":"Event-related EEG\/MEG synchronization and desynchronization: Basic principles","volume":"110","author":"Pfurtscheller","year":"1999","journal-title":"Clin. Neurophysiol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/0304-3940(94)90127-9","article-title":"Event-related synchronization of mu rhythm in the EEG over the cortical hand area in man","volume":"174","author":"Pfurtscheller","year":"1994","journal-title":"Neurosci. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/S0924-980X(97)00070-2","article-title":"Event-related beta synchronization after wrist, finger and thumb movement","volume":"109","author":"Pfurtscheller","year":"1998","journal-title":"Electroencephalogr. Clin. Neurophysiol. Electromyogr. Mot. Control"},{"key":"ref_44","unstructured":"Ang, K.K., Chin, Z.Y., Zhang, H., and Guan, C. (2008, January 1\u20138). Filter bank common spatial pattern (FBCSP) in brain-computer interface. Proceedings of the IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China."},{"key":"ref_45","unstructured":"Bergstra, J., Desjardins, G., Lamblin, P., and Bengio, Y. (2009). Quadratic Polynomials Learn Better Image Features, D\u00e9partement d\u2019Informatique et de Recherche Op\u00e9rationnelle, Universit\u00e9 de Montr\u00e9al. Technical Report 1337."},{"key":"ref_46","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_48","first-page":"5574","article-title":"What uncertainties do we need in bayesian deep learning for computer vision?","volume":"30","author":"Kendall","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_49","unstructured":"Duerr, O., Sick, B., and Murina, E. (2020). Probabilistic Deep Learning: With Python, Keras and TensorFlow Probability, Manning Publications."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-021-84854-x","article-title":"Dropconnect is effective in modeling uncertainty of bayesian deep networks","volume":"11","author":"Mobiny","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_51","unstructured":"Srivastava, N. (2013). Improving Neural Networks with Dropout. [Master\u2019s Thesis, University of Toronto]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7241\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:23:40Z","timestamp":1760167420000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7241"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,30]]},"references-count":51,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["s21217241"],"URL":"https:\/\/doi.org\/10.3390\/s21217241","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,30]]}}}