{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T14:36:13Z","timestamp":1777127773245,"version":"3.51.4"},"reference-count":41,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,1,8]],"date-time":"2019-01-08T00:00:00Z","timestamp":1546905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001655","name":"Deutscher Akademischer Austauschdienst","doi-asserted-by":"publisher","award":["91645515"],"award-info":[{"award-number":["91645515"]}],"id":[{"id":"10.13039\/501100001655","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["NO grant number"],"award-info":[{"award-number":["NO grant number"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005713","name":"Technische Universit\u00e4t M\u00fcnchen","doi-asserted-by":"publisher","award":["funding programme Open Access Publishing"],"award-info":[{"award-number":["funding programme Open Access Publishing"]}],"id":[{"id":"10.13039\/501100005713","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject\u2019s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from \u201cBCI Competition IV\u201d. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.<\/jats:p>","DOI":"10.3390\/s19010210","type":"journal-article","created":{"date-parts":[[2019,1,9]],"date-time":"2019-01-09T03:06:06Z","timestamp":1547003166000},"page":"210","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":180,"title":["Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3257-0211","authenticated-orcid":false,"given":"Zied","family":"Tayeb","sequence":"first","affiliation":[{"name":"Institute for Cognitive Systems, Technical University of Munich, 80333 Munich, Germany"},{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Juri","family":"Fedjaev","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Nejla","family":"Ghaboosi","sequence":"additional","affiliation":[{"name":"Research and Development, Integrated Research, Sydney 2060, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6591-1118","authenticated-orcid":false,"given":"Christoph","family":"Richter","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Lukas","family":"Everding","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Xingwei","family":"Qu","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Yingyu","family":"Wu","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0770-8717","authenticated-orcid":false,"given":"Gordon","family":"Cheng","sequence":"additional","affiliation":[{"name":"Institute for Cognitive Systems, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5998-9640","authenticated-orcid":false,"given":"J\u00f6rg","family":"Conradt","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory, Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2045","DOI":"10.1038\/srep38565","article-title":"Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks","volume":"6","author":"Meng","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1109\/MRA.2012.2229936","article-title":"Brain-Controlled Wheelchairs: A Robotic Architecture","volume":"20","author":"Carlson","year":"2013","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1152\/physrev.00027.2016","article-title":"Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation","volume":"97","author":"Lebedev","year":"2017","journal-title":"Physiol. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Lecun","year":"2015","journal-title":"Nature"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"016003","DOI":"10.1088\/1741-2560\/14\/1\/016003","article-title":"A novel deep learning approach for classification of EEG motor imagery signals","volume":"14","author":"Tabar","year":"2016","journal-title":"J. Neural Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Thomas, J., Maszczyk, T., Sinha, N., Kluge, T., and Dauwels, J. (2017, January 5\u20138). Deep learning-based classification for brain-computer interfaces. Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada.","DOI":"10.1109\/SMC.2017.8122608"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5619","DOI":"10.1109\/TNNLS.2018.2789927","article-title":"Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks","volume":"29","author":"Sakhavi","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, J., Yan, C., and Gong, X. (2017, January 22\u201325). Deep convolutional neural network for decoding motor imagery based brain computer interface. Proceedings of the 2017 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Xiamen, China.","DOI":"10.1109\/ICSPCC.2017.8242581"},{"key":"ref_10","unstructured":"Leeb, R., Brunner, C., Mueller-Put, G., Schloegl, A., and Pfurtscheller, G. (2008). BCI Competition 2008-Graz Data Set b, Graz University of Technology."},{"key":"ref_11","unstructured":"Greaves, A.S. (2018, March 12). Classification of EEG with Recurrent Neural Networks. Available online: https:\/\/cs224d.stanford.edu\/reports\/GreavesAlex.pdf."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Forney, E.M., and Anderson, C.W. (August, January 31). Classification of EEG during imagined mental tasks by forecasting with Elman Recurrent Neural Networks. Proceedings of the The 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.","DOI":"10.1109\/IJCNN.2011.6033579"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hema, C.R., Paulraj, M.P., Yaacob, S., Adom, A.H., and Nagarajan, R. (2008, January 1\u20133). Recognition of motor imagery of hand movements for a BMI using PCA features. Proceedings of the 2008 International Conference on Electronic Design, Penang, Malaysia.","DOI":"10.1109\/ICED.2008.4786683"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, X., Yao, L., Huang, C., Sheng, Q.Z., and Wang, X. (arXiv, 2017). Enhancing mind controlled smart living through recurrent neural networks, arXiv.","DOI":"10.1007\/978-3-319-70096-0_76"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_16","unstructured":"An, J., and Cho, S. (2016, January 18\u201320). Hand motion identification of grasp-and-lift task from electroencephalography recordings using recurrent neural networks. Proceedings of the 2016 International Conference on Big Data and Smart Computing (BigComp), Hong Kong, China."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cho, K., Merrienboer, B.V., Bahdanau, D., and Bengio, Y. (arXiv, 2014). On the properties of neural machine translation: Encoder-decoder approaches, arXiv.","DOI":"10.3115\/v1\/W14-4012"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep learning with convolutional neural networks for EEG decoding and visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"ref_19","unstructured":"Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., and Lance, B.J. (arXiv, 2016). EEGnet: A compact convolutional network for EEG-based brain-computer interfaces, arXiv."},{"key":"ref_20","unstructured":"Bashivan, P., Rish, I., Yeasin, M., and Codella, N. (arXiv, 2015). Learning representations from EEG with deep recurrent-convolutional neural networks, arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Popov, E., and Fomenkov, S. (2016, January 19\u201320). Classification of hand motions in EEG signals using recurrent neural networks. Proceedings of the 2016 2nd International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM), Chelyabinsk, Russia.","DOI":"10.1109\/ICIEAM.2016.7911620"},{"key":"ref_22","unstructured":"(2018, March 12). Guger Technologies. Available online: http:\/\/www.gtec.at\/."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"065003","DOI":"10.1088\/1741-2552\/aae186","article-title":"Gumpy: A Python toolbox suitable for hybrid brain\u2013computer interfaces","volume":"15","author":"Tayeb","year":"2018","journal-title":"J. Neural Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.jneumeth.2010.07.015","article-title":"FASTER: Fully Automated Statistical Thresholding for EEG artifact Rejection","volume":"192","author":"Nolan","year":"2010","journal-title":"J. Neurosci. Methods"},{"key":"ref_25","unstructured":"Team, T.T.D. (arXiv, 2016). Theano: A Python framework for fast computation of mathematical expressions, arXiv."},{"key":"ref_26","unstructured":"Chollet, F. (2018, April 04). Keras, 2015. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_27","first-page":"625","article-title":"Why Does Unsupervised Pre-training Help Deep Learning?","volume":"11","author":"Erhan","year":"2010","journal-title":"J. Mach. Learn."},{"key":"ref_28","unstructured":"Sun, D.L., and Smith, J.O. (2018, March 26). Estimating a Signal from a Magnitude Spectrogram via Convex Optimization. Available online: https:\/\/arxiv.org\/pdf\/1209.2076.pdf."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1842","DOI":"10.1016\/S1388-2457(99)00141-8","article-title":"Event-related EEG\/MEG synchronization and desynchronization: Basic principles","volume":"110","author":"Pfurtscheller","year":"1999","journal-title":"Clin. Neurophysiol."},{"key":"ref_30","unstructured":"loffe, S., and Szegedy, C. (arXiv, 2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arXiv."},{"key":"ref_31","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2014). Adam: A method for stochastic optimization, arXiv."},{"key":"ref_32","unstructured":"Liang, M., and Hu, X. (2015, January 7\u201312). Recurrent convolutional neural network for object recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_33","first-page":"1991","article-title":"Multiclass Common Spatial Patterns and Information Theoretic Feature Extraction","volume":"8","author":"Buss","year":"2008","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Brodu, N., Lotte, F., and Lecuyer, A. (2011, January 11\u201315). Comparative study of band-power extraction techniques for Motor Imagery classification. Proceedings of the 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), Paris, France.","DOI":"10.1109\/CCMB.2011.5952105"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1016\/S1388-2457(98)00038-8","article-title":"Designing optimal spatial filters for single-trial EEG classification in a movement task","volume":"110","author":"Gerking","year":"1999","journal-title":"Clin. Neurophysiol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sherwani, F., Shanta, S., Ibrahim, B.S.K.K., and Huq, M.S. (2016, January 4\u20138). Wavelet based feature extraction for classification of motor imagery signals. Proceedings of the 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES), Kuala Lumpur, Malaysia.","DOI":"10.1109\/IECBES.2016.7843474"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1016\/0167-8655(94)90127-9","article-title":"Floating search methods in feature selection","volume":"15","author":"Pudil","year":"2008","journal-title":"Pattern Recognit. Lett."},{"key":"ref_38","unstructured":"(2018, April 15). Lab Streaming Layer. Available online: https:\/\/github.com\/sccn\/labstreaminglayer."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hessel, M., Soyer, H., Espeholt, L., Czarnecki, W., Schmitt, S., and van Hasselt, H. (arXiv, 2018). Multi-task deep reinforcement learning with popart, arXiv.","DOI":"10.1609\/aaai.v33i01.33013796"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1007\/s11704-016-6159-1","article-title":"A survey of neural network accelerators","volume":"11","author":"Li","year":"2017","journal-title":"Front. Comput. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1126\/science.1254642","article-title":"A million spiking-neuron integrated circuit with a scalable communication network and interface","volume":"345","author":"Merolla","year":"2014","journal-title":"Science"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/1\/210\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:24:25Z","timestamp":1760185465000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/1\/210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,8]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,1]]}},"alternative-id":["s19010210"],"URL":"https:\/\/doi.org\/10.3390\/s19010210","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints201809.0481.v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,8]]}}}