{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T10:54:07Z","timestamp":1778324047318,"version":"3.51.4"},"reference-count":40,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T00:00:00Z","timestamp":1645488000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In recent years, the successful application of Deep Learning methods to classification problems has had a huge impact in many domains. (1) Background: In biomedical engineering, the problem of gesture recognition based on electromyography is often addressed as an image classification problem using Convolutional Neural Networks. Recently, a specific class of these models called Temporal Convolutional Networks (TCNs) has been successfully applied to this task. (2) Methods: In this paper, we approach electromyography-based hand gesture recognition as a sequence classification problem using TCNs. Specifically, we investigate the real-time behavior of our previous TCN model by performing a simulation experiment on a recorded sEMG dataset. (3) Results: The proposed network trained with data augmentation yields a small improvement in accuracy compared to our existing model. However, the classification accuracy is decreased in the real-time evaluation, showing that the proposed TCN architecture is not suitable for such applications. (4) Conclusions: The real-time analysis helps in understanding the limitations of the model and exploring new ways to improve its performance.<\/jats:p>","DOI":"10.3390\/s22051694","type":"journal-article","created":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T22:35:00Z","timestamp":1645569300000},"page":"1694","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Real-Time Analysis of Hand Gesture Recognition with Temporal Convolutional Networks"],"prefix":"10.3390","volume":"22","author":[{"given":"Panagiotis","family":"Tsinganos","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Patras, 265 04 Patras, Greece"},{"name":"Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, 1050 Ixelles, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bart","family":"Jansen","sequence":"additional","affiliation":[{"name":"Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, 1050 Ixelles, Belgium"},{"name":"Imec, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1180-1968","authenticated-orcid":false,"given":"Jan","family":"Cornelis","sequence":"additional","affiliation":[{"name":"Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel, 1050 Ixelles, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3872-4325","authenticated-orcid":false,"given":"Athanassios","family":"Skodras","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Patras, 265 04 Patras, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-012-9356-9","article-title":"Vision based hand gesture recognition for human computer interaction: A survey","volume":"43","author":"Rautaray","year":"2015","journal-title":"Artif. Intell. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, X., Zhang, X., Zhao, Z.Y., Yang, J.H., Lantz, V., and Wang, K.Q. (2007, January 11\u201313). Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers. Proceedings of the 2007 11th IEEE International Symposium on Wearable Computers, Boston, MA, USA.","DOI":"10.1109\/ISWC.2007.4373769"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2566","DOI":"10.1016\/j.ridd.2011.07.002","article-title":"A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities","volume":"32","author":"Chang","year":"2011","journal-title":"Res. Dev. Disabil."},{"key":"ref_4","unstructured":"Omelina, L., Jansen, B., Bonnech\u00e8re, B., Van Sint Jan, S., and Cornelis, J. (2012, January 10\u201312). Serious games for physical rehabilitation: Designing highly configurable and adaptable games. Proceedings of the 9th International Conference on Disability, Virtual Reality & Associated Technologies, Laval, France."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1682\/JRRD.2010.09.0177","article-title":"Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use","volume":"48","author":"Scheme","year":"2011","journal-title":"J. Rehabil. Res. Dev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.patrec.2019.07.021","article-title":"EMG-based online classification of gestures with recurrent neural networks","volume":"128","author":"Neto","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, Z., Zuo, J., Han, Z., Han, X., Sun, C., and Wang, Z. (2020, January 26\u201328). Intelligent Classification of Multi-Gesture EMG Signals Based on LSTM. Proceedings of the 2020 International Conference on Artificial Intelligence and Electromechanical Automation (AIEA), Tianjin, China.","DOI":"10.1109\/AIEA51086.2020.00020"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Azhiri, R.B., Esmaeili, M., and Nourani, M. (2021). Real-time EMG signal classification via recurrent neural networks. arXiv.","DOI":"10.1109\/BIBM52615.2021.9669872"},{"key":"ref_9","unstructured":"Bai, S., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Lea, C., Flynn, M.D., Vidal, R., Reiter, A., and Hager, G.D. (2017, January 21\u201326). Temporal convolutional networks for action segmentation and detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.113"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/10.204774","article-title":"A new strategy for multifunction myoelectric control","volume":"40","author":"Hudgins","year":"1993","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1743-0003-6-41","article-title":"Multi-subject\/daily-life activity EMG-based control of mechanical hands","volume":"6","author":"Castellini","year":"2009","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kuzborskij, I., Gijsberts, A., and Caputo, B. (September, January 28). On the challenge of classifying 52 hand movements from surface electromyography. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA.","DOI":"10.1109\/EMBC.2012.6347099"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"140053","DOI":"10.1038\/sdata.2014.53","article-title":"Electromyography data for non-invasive naturally-controlled robotic hand prostheses","volume":"1","author":"Atzori","year":"2014","journal-title":"Sci. Data"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1109\/TNSRE.2014.2303394","article-title":"Movement Error Rate for Evaluation of Machine Learning Methods for sEMG-Based Hand Movement Classification","volume":"22","author":"Gijsberts","year":"2014","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Atzori, M., Gijsberts, A., Heynen, S., Hager, A.G.M., Deriaz, O., Van Der Smagt, P., Castellini, C., Caputo, B., and Muller, H. (2012, January 24\u201327). Building the Ninapro database: A resource for the biorobotics community. Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, Rome, Italy.","DOI":"10.1109\/BioRob.2012.6290287"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Park, K.H., and Lee, S.W. (2016, January 22\u201324). Movement intention decoding based on deep learning for multiuser myoelectric interfaces. Proceedings of the 2016 4th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Korea.","DOI":"10.1109\/IWW-BCI.2016.7457459"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"9","DOI":"10.3389\/fnbot.2016.00009","article-title":"Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands","volume":"10","author":"Atzori","year":"2016","journal-title":"Front. Neurorobot."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., and Skodras, A. (2018, January 19\u201321). Deep Learning in EMG-based Gesture Recognition. Proceedings of the 5th International Conference on Physiological Computing Systems, Seville, Spain.","DOI":"10.5220\/0006960201070114"},{"key":"ref_20","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"36571","DOI":"10.1038\/srep36571","article-title":"Gesture recognition by instantaneous surface EMG images","volume":"6","author":"Geng","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.patrec.2017.12.005","article-title":"A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface","volume":"119","author":"Wei","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_23","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1109\/TNSRE.2013.2287383","article-title":"Is Accurate Mapping of EMG Signals on Kinematics Needed for Precise Online Myoelectric Control?","volume":"22","author":"Jiang","year":"2014","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1109\/TNSRE.2013.2282898","article-title":"Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms","volume":"22","author":"Muceli","year":"2014","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1109\/TNSRE.2014.2366752","article-title":"Spatial Correlation of High Density EMG Signals Provides Features Robust to Electrode Number and Shift in Pattern Recognition for Myocontrol","volume":"23","author":"Stango","year":"2015","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_27","unstructured":"Tsinganos, P. (2021). Multi-Channel EMG Pattern Classification Based on Deep Learning. [Ph.D. Thesis, University of Patras]."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Du, Y., Jin, W., Wei, W., Hu, Y., and Geng, W. (2017). Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation. Sensors, 17.","DOI":"10.3390\/s17030458"},{"key":"ref_29","unstructured":"Li, Y., Wang, N., Shi, J., Liu, J., and Hou, X. (2016). Revisiting Batch Normalization For Practical Domain Adaptation. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning","volume":"27","author":"Fall","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., and Skodras, A. (2019, January 12\u201317). Improved Gesture Recognition Based on sEMG Signals and TCN. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683239"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., and Skodras, A. (2020). Data Augmentation of Surface Electromyography for Hand Gesture Recognition. Sensors, 20.","DOI":"10.3390\/s20174892"},{"key":"ref_33","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the 2016 International Conference on Learning Representations (ICLR), San Juan, Puerto Rico."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016, January 12\u201317). Hierarchical Attention Networks for Document Classification. Proceedings of the NAACL-HLT, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-1174"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Yang, K., Qian, J., and Zhang, L. (2019). Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network. Sensors, 19.","DOI":"10.3390\/s19143170"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1109\/TBME.2003.813539","article-title":"A robust, real-time control scheme for multifunction myoelectric control","volume":"50","author":"Englehart","year":"2003","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1109\/TBME.2019.2943309","article-title":"Stable Responsive EMG Sequence Prediction and Adaptive Reinforcement With Temporal Convolutional Networks","volume":"67","author":"Betthauser","year":"2020","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1109\/TBCAS.2019.2959160","article-title":"Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor","volume":"14","author":"Zanghieri","year":"2020","journal-title":"IEEE Trans. Biomed. Circuits Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1694\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:24:30Z","timestamp":1760135070000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1694"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,22]]},"references-count":40,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22051694"],"URL":"https:\/\/doi.org\/10.3390\/s22051694","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,22]]}}}