{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T06:14:11Z","timestamp":1778307251924,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,4,12]],"date-time":"2023-04-12T00:00:00Z","timestamp":1681257600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012511","name":"National Polytechnic School","doi-asserted-by":"publisher","award":["PIGR-19-07"],"award-info":[{"award-number":["PIGR-19-07"]}],"id":[{"id":"10.13039\/501100012511","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In recent years, hand gesture recognition (HGR) technologies that use electromyography (EMG) signals have been of considerable interest in developing human\u2013machine interfaces. Most state-of-the-art HGR approaches are based mainly on supervised machine learning (ML). However, the use of reinforcement learning (RL) techniques to classify EMGs is still a new and open research topic. Methods based on RL have some advantages such as promising classification performance and online learning from the user\u2019s experience. In this work, we propose a user-specific HGR system based on an RL-based agent that learns to characterize EMG signals from five different hand gestures using Deep Q-network (DQN) and Double-Deep Q-Network (Double-DQN) algorithms. Both methods use a feed-forward artificial neural network (ANN) for the representation of the agent policy. We also performed additional tests by adding a long\u2013short-term memory (LSTM) layer to the ANN to analyze and compare its performance. We performed experiments using training, validation, and test sets from our public dataset, EMG-EPN-612. The final accuracy results demonstrate that the best model was DQN without LSTM, obtaining classification and recognition accuracies of up to 90.37%\u00b110.7% and 82.52%\u00b110.9%, respectively. The results obtained in this work demonstrate that RL methods such as DQN and Double-DQN can obtain promising results for classification and recognition problems based on EMG signals.<\/jats:p>","DOI":"10.3390\/s23083905","type":"journal-article","created":{"date-parts":[[2023,4,12]],"date-time":"2023-04-12T02:08:11Z","timestamp":1681265291000},"page":"3905","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3502-020X","authenticated-orcid":false,"given":"\u00c1ngel Leonardo","family":"Valdivieso Caraguay","sequence":"first","affiliation":[{"name":"Artificial Intelligence and Computer Vision Research Lab, Escuela Polit\u00e9cnica Nacional, Quito 170517, Ecuador"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6372-7405","authenticated-orcid":false,"given":"Juan Pablo","family":"V\u00e1sconez","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, Universidad Andres Bello, Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5184-3759","authenticated-orcid":false,"given":"Lorena Isabel","family":"Barona L\u00f3pez","sequence":"additional","affiliation":[{"name":"Artificial Intelligence and Computer Vision Research Lab, Escuela Polit\u00e9cnica Nacional, Quito 170517, Ecuador"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5275-7262","authenticated-orcid":false,"given":"Marco E.","family":"Benalc\u00e1zar","sequence":"additional","affiliation":[{"name":"Artificial Intelligence and Computer Vision Research Lab, Escuela Polit\u00e9cnica Nacional, Quito 170517, Ecuador"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jaramillo-Y\u00e1nez, A., Benalc\u00e1zar, M.E., and Mena-Maldonado, E. (2020). Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review. Sensors, 20.","DOI":"10.3390\/s20092467"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"012012","DOI":"10.1088\/1757-899X\/99\/1\/012012","article-title":"An overview of hand gestures recognition system techniques","volume":"Volume 99","author":"Sulaiman","year":"2015","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Romero, R., Cruz, P.J., V\u00e1sconez, J.P., Benalc\u00e1zar, M., \u00c1lvarez, R., Barona, L., and Valdivieso, \u00c1.L. (2022, January 7\u20139). Hand Gesture and Arm Movement Recognition for Multimodal Control of a 3-DOF Helicopter. Proceedings of the International Conference on Robot Intelligence Technology and Applications, Daejeon, Republic of Korea.","DOI":"10.1007\/978-3-030-97672-9_32"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chico, A., Cruz, P.J., V\u00e1sconez, J.P., Benalc\u00e1zar, M.E., \u00c1lvarez, R., Barona, L., and Valdivieso, \u00c1.L. (2021, January 12\u201315). Hand Gesture Recognition and Tracking Control for a Virtual UR5 Robot Manipulator. Proceedings of the 2021 IEEE Fifth Ecuador Technical Chapters Meeting (ETCM), Cuenca, Ecuador.","DOI":"10.1109\/ETCM53643.2021.9590677"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Colli Alfaro, J.G., and Trejos, A.L. (2022). User-Independent Hand Gesture Recognition Classification Models Using Sensor Fusion. Sensors, 22.","DOI":"10.3390\/s22041321"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3172","DOI":"10.1109\/TCYB.2020.3007173","article-title":"A hierarchical hand gesture recognition framework for sports referee training-based EMG and accelerometer sensors","volume":"52","author":"Pan","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Song, L., Zhang, J., Song, Y., and Yan, M. (2022). Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals. Sensors, 22.","DOI":"10.3390\/s22155855"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, L., Chen, J., and Zhu, W. (2020). Dynamic hand gesture recognition based on a leap motion controller and two-layer bidirectional recurrent neural network. Sensors, 20.","DOI":"10.3390\/s20072106"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nuzzi, C., Pasinetti, S., Lancini, M., Docchio, F., and Sansoni, G. (2018, January 16\u201318). Deep learning based machine vision: First steps towards a hand gesture recognition set up for collaborative robots. Proceedings of the 2018 Workshop on Metrology for Industry 4.0 and IoT, Brescia, Italy.","DOI":"10.1109\/METROI4.2018.8439044"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Chamorro, S., Collier, J., and Grondin, F. (2020, January 4\u20136). Neural Network Based Lidar Gesture Recognition for Realtime Robot Teleoperation. Proceedings of the 2021 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/SSRR53300.2021.9597855"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Oudah, M., Al-Naji, A., and Chahl, J. (2020). Hand gesture recognition based on computer vision: A review of techniques. J. Imaging, 6.","DOI":"10.3390\/jimaging6080073"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mujahid, A., Awan, M.J., Yasin, A., Mohammed, M.A., Dama\u0161evi\u010dius, R., Maskeli\u016bnas, R., and Abdulkareem, K.H. (2021). Real-time hand gesture recognition based on deep learning YOLOv3 model. Appl. Sci., 11.","DOI":"10.3390\/app11094164"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kim, M., Cho, J., Lee, S., and Jung, Y. (2019). IMU sensor-based hand gesture recognition for human\u2013machine interfaces. Sensors, 19.","DOI":"10.3390\/s19183827"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2000261","DOI":"10.1002\/advs.202000261","article-title":"Machine learning glove using self-powered conductive superhydrophobic triboelectric textile for gesture recognition in VR\/AR applications","volume":"7","author":"Wen","year":"2020","journal-title":"Adv. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Benalc\u00e1zar, M.E., Jaramillo, A.G., Zea, A., P\u00e1ez, A., and Andaluz, V.H. (September, January 28). Hand gesture recognition using machine learning and the Myo armband. Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece.","DOI":"10.23919\/EUSIPCO.2017.8081366"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gopal, P., Gesta, A., and Mohebbi, A. (2022). A systematic study on electromyography-based hand gesture recognition for assistive robots using deep learning and machine learning models. Sensors, 22.","DOI":"10.3390\/s22103650"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2623","DOI":"10.1109\/TNSRE.2022.3205026","article-title":"Wearable Real-Time Gesture Recognition Scheme Based on A-Mode Ultrasound","volume":"30","author":"Lu","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"McIntosh, J., Marzo, A., Fraser, M., and Phillips, C. (2017, January 6\u201311). Echoflex: Hand gesture recognition using ultrasound imaging. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025807"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1109\/THMS.2021.3086003","article-title":"Human\u2013machine interaction sensing technology based on hand gesture recognition: A review","volume":"51","author":"Guo","year":"2021","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Donati, R., Kartsch, V., Benini, L., and Benatti, S. (2022, January 11\u201315). BioWolf16: A 16-channel, 24-bit, 4kSPS Ultra-Low Power Platform for Wearable Clinical-grade Bio-potential Parallel Processing and Streaming. Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK.","DOI":"10.1109\/EMBC48229.2022.9871898"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"9","DOI":"10.3389\/fnbot.2016.00009","article-title":"Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands","volume":"10","author":"Atzori","year":"2016","journal-title":"Front. Neurorobot."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xing, K., Ding, Z., Jiang, S., Ma, X., Yang, K., Yang, C., Li, X., and Jiang, F. (2018, January 18\u201321). Hand gesture recognition based on deep learning method. Proceedings of the 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC), Guangzhou, China.","DOI":"10.1109\/DSC.2018.00087"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1007\/BF02350985","article-title":"Surface electromyogram signal modelling","volume":"42","author":"McGill","year":"2004","journal-title":"Med Biol. Eng. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sugiyama, M., and Kawanabe, M. (2012). Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation, MIT Press.","DOI":"10.7551\/mitpress\/9780262017091.001.0001"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1109\/TBME.2003.813539","article-title":"A robust, real-time control scheme for multifunction myoelectric control","volume":"50","author":"Englehart","year":"2003","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep learning for electromyographic hand gesture signal classification using transfer learning","volume":"27","author":"Fall","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1007\/s40747-021-00324-x","article-title":"Hand gesture classification using a novel CNN-crow search algorithm","volume":"7","author":"Gadekallu","year":"2021","journal-title":"Complex Intell. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Devaraj, A., and Nair, A.K. (2020, January 28\u201330). Hand gesture signal classification using machine learning. Proceedings of the 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India.","DOI":"10.1109\/ICCSP48568.2020.9182045"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jabbari, M., Khushaba, R.N., and Nazarpour, K. (2020, January 20\u201324). EMG-based hand gesture classification with long short-term memory deep recurrent neural networks. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9175279"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"V\u00e1sconez, J.P., L\u00f3pez, L.I.B., Caraguay, \u00c1.L.V., Cruz, P.J., \u00c1lvarez, R., and Benalc\u00e1zar, M.E. (2021, January 14\u201317). A Hand Gesture Recognition System Using EMG and Reinforcement Learning: A Q-Learning Approach. Proceedings of the International Conference on Artificial Neural Networks, Bratislava, Slovakia.","DOI":"10.1007\/978-3-030-86380-7_47"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2063","DOI":"10.1109\/TNNLS.2018.2790388","article-title":"Applications of deep learning and reinforcement learning to biological data","volume":"29","author":"Mahmud","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mahmoudi, B., and Sanchez, J.C. (2011). A symbiotic brain-machine interface through value-based decision making. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0014760"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"565","DOI":"10.4028\/www.scientific.net\/AMM.461.565","article-title":"A multi-step neural control for motor brain-machine interface by reinforcement learning","volume":"Volume 461","author":"Wang","year":"2014","journal-title":"Applied Mechanics and Materials"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Song, C., Chen, C., Li, Y., and Wu, X. (2018, January 25\u201327). Deep Reinforcement Learning Apply in Electromyography Data Classification. Proceedings of the 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), Shenzhen, China.","DOI":"10.1109\/CBS.2018.8612213"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5111","DOI":"10.3233\/JIFS-169795","article-title":"Neural reinforcement learning classifier for elbow, finger and hand movements","volume":"35","author":"Kukker","year":"2018","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sharma, R., and Kukker, A. (2017, January 27\u201330). Neural Reinforcement Learning based Identifier for Typing Keys using Forearm EMG Signals. Proceedings of the 9th International Conference on Signal Processing Systems, Auckland, New Zealand.","DOI":"10.1145\/3163080.3163117"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"V\u00e1sconez, J.P., Barona L\u00f3pez, L.I., Valdivieso Caraguay, A.L., and Benalc\u00e1zar, M.E. (2022). Hand Gesture Recognition Using EMG-IMU Signals and Deep Q-Networks. Sensors, 22.","DOI":"10.3390\/s22249613"},{"key":"ref_38","unstructured":"Benalc\u00e1zar, M., Barona, L., Valdivieso, L., Aguas, X., and Zea, J. (2020). EMG-EPN-612 Dataset, CERN."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Barona L\u00f3pez, L.I., Valdivieso Caraguay, \u00c1.L., Vimos, V.H., Zea, J.A., V\u00e1sconez, J.P., \u00c1lvarez, M., and Benalc\u00e1zar, M.E. (2020). An Energy-Based Method for Orientation Correction of EMG Bracelet Sensors in Hand Gesture Recognition Systems. Sensors, 20.","DOI":"10.3390\/s20216327"},{"key":"ref_40","unstructured":"Benalc\u00e1zar, M., Barona, L., Valdivieso, L., Aguas, X., and Zea, J. (2023, April 05). Artificial Intelligence and Computer Vision Research Lab\u2014EMG Gesture Recognition Evaluator. Available online: https:\/\/aplicaciones-ia.epn.edu.ec\/webapps\/home\/session.html?app=EMGGestureRecognitionEvaluator."},{"key":"ref_41","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_44","unstructured":"Kapturowski, S., Ostrovski, G., Quan, J., Munos, R., and Dabney, W. (May, January 30). Recurrent experience replay in distributed reinforcement learning. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_45","unstructured":"Hausknecht, M., and Stone, P. (2015, January 12\u201314). Deep recurrent q-learning for partially observable mdps. Proceedings of the 2015 AAAI Fall Symposium Series, Arlington, VA, USA."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Oh, H., and Kaneko, T. (December, January 30). Deep recurrent Q-network with truncated history. Proceedings of the 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI), Taichung, Taiwan.","DOI":"10.1109\/TAAI.2018.00017"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Seok, W., Kim, Y., and Park, C. (2018, January 10\u201312). Pattern recognition of human arm movement using deep reinforcement learning. Proceedings of the 2018 International Conference on Information Networking (ICOIN), Chiang Mai, Thailand.","DOI":"10.1109\/ICOIN.2018.8343257"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3905\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:14:16Z","timestamp":1760123656000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3905"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,12]]},"references-count":47,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23083905"],"URL":"https:\/\/doi.org\/10.3390\/s23083905","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,12]]}}}