{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T06:37:27Z","timestamp":1763707047648,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T00:00:00Z","timestamp":1744588800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Financiadora de Estudos e Projetos\u2014MCTI\/FINEP","award":["01.21.0101.00","01.24.0167.00","001"],"award-info":[{"award-number":["01.21.0101.00","01.24.0167.00","001"]}]},{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior\u2014Capes","doi-asserted-by":"publisher","award":["01.21.0101.00","01.24.0167.00","001"],"award-info":[{"award-number":["01.21.0101.00","01.24.0167.00","001"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>To improve the adaptability of the hand prosthesis, we propose a new smart control for a physiological finger prosthesis using the advantages of the deep deterministic policy gradient (DDPG) algorithm. A rigid body model was developed to represent the finger as a training environment. The geometric characteristics and physiological physical properties of the finger available in the literature were assumed, but the joint\u2019s stiffness and damping were neglected. The standard DDPG algorithm was modified to train an artificial neural network (ANN) to perform two predetermined trajectories: linear and sinusoidal. The ANN was evaluated through the use of a computational model that simulated the functionality of the finger prosthesis. The model demonstrated the capacity to successfully execute both sinusoidal and linear trajectories, exhibiting a mean error of 3.984\u00b12.899 mm for the sinusoidal trajectory and 3.220\u00b11.419 mm for the linear trajectory. Observing the torques, it was found that the ANN used different strategies to control the movement in order to adapt to the different trajectories. Allowing the ANN to use a combination of both trajectories, our model was able to perform trajectories that differed from purely linear and sinusoidal, showing its ability to adapt to the movement of the physiological finger. The results showed that it was possible to develop a controller for multiple trajectories, which is essential to provide more integrated and personalized prostheses.<\/jats:p>","DOI":"10.3390\/robotics14040049","type":"journal-article","created":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T07:42:01Z","timestamp":1744616521000},"page":"49","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A New Proposal for Intelligent Continuous Controller of Robotic Finger Prostheses Using Deep Deterministic Policy Gradient Algorithm Through Simulated Assessments"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7337-9128","authenticated-orcid":false,"given":"Guilherme de Paula","family":"R\u00fabio","sequence":"first","affiliation":[{"name":"Graduate Program in Mechanical Engineering, Department of Mechanical Engineering, Universidade Federal de Minas Gerais, Avenida Presidente Ant\u00f4nio Carlos 6627, Pampulha, Belo Horizonte 31270-901, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0244-5044","authenticated-orcid":false,"given":"Matheus Carvalho Barbosa","family":"Costa","sequence":"additional","affiliation":[{"name":"Graduate Program in Mechanical Engineering, Department of Mechanical Engineering, Universidade Federal de Minas Gerais, Avenida Presidente Ant\u00f4nio Carlos 6627, Pampulha, Belo Horizonte 31270-901, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1916-0517","authenticated-orcid":false,"given":"Claysson Bruno Santos","family":"Vimieiro","sequence":"additional","affiliation":[{"name":"Graduate Program in Mechanical Engineering, Department of Mechanical Engineering, Universidade Federal de Minas Gerais, Avenida Presidente Ant\u00f4nio Carlos 6627, Pampulha, Belo Horizonte 31270-901, MG, Brazil"},{"name":"Graduate Program in Mechanical Engineering, Pontif\u00edcia Universidade Cat\u00f3lica de Minas Gerais, Avenida Dom Jos\u00e9 Gaspar 500, Cora\u00e7\u00e3o Eucar\u00edstico, Belo Horizonte 30535-901, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"10292","DOI":"10.1109\/JSEN.2022.3169492","article-title":"Non-Invasive Human-Machine Interface (HMI) Systems with Hybrid On-Body Sensors for Controlling Upper-Limb Prosthesis: A Review","volume":"22","author":"Zhou","year":"2022","journal-title":"IEEE Sens. J."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, K., Zhang, J., Wang, L., Zhang, M., Li, J., and Bao, S. (2020). A review of the key technologies for sEMG-based human-robot interaction systems. Biomed. Signal Process. Control, 62.","DOI":"10.1016\/j.bspc.2020.102074"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"39564","DOI":"10.1109\/ACCESS.2019.2906584","article-title":"A Review on Electromyography Decoding and Pattern Recognition for Human-Machine Interaction","volume":"7","author":"Simao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Merletti, R., and Farina, D. (2016). Surface Electromyography: Physiology, Engineering, and Applications, Wiley.","DOI":"10.1002\/9781119082934"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.bspc.2019.02.011","article-title":"A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration","volume":"51","author":"Bi","year":"2019","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Liao, Z., Chen, B., Bai, D., Xu, J., Zheng, Q., Liu, K., and Wu, H. (2023). Human\u2013robot interface based on sEMG envelope signal for the collaborative wearable robot. Biomim. Intell. Robot., 3.","DOI":"10.1016\/j.birob.2022.100079"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.procs.2017.11.259","article-title":"Electromyogram (EMG) signal detection, classification of EMG signals and diagnosis of neuropathy muscle disease","volume":"120","author":"Sadikoglu","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106327","DOI":"10.1016\/j.engappai.2023.106327","article-title":"A comparison of EMG-based hand gesture recognition systems based on supervised and reinforcement learning","volume":"123","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nastarin, A., Akter, A., and Awal, M.A. (2019, January 26\u201328). Robust Control of Hand Prostheses from Surface EMG Signal for Transradial Amputees. Proceedings of the 2019 5th International Conference on Advances in Electrical Engineering (ICAEE), Dhaka, Bangladesh.","DOI":"10.1109\/ICAEE48663.2019.8975630"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.18517\/ijaseit.9.4.4887","article-title":"Design and Implementation of Prosthetic Hand Control Using Myoelectric Signal","volume":"9","author":"Rahmatillah","year":"2019","journal-title":"Int. J. Adv. Sci. Eng. Inf. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1109\/THMS.2016.2641389","article-title":"Toward an Enhanced Human\u2013Machine Interface for Upper-Limb Prosthesis Control with Combined EMG and NIRS Signals","volume":"47","author":"Guo","year":"2017","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/TNSRE.2016.2554884","article-title":"Online Bimanual Manipulation Using Surface Electromyography and Incremental Learning","volume":"25","author":"Strazzulla","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ying, Z., Zhang, X., Li, S., Nakashima, K., Shu, L., and Sugita, N. (2024, January 13\u201317). Real-time Dexterous Prosthesis Hand Control by Decoding Neural Information Based on EMG Decomposition. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10611356"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3145","DOI":"10.1007\/s40435-023-01130-8","article-title":"Silveira, A. Multivariable grasping force control of myoelectric multi-fingered hand prosthesis","volume":"11","author":"Dutra","year":"2023","journal-title":"Int. J. Dyn. Control."},{"key":"ref_15","first-page":"495","article-title":"Intelligent Controller Design for Multifunctional Prosthetics Hand","volume":"6","author":"Ghazali","year":"2017","journal-title":"Int. J. Mech. Eng. Robot. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.bspc.2018.06.012","article-title":"Estimation of continuous and constraint-free 3 DoF wrist movements from surface electromyogram signal using kernel recursive least square tracker","volume":"46","author":"Bakshi","year":"2018","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"026011","DOI":"10.1088\/1741-2552\/aa58f5","article-title":"Neuromusculoskeletal model self-calibration for on-line sequential bayesian moment estimation","volume":"14","author":"Bueno","year":"2017","journal-title":"J. Neural Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/THMS.2021.3129708","article-title":"Finite-Time Observer-Based Variable Impedance Control of Cable-Driven Continuum Manipulators","volume":"52","author":"Liang","year":"2022","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4814","DOI":"10.1109\/TMECH.2024.3394491","article-title":"Adaptive Human\u2013Robot Interaction Torque Estimation with High Accuracy and Strong Tracking Ability for a Lower Limb Rehabilitation Robot","volume":"29","author":"Liang","year":"2024","journal-title":"IEEE\/ASME Trans. Mechatronics"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, Q., Hong, Z., and Zhong, Y. (2022). Learn to swim: Online motion control of an underactuated robotic eel based on deep reinforcement learning. Biomim. Intell. Robot., 2.","DOI":"10.1016\/j.birob.2022.100066"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"104797","DOI":"10.1016\/j.engappai.2022.104797","article-title":"Control with adaptive Q-learning: A comparison for two classical control problems","volume":"112","author":"Figueiredo","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, Z., Ma, H., Ding, Y., Wang, C., and Jin, Y. (2020, January 27\u201329). Motion Planning of Six-DOF Arm Robot Based on Improved DDPG Algorithm. Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China.","DOI":"10.23919\/CCC50068.2020.9188521"},{"key":"ref_23","unstructured":"Lillicrap, T. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dong, R., Du, J., Liu, Y., Heidari, A.A., and Chen, H. (2023). An enhanced deep deterministic policy gradient algorithm for intelligent control of robotic arms. Front. Neuroinform., 17.","DOI":"10.3389\/fninf.2023.1096053"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"105321","DOI":"10.1016\/j.engappai.2022.105321","article-title":"Autonomous unmanned aerial vehicle navigation using reinforcement learning: A systematic review","volume":"115","author":"AlMahamid","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"108911","DOI":"10.1016\/j.engappai.2024.108911","article-title":"A survey on reinforcement learning in aviation applications","volume":"136","author":"Razzaghi","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_27","unstructured":"Mnih, V. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105798","DOI":"10.1016\/j.engappai.2022.105798","article-title":"Optimization of reward shaping function based on genetic algorithm applied to a cross validated deep deterministic policy gradient in a powered landing guidance problem","volume":"120","author":"Nugroho","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/j.procir.2018.03.067","article-title":"Continuous motion planning for industrial robots based on direct sensory input","volume":"72","author":"Meyes","year":"2018","journal-title":"Procedia CIRP"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e30697","DOI":"10.1016\/j.heliyon.2024.e30697","article-title":"Deep deterministic policy gradient algorithm: A systematic review","volume":"10","author":"Sumiea","year":"2024","journal-title":"Heliyon"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kumar, P., Tripathi, S., and Misra, R. (2023, January 15\u201318). Proximal Policy Optimization based computations offloading for delay optimization in UAV-assisted mobile edge computing. Proceedings of the 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy.","DOI":"10.1109\/BigData59044.2023.10386861"},{"key":"ref_32","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft Actor-Critic Algorithms and Applications. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1007\/s40430-024-05028-0","article-title":"Squat and tuck jump maneuver for single-legged robot with an active toe joint using model-free deep reinforcement learning","volume":"46","author":"Moslemi","year":"2024","journal-title":"J. Braz. Soc. Mech. Sci. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"59225","DOI":"10.1109\/ACCESS.2020.2982702","article-title":"DDPG-based decision-making strategy of adaptive cruising for heavy vehicles considering stability","volume":"8","author":"Sun","year":"2020","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"121519","DOI":"10.1016\/j.apenergy.2023.121519","article-title":"Reward adaptive wind power tracking control based on deep deterministic policy gradient","volume":"348","author":"Chen","year":"2023","journal-title":"Appl. Energy"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, Z., Li, S., Zhu, K., Li, S., and Chen, H. (2024, January 24\u201326). Optimization of Hydraulic Robot Arm Motion Control Algorithm Based on Neural Network Algorithm. Proceedings of the 2024 IEEE 4th International Conference on Electronic Technology, Communication and Information (ICETCI), Changchun, China.","DOI":"10.1109\/ICETCI61221.2024.10594685"},{"key":"ref_37","unstructured":"Spong, M.W., Hutchinson, S., and Vidyasagar, M. (2020). Robot Modeling and Control, John Wiley & Sons."},{"key":"ref_38","unstructured":"Thomas, S.J., Zeni, J.A., and Winter, D.A. (2022). Winter\u2019s Biomechanics and Motor Control of Human Movement, John Wiley & Sons."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1002\/hfm.20714","article-title":"Dorsal and palmar aspect dimensions of hand anthropometry for designing hand tools and protections","volume":"28","author":"Vergara","year":"2018","journal-title":"Hum. Factors Ergon. Manuf. Serv. Ind."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1682\/JRRD.2010.09.0177","article-title":"Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use","volume":"48","author":"Scheme","year":"2011","journal-title":"J. Rehabil. Res. Dev."},{"key":"ref_41","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv."},{"key":"ref_42","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv."},{"key":"ref_43","unstructured":"Chollet, F. (2024, March 22). Keras. Available online: https:\/\/keras.io."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"105853","DOI":"10.1016\/j.engappai.2023.105853","article-title":"Development and testing of a virtual simulator for a myoelectric prosthesis prototype\u2014The PRISMA Hand II\u2014to improve its usability and acceptability","volume":"121","author":"Leccia","year":"2023","journal-title":"Eng. Appl. Artif. Intell."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/4\/49\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:14:11Z","timestamp":1760030051000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/4\/49"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,14]]},"references-count":44,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["robotics14040049"],"URL":"https:\/\/doi.org\/10.3390\/robotics14040049","relation":{},"ISSN":["2218-6581"],"issn-type":[{"type":"electronic","value":"2218-6581"}],"subject":[],"published":{"date-parts":[[2025,4,14]]}}}