{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T15:59:27Z","timestamp":1780502367130,"version":"3.54.1"},"reference-count":31,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,28]],"date-time":"2023-12-28T00:00:00Z","timestamp":1703721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Basic Science (Natural Science) research project of higher education institutions in Jiangsu Province","award":["23KJD520011"],"award-info":[{"award-number":["23KJD520011"]}]},{"name":"Basic Science (Natural Science) research project of higher education institutions in Jiangsu Province","award":["22KJD520008"],"award-info":[{"award-number":["22KJD520008"]}]},{"name":"Basic Science (Natural Science) research project of higher education institutions in Jiangsu Province","award":["21KJD210004"],"award-info":[{"award-number":["21KJD210004"]}]},{"name":"Basic Science (Natural Science) research project of higher education institutions in Jiangsu Province","award":["17KJB520031"],"award-info":[{"award-number":["17KJB520031"]}]},{"name":"Basic Science (Natural Science) research project of higher education institutions in Jiangsu Province","award":["22KJB520032"],"award-info":[{"award-number":["22KJB520032"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Servo motors play an important role in automation equipment and have been used in several manufacturing fields. However, the commonly used control methods need their parameters to be set manually, which is rather difficult, and this means that these methods generally cannot adapt to changes in operation conditions. Therefore, in this study, we propose an intelligent control method for a servo motor based on reinforcement learning and that can train an agent to produce a duty cycle according to the servo error between the current state and the target speed or torque. The proposed method can adjust its control strategy online to reduce the servo error caused by a change in operation conditions. We verify its performance on three different servo motors and control tasks. The experimental results show that the proposed method can achieve smaller servo errors than others in most cases.<\/jats:p>","DOI":"10.3390\/a17010014","type":"journal-article","created":{"date-parts":[[2023,12,28]],"date-time":"2023-12-28T09:35:21Z","timestamp":1703756121000},"page":"14","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["An Intelligent Control Method for Servo Motor Based on Reinforcement Learning"],"prefix":"10.3390","volume":"17","author":[{"given":"Depeng","family":"Gao","sequence":"first","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuai","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an 710000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuwei","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9881-8489","authenticated-orcid":false,"given":"Haifei","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangxiang","family":"Mei","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuxi","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianlin","family":"Qiu","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Nantong Institute of Technology, Nantong 226001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,28]]},"reference":[{"key":"ref_1","unstructured":"Jinkun, L. (2004). MATLAB Simulation of Advanced PID Control, Electronic Industry Press. [2nd ed.]."},{"key":"ref_2","unstructured":"Baojun, G., Yanping, L., and Dajun, T. (2020). Electromechanics, Higher Education Press."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1016\/j.isatra.2023.04.005","article-title":"Intelligent PID control of an industrial electro-hydraulic system","volume":"139","author":"Coskun","year":"2023","journal-title":"ISA Trans."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, P., He, Z., Chen, C., and Xu, J. (2018). Control Strategy of Speed Servo Systems Based on Deep Reinforcement Learning. Algorithms, 11.","DOI":"10.3390\/a11050065"},{"key":"ref_5","first-page":"18","article-title":"Control strategy of electro-mechanical actuator based on deep reinforcement learning-PI control","volume":"49","author":"Maosheng","year":"2022","journal-title":"Appl. Sci. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, C.-H., Guo, C.-W.C., Tsay, D.-M., and Perng, J.-W. (2021). PMSM Speed Control Based on Particle Swarm Optimization and Deep Deterministic Policy Gradient under Load Disturbance. Machines, 9.","DOI":"10.3390\/machines9120343"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4650","DOI":"10.1109\/TII.2019.2948387","article-title":"Controller Design for Electrical Drives by Deep Reinforcement Learning: A Proof of Concept","volume":"16","author":"Schenke","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6030","DOI":"10.1109\/TITS.2023.3251376","article-title":"Ensemble quantile networks: Uncertainty-aware reinforcement learning with applications in autonomous driving","volume":"24","author":"Hoel","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"7932","DOI":"10.1109\/TITS.2023.3266885","article-title":"Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving","volume":"24","author":"Zhou","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"10233","DOI":"10.1109\/TII.2023.3240585","article-title":"Transformer-Based Imitative Reinforcement Learning for Multirobot Path Planning","volume":"19","author":"Chen","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"119910","DOI":"10.1016\/j.eswa.2023.119910","article-title":"Reinforcement learning-based multi-strategy cuckoo search algorithm for 3D UAV path planning","volume":"223","author":"Yu","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Orr, J., and Dutta, A. (2023). Multi-agent deep reinforcement learning for multi-robot applications: A survey. Sensors, 23.","DOI":"10.3390\/s23073625"},{"key":"ref_13","first-page":"1652","article-title":"Don\u2019t start from scratch: Leveraging prior data to automate robotic reinforcement learning","volume":"205","author":"Walke","year":"2023","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1088\/1742-6596\/2183\/1\/012024","article-title":"Reinforcement learning control method of torque stability of three-phase permanent magnet synchronous motor","volume":"2183","author":"Tian","year":"2022","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_16","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5409","DOI":"10.1007\/s00521-020-05352-1","article-title":"Deep reinforcement learning for permanent magnet synchronous motor speed control systems","volume":"33","author":"Song","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_18","first-page":"111153","article-title":"Actor\u2013critic learning based PID control for robotic manipulators","volume":"151","author":"Hamed","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_19","unstructured":"Sutton, R.S., McAllester, D., Singh, S., and Mansour, Y. (2000). Policy Gradient Methods for Reinforcement Learning with Function Approximation, Morgan Kaufmann Publisher. Advances in Neural Information Processing Systems."},{"key":"ref_20","first-page":"647","article-title":"Learning General Policies with Policy Gradient Methods","volume":"19","author":"Bonet","year":"2023","journal-title":"Proc. Int. Conf. Princ. Knowl. Represent. Reason."},{"key":"ref_21","unstructured":"Scott, F., Herke, V.H., and David, M. (2018). Addressing Function Approximation Error in Actor-Critic Methods. International Conference on Machine Learning. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2433","DOI":"10.1007\/s10994-023-06303-2","article-title":"On the sample complexity of actor-critic method for reinforcement learning with function approximation","volume":"112","author":"Kumar","year":"2023","journal-title":"Mach. Learn."},{"key":"ref_23","unstructured":"Van, H.H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_24","first-page":"1406","article-title":"A Review of Deep Reinforcement Learning Based on Value Function and Strategy Gradient","volume":"42","author":"Jianwei","year":"2019","journal-title":"Chin. J. Comput."},{"key":"ref_25","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv."},{"key":"ref_26","unstructured":"Schaul, T., Horgan, D., Gregor, K., and Silver, D. (2015, January 6\u201311). Universal Value Function Approximators. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.acha.2022.12.002","article-title":"The universal approximation theorem for complex-valued neural networks","volume":"64","author":"Voigtlaender","year":"2023","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_28","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft actor-critic algorithms and applications. arXiv."},{"key":"ref_29","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2498","DOI":"10.21105\/joss.02498","article-title":"Gym-electric-motor (GEM): A python toolbox for the simulation of electric drive systems","volume":"6","author":"Balakrishna","year":"2021","journal-title":"J. Open Source Softw."},{"key":"ref_31","unstructured":"Diederik, P.K., and Jimmy, B. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/1\/14\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:43:25Z","timestamp":1760132605000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/1\/14"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,28]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["a17010014"],"URL":"https:\/\/doi.org\/10.3390\/a17010014","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,28]]}}}