{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T12:49:03Z","timestamp":1775479743459,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,9,30]],"date-time":"2018-09-30T00:00:00Z","timestamp":1538265600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This paper proposes a hybrid Zeigler-Nichols (Z-N) reinforcement learning approach for online tuning of the parameters of the Proportional Integral Derivative (PID) for controlling the speed of a DC motor. The PID gains are set by the Z-N method, and are then adapted online through the fuzzy Q-Learning agent. The fuzzy Q-Learning agent is used instead of the conventional Q-Learning, in order to deal with the continuous state-action space. The fuzzy Q-Learning agent defines its state according to the value of the error. The output signal of the agent consists of three output variables, in which each one defines the percentage change of each gain. Each gain can be increased or decreased from 0% to 50% of its initial value. Through this method, the gains of the controller are adjusted online via the interaction of the environment. The knowledge of the expert is not a necessity during the setup process. The simulation results highlight the performance of the proposed control strategy. After the exploration phase, the settling time is reduced in the steady states. In the transient states, the response has less amplitude oscillations and reaches the equilibrium point faster than the conventional PID controller.<\/jats:p>","DOI":"10.3390\/a11100148","type":"journal-article","created":{"date-parts":[[2018,10,2]],"date-time":"2018-10-02T08:23:50Z","timestamp":1538468630000},"page":"148","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Fuzzy Q-Learning Agent for Online Tuning of PID Controller for DC Motor Speed Control"],"prefix":"10.3390","volume":"11","author":[{"given":"Panagiotis","family":"Kofinas","sequence":"first","affiliation":[{"name":"Department of Industrial Design and Production Engineering, University of West Attica, GR-12241 Egaleo-Athens, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anastasios I.","family":"Dounis","sequence":"additional","affiliation":[{"name":"Department of Industrial Design and Production Engineering, University of West Attica, GR-12241 Egaleo-Athens, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1177\/0142331214535619","article-title":"DC motor speed control by self-tuning fuzzy PID algorithm","volume":"37","author":"Javadi","year":"2015","journal-title":"Trans. Inst. Meas. Control"},{"key":"ref_2","unstructured":"Meshram, P.M., and Kanojiya, R.G. (2012, January 30\u201331). Tuning of PID Controller using Ziegler-Nichols Method for Speed Control of DC Motor. Proceedings of the IEEE International Conference on Advances in Engineering, Science and Management (ICAESM-2012), Nagapattinam, Tamil Nadu, India."},{"key":"ref_3","unstructured":"Wang, L.-X. (1997). A Course in Fuzzy Systems and Control, Prentice Hall PTR."},{"key":"ref_4","first-page":"6988","article-title":"Research of PID Control Algorithm Based on Neural Network","volume":"13","author":"Liu","year":"2011","journal-title":"Energy Procedia"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/S1474-6670(17)43373-8","article-title":"Neural Network Based Adaptive PID Controller","volume":"30","author":"Badr","year":"1997","journal-title":"IFAC Proc. Vol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1016\/S1474-6670(17)38283-6","article-title":"A new on-line PID tuning method using neural networks","volume":"33","author":"Rad","year":"2000","journal-title":"IFAC Proc. Vol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Du, X., Wang, J., Jegatheesan, V., and Shi, G. (2018). Dissolved Oxygen Control in Activated Sludge Process Using a Neural Network-Based Adaptive PID Algorithm. Appl. Sci., 8.","DOI":"10.3390\/app8020261"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Muhammad, Z., Yusoff, Z.M., Kasuan, N., Nordin, M.N.N., Rahiman, M.H.F., and Taib, M.N. (2013, January 19\u201320). Online Tuning PID using Fuzzy Logic Controller with Self-Tuning Method. Proceedings of the IEEE 3rd International Conference on System Engineering and Technology, Shah Alam, Malaysia.","DOI":"10.1109\/ICSEngT.2013.6650150"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.renene.2013.04.014","article-title":"Adaptive fuzzy gain scheduling PID controller for maximum power point tracking of photovoltaic system","volume":"60","author":"Dounis","year":"2013","journal-title":"Renew. Energy"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Qin, Y., Sun, L., Hua, Q., and Liu, P. (2018). A Fuzzy Adaptive PID Controller Design for Fuel Cell Power Plant. Sustainability, 10.","DOI":"10.3390\/su10072438"},{"key":"ref_11","unstructured":"Amaral, J.F.M., Tanscheit, R., and Pacheco, M.A.C. (2001, January 11\u201315). Tuning PID Controllers through Genetic Algorithms. Proceedings of the 2001 WSES International Conference on Evolutionary Computation, Puerto De La, Spain."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1155\/2014\/791230","article-title":"Genetic Algorithm Based PID Controller Tuning Approach for Continuous Stirred Tank Reactor","volume":"2014","author":"Jayachitra","year":"2014","journal-title":"Adv. Artif. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sheng, L., and Li, W. (2018). Optimization Design by Genetic Algorithm Controller for Trajectory Control of a 3-RRR Parallel Robot. Algorithms, 11.","DOI":"10.3390\/a11010007"},{"key":"ref_14","first-page":"321","article-title":"An anti-windup self-tuning fuzzy PID controller for speed control of brushless DC motor","volume":"58","author":"Jigang","year":"2017","journal-title":"J. Control Meas. Electron. Comput. Commun."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, S.M., Zhou, X., and Yang, L. (2011, January 16\u201318). Adaptive PID regulator based on neural network for DC motor speed control. Proceedings of the International Conference on Electrical and Control Engineering, Yichang, China.","DOI":"10.1109\/ICECENG.2011.6057865"},{"key":"ref_16","unstructured":"Ji, H., and Li, Z. (2011, January 10\u201311). Design of Neural Network PID Controller Based on Brushless DC Motor. Proceedings of the Second International Conference on Intelligent Computation Technology and Automation, Changsha, China."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Elsrogy, W.M., Fkirin, M.A., and Hassan, M.A.M. (2013, January 6\u20138). Speed Control of DC Motor Using PID Controller Based on Artificial Intelligence Techniques. Proceedings of the International Conference on Control, Decision and Information Technologies, Hammamet, Tunisia.","DOI":"10.1109\/CoDIT.2013.6689543"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.jprocont.2018.07.013","article-title":"Control of a bioreactor using a new partially supervised reinforcement learning algorithm","volume":"69","author":"Pandian","year":"2018","journal-title":"J. Process Control"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.isatra.2018.01.014","article-title":"Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning","volume":"73","author":"Radac","year":"2018","journal-title":"ISA Trans."},{"key":"ref_20","first-page":"343","article-title":"Tuning Methods of PID Controller for DC Motor Speed Control","volume":"3","author":"Abdulameer","year":"2016","journal-title":"Indones. J. Electr. Eng. Comput. Sci."},{"key":"ref_21","unstructured":"Bhagat, N.A., Bhaganagare, M., and Pandey, P.C. (2009). DC Motor Speed Control Using PID Controllers, EE Dept., IIT Bombay. EE 616 Electronic System Design Course Project."},{"key":"ref_22","unstructured":"Tsoukalas, L.H., and Uhring, R.E. (1996). Fuzzy and Neural Approaches in Engineering, John Wiley & Sons, Inc. Press. [1st ed.]."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0020-7373(75)80002-2","article-title":"An experiment in linguistic synthesis with a fuzzy logic controller","volume":"7","author":"Mamdani","year":"1975","journal-title":"Int. J. Man-Mach. Stud."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/TSMC.1985.6313399","article-title":"Fuzzy identification of systems and its applications to modeling and control","volume":"15","author":"Takagi","year":"1985","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_25","unstructured":"Russel, S., and Norving, P. (1995). Artificial Intelligence: A Modern Approach, Prentice Hall PTR."},{"key":"ref_26","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Reinforcement Signals. [Ph.D. Thesis, University of Cambridge]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sutton, R., and Barto, A. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_28","unstructured":"Fran\u00e7ois-Lavet, V., Fonteneau, R., and Ernst, D. (arXiv, 2015). How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies, arXiv."},{"key":"ref_29","unstructured":"Glorennec, P.Y., and Jouffe, L. (1997, January 5). Fuzzy Q-Learning. Proceedings of the 6th International Fuzzy Systems Conference, Barcelona, Spain."},{"key":"ref_30","unstructured":"Wang, L.X. (1992, January 8\u201312). Fuzzy Systems are Universal approximators. Proceedings of the First IEEE Conference on Fuzzy System, San Diego, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H. (2012). Reinforcement Learning in Continuous State and Action Spaces. Reinforcement Learning: State of the Art, Springer.","DOI":"10.1007\/978-3-642-27645-3_7"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1109\/21.370193","article-title":"Fuzzy logic controllers are universal approximators","volume":"25","author":"Castro","year":"1995","journal-title":"IEEE Trans. Syst. Man Cybern."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/10\/148\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:23:27Z","timestamp":1760196207000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/10\/148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,30]]},"references-count":32,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["a11100148"],"URL":"https:\/\/doi.org\/10.3390\/a11100148","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,30]]}}}