{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,11]],"date-time":"2026-01-11T01:33:38Z","timestamp":1768095218025,"version":"3.49.0"},"reference-count":27,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,16]],"date-time":"2025-03-16T00:00:00Z","timestamp":1742083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This paper investigates the design and MATLAB\/Simulink implementation of two intelligent neural reinforcement learning control algorithms based on deep learning neural network structures (RL DLNNs), for a complex Heating Ventilation Air Conditioning (HVAC) centrifugal chiller system (CCS). Our motivation to design such control strategies lies in this system\u2019s significant control-related challenges, namely its high dimensionality and strongly nonlinear multi-input multi-output (MIMO) structure, coupled with strong constraints and a substantial impact of measured disturbance on tracking performance. As a beneficial vehicle for \u201cproof of concept\u201d, two simplified CCS MIMO models were derived, and an extensive number of simulations were run to demonstrate the effectiveness of both RL DLNN control algorithm implementations compared with two conventional control algorithms. The experiments involving the two investigated data-driven advanced neural control algorithms prove their high potential to adapt to various types of nonlinearities, singularities, dimensions, disruptions, constraints, and uncertainties that inherently characterize real-world processes.<\/jats:p>","DOI":"10.3390\/a18030170","type":"journal-article","created":{"date-parts":[[2025,3,17]],"date-time":"2025-03-17T04:29:28Z","timestamp":1742185768000},"page":"170","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Investigations into the Design and Implementation of Reinforcement Learning Using Deep Learning Neural Networks"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2742-5119","authenticated-orcid":false,"given":"Roxana-Elena","family":"Tudoroiu","sequence":"first","affiliation":[{"name":"Sciences and Electrical Engineering, University from Petrosani, 332006 Petrosani, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammed","family":"Zaheeruddin","sequence":"additional","affiliation":[{"name":"Centre for Building Studies, Concordia University from Montreal, Montreal, QC H3G 1M8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6617-073X","authenticated-orcid":false,"given":"Daniel-Ioan","family":"Curiac","sequence":"additional","affiliation":[{"name":"Department of Automation and Applied Informatics, Polytechnic University of Timisoara, 300006 Timisoara, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mihai Sorin","family":"Radu","sequence":"additional","affiliation":[{"name":"Sciences and Electrical Engineering, University from Petrosani, 332006 Petrosani, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicolae","family":"Tudoroiu","sequence":"additional","affiliation":[{"name":"Engineering Technologies Department, John Abbott Coll\u00e8ge in Sainte-Anne-de-Bellevue, Montr\u00e9al, QC H9X 3L9, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,16]]},"reference":[{"key":"ref_1","unstructured":"Li, P., Li, Y., and Seem, J.E. (2010, January 12\u201315). Modelica Based Dynamic Modeling of Water-Cooled Centrifugal Chillers. Proceedings of the International Refrigeration and Air Conditioning Conference, West Lafayette, IN, USA. Available online: https:\/\/docs.lib.purdue.edu\/iracc\/1091."},{"key":"ref_2","first-page":"121","article-title":"Modeling Study of a Centrifugal Compressor","volume":"104","author":"Popovic","year":"1998","journal-title":"ASHRAE Trans."},{"key":"ref_3","unstructured":"Ning, M. (2008). Neural Network Based Optimal Control of HVAC&R Systems. [Ph.D. Thesis, Department of Building, Civil and Environmental Engineering, Concordia University]."},{"key":"ref_4","first-page":"445","article-title":"System Simulation of the performance of a Centrifugal chiller Using a Shell-and-Tube-Type Water-Cooled Condenser and R-11 as Refrigerant","volume":"95","author":"Wong","year":"1989","journal-title":"ASHRAE Trans."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1177\/014362440002100201","article-title":"Mechanistic Model of a Centrifugal Chiller to Study HVAC Dynamics","volume":"21","author":"Wang","year":"2000","journal-title":"Build. Serv. Eng. Res. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/S0140-7007(98)00003-6","article-title":"Steady-State Model of Centrifugal Liquid Chillers","volume":"21","author":"Browne","year":"1998","journal-title":"Int. J. Refrig."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1950031","DOI":"10.1142\/S2010132519500317","article-title":"A Model and Multi-Mode Control of a Centrifugal Chiller System: A Computer Simulation Study","volume":"27","author":"Li","year":"2019","journal-title":"Int. J. Air-Cond. Refrig."},{"key":"ref_8","unstructured":"Tsihrintzis, G., and Jain, L. (2020). The Implementation and the Design of a Hybrid digital PI Control Strategy Based on MISO Adaptive Neural Network Fuzzy Inference System Models\u2013A MIMO Centrifugal Chiller Case Study. Machine Learning Paradigms. Learning and Analytics in Intelligent Systems, Springer."},{"key":"ref_9","unstructured":"Klir, G.J., and Yuan, B. (1996). Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems-Selected Papers by Lotfi Zadeh. Advances in Fuzzy Systems\u2014Applications and Theory, State University of New York at Binghamton, USA."},{"key":"ref_10","unstructured":"Horton, M.J., Mars, D., Opaluch, W., and Disanno, S. (2009). Neural Networks and Learning Machines, Pearson Education, Inc.. [3rd ed.]."},{"key":"ref_11","unstructured":"Davis, G., and Quadrata, P.L. (1992). Introduction to Artificial Neural Systems, West Publishing Company. [1st ed.]."},{"key":"ref_12","unstructured":"Ljung, L. (2024, June 12). System Identification Toolbox Getting Started Guide. The MathWorks, Inc.,1 Apple Hill Drive Natick, MA 01760-2098 \u00a9 COPYRIGHT 1988\u20132023 by The MathWorks, Inc., MATLAB R2023a. Available online: https:\/\/control.dii.unisi.it\/sysid\/book\/ident_gs.pdf."},{"key":"ref_13","first-page":"32","article-title":"ANFIS: Adaptive Neuro-Fuzzy Inference System-A Survey","volume":"123","author":"Walia","year":"2015","journal-title":"Int. J. Comput. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1109\/21.256541","article-title":"ANFIS: Adaptive-Network-Based Fuzzy Inference System","volume":"23","author":"Jang","year":"1993","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Almabrok, A., Psarakis, M., and Dounis, A. (2018). Fast Tuning of the PID Controller in An HVAC System Using the Big Bang\u2013Big Crunch Algorithm and FPGA Technology. Algorithms, 11.","DOI":"10.3390\/a11100146"},{"key":"ref_16","unstructured":"MathWorks Support Documentation-Help Center (2024, June 12). Estimate ARMAX or ARMA Model. Available online: https:\/\/www.mathworks.com\/help\/ident\/ref\/armax.html."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Namdari, A., Samani, M.A., and Durrani, T.S. (2022). Lithium-Ion Battery Prognostics through Reinforcement Learning Based on Entropy Measures. Algorithms, 15.","DOI":"10.3390\/a15110393"},{"key":"ref_18","unstructured":"MathWorks Support Documentation-Help Center (2024, September 10). Reinforcement Learning Using Deep Neural Networks. Available online: https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/reinforcement-learning-using-deep-neural-networks.html."},{"key":"ref_19","unstructured":"MathWorks Support Documentation-Help Center (2024, September 10). Reinforcement Learning for Control Systems Applications. Available online: https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/reinforcement-learning-for-control-systems-applications.html."},{"key":"ref_20","unstructured":"MathWorks Support Documentation-Help Center (2024, September 10). Generate Reward Function from a Model Predictive Controller for a Servomotor. Available online: https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/generate-reward-fcn-from-mpc-for-servomotor.html."},{"key":"ref_21","unstructured":"MathWorks Support Documentation-Help Center (2024, September 20). Generate Reward Function from a Model Verification Block for a Water Tank System. Available online: https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/generate-reward-fcn-from-verification-block-for-watertank.html."},{"key":"ref_22","unstructured":"Uzunovic, T., Zunic, E., Badnjevic, A., Miokovic, I., and Konjicija, S. (2010, January 24\u201328). Implementation of digital PID controller. Proceedings of the IEEE 33rd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia."},{"key":"ref_23","unstructured":"Grosse, G., Maddison, C., Bae, J., Pitis, S., and Introduction to Machine Learning (2025, February 23). Lecture 11-Reinforcement Learning, University of Toronto, Fall. Available online: https:\/\/www.cs.toronto.edu\/~rgrosse\/courses\/csc311_f20\/slides\/lec11.pdf."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Abdel-Jaber, H., Devassy, D., Al Salam, A., Hidaytallah, L., and El-Amir, M. (2022). A Review of Deep Learning Algorithms and Their Applications in Healthcare. Algorithms, 15.","DOI":"10.3390\/a15020071"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kranthi Kumar, P., and Detroja, K.P. (2023, January 13\u201316). Design of Reinforcement Learning based PI controller for nonlinear Multivariable System. Proceedings of the 2023 European Control Conference (ECC), Bucharest, Romania.","DOI":"10.23919\/ECC57647.2023.10178182"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"17953","DOI":"10.1007\/s00521-023-08667-x","article-title":"Deep reinforcement learning-based proportional\u2013integral control for dual-active-bridge converter","volume":"35","author":"You","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_27","unstructured":"MathWorks Support Documentation-Help Center (2024, September 20). Resolving Low-Level Graphics Issues. Available online: https:\/\/www.mathworks.com\/help\/matlab\/creating-plots\/resolving-low-level-graphics-issues.html."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/3\/170\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:54:37Z","timestamp":1760028877000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/3\/170"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,16]]},"references-count":27,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["a18030170"],"URL":"https:\/\/doi.org\/10.3390\/a18030170","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,16]]}}}