{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T05:25:49Z","timestamp":1765430749892,"version":"3.46.0"},"reference-count":20,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2025,12,9]],"date-time":"2025-12-09T00:00:00Z","timestamp":1765238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>In this paper, we describe the procedure of implementing a reinforcement learning algorithm, TD3, to learn the performance of a cooling water pump and how this type of learning can be used to detect degradations and evaluate its health condition. These types of machine learning algorithms have not been used extensively in the scientific literature to monitor the degradation of industrial components, so this study attempts to fill this gap, presenting the main characteristics of these algorithms\u2019 application in a real case. The method presented consists of several models for predicting the expected evolution of significant behavior variables when no anomalies exist, showing the performance of different aspects of the pump. Examples of these variables are bearing temperatures or vibrations in different pump locations. All of the data used in this paper come from the SCADA system of the power plant where the cooling water pump is located.<\/jats:p>","DOI":"10.3390\/computers14120540","type":"journal-article","created":{"date-parts":[[2025,12,9]],"date-time":"2025-12-09T15:50:02Z","timestamp":1765295402000},"page":"540","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["TD3 Reinforcement Learning Algorithm Used for Health Condition Monitoring of a Cooling Water Pump"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5192-8587","authenticated-orcid":false,"given":"Miguel A.","family":"Sanz-Bobi","sequence":"first","affiliation":[{"name":"Institute for Research in Technology, ICAI School of Engineering, Pontifical Comillas University, Rey Francisco 4, 28015 Madrid, Spain"}]},{"given":"In\u00e9s","family":"Rodriguez","sequence":"additional","affiliation":[{"name":"Institute for Research in Technology, ICAI School of Engineering, Pontifical Comillas University, Rey Francisco 4, 28015 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8045-8323","authenticated-orcid":false,"given":"F. Javier","family":"Bellido-L\u00f3pez","sequence":"additional","affiliation":[{"name":"Institute for Research in Technology, ICAI School of Engineering, Pontifical Comillas University, Rey Francisco 4, 28015 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8844-441X","authenticated-orcid":false,"given":"Antonio","family":"Mu\u00f1oz","sequence":"additional","affiliation":[{"name":"Institute for Research in Technology, ICAI School of Engineering, Pontifical Comillas University, Rey Francisco 4, 28015 Madrid, Spain"}]},{"given":"Javier","family":"Anguera","sequence":"additional","affiliation":[{"name":"Enel Green Power and Thermal Generation, Endesa\u2014Gas Maintenance Iberia, Ribera del Loira 60, 28015 Madrid, Spain"}]},{"given":"Daniel","family":"Gonzalez-Calvo","sequence":"additional","affiliation":[{"name":"Enel Green Power and Thermal Generation, Endesa\u2014Gas Maintenance Iberia, Ribera del Loira 60, 28015 Madrid, Spain"}]},{"given":"Tomas","family":"Alvarez-Tejedor","sequence":"additional","affiliation":[{"name":"Enel Green Power and Thermal Generation, Endesa\u2014Gas Maintenance Iberia, Ribera del Loira 60, 28015 Madrid, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,9]]},"reference":[{"key":"ref_1","first-page":"395","article-title":"A Review of Machine Learning Tools and Techniques for Anomaly Detection","volume":"361","author":"Choudrie","year":"2023","journal-title":"ICT for Intelligent Systems. ICTIS 2023. Smart Innovation, Systems and Technologies"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439950","article-title":"Deep Learning for Anomaly Detection: A Review","volume":"54","author":"Pang","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"108119","DOI":"10.1016\/j.ress.2021.108119","article-title":"Prognostics and health management (PHM): Where are we and where do we (need to) go in theory and practice","volume":"218","author":"Zio","year":"2022","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"106671","DOI":"10.1016\/j.rineng.2025.106671","article-title":"Comprehensive review of shipboard maintenance management strategies","volume":"27","author":"Park","year":"2025","journal-title":"Results Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Calabrese, F., Regattieri, A., Bortolini, M., and Galizia, F.G. (2022). Data-driven fault detection and diagnosis: Challenges and opportunities in real-world scenarios. Appl. Sci., 12.","DOI":"10.3390\/app12189212"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105809","DOI":"10.1016\/j.rineng.2025.105809","article-title":"A novel method for evaluation of the maintenance impact in the health of industrial components","volume":"27","year":"2025","journal-title":"Results Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Supramaniam, A., Syed Ahmad, S.S., and Mohd Yusoh, Z.Y. (2024, January 26\u201328). Predictive Maintenance using Deep Reinforcement Learning. Proceedings of the IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), Kota Kinabalu, Malaysia.","DOI":"10.1109\/IICAIET62352.2024.10730350"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"108908","DOI":"10.1016\/j.ress.2022.108908","article-title":"Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics","volume":"230","author":"Lee","year":"2023","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Senthil, C., and Sudhakara Pandian, R. (2022). Proactive Maintenance Model Using Reinforcement Learning Algorithm in Rubber Industry. Processes, 10.","DOI":"10.3390\/pr10020371"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"102406","DOI":"10.1016\/j.rcim.2022.102406","article-title":"Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines","volume":"78","author":"Kubler","year":"2022","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"124431","DOI":"10.1016\/j.apenergy.2024.124431","article-title":"A deep reinforcement learning ensemble for maintenance scheduling in offshore wind farms","volume":"377","author":"Lee","year":"2025","journal-title":"Appl. Energy"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hao, Z., Di Maio, F., Pinciroli, L., and Zio, E. (September, January 28). Optimal Prescriptive Maintenance of Nuclear Power Plants by Deep Reinforcement Learning. Proceedings of the 32nd European Safety and Reliability Conference, ESREL 2022\u2014Understanding and Managing Risk and Reliability for a Sustainable Future, Dublin, Ireland.","DOI":"10.3850\/978-981-18-5183-4_S22-02-466-cd"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bowman, C.F., and Bowman, S.N. (2021). Engineering of Power Plant and Industrial Cooling Water Systems, CRC Press. [1st ed.].","DOI":"10.1201\/9781003172437"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"106885","DOI":"10.1016\/j.ress.2020.106885","article-title":"Failure mode and effect analysis improvement: A systematic literature review and future research agenda","volume":"199","author":"Huang","year":"2020","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_15","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press."},{"key":"ref_16","unstructured":"Fujimoto, S., van Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the 35th International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Zhang, L., Ma, Q., and Xue, J. (2023, January 17\u201319). The LSTM-PER-TD3 Algorithm for Deep Reinforcement Learning in Continuous Control Tasks. Proceedings of the 2023 China Automation Congress (CAC), Chongqing, China.","DOI":"10.1109\/CAC59555.2023.10450268"},{"key":"ref_18","unstructured":"(2025, July 22). Gymnasium. Available online: https:\/\/gymnasium.farama.org\/."},{"key":"ref_19","first-page":"12348","article-title":"Stable-baselines3: Reliable reinforcement learning implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bellido-Lopez, F.J., Sanz-Bobi, M.A., Mu\u00f1oz, A., Gonzalez-Calvo, D., and Alvarez-Tejedor, T. (2025). Maintenance-Aware Risk Curves: Correcting Degradation Models with Intervention Effectiveness. Appl. Sci., 15.","DOI":"10.3390\/app152010998"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/12\/540\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T05:22:58Z","timestamp":1765430578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/12\/540"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,9]]},"references-count":20,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["computers14120540"],"URL":"https:\/\/doi.org\/10.3390\/computers14120540","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2025,12,9]]}}}