{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T15:57:55Z","timestamp":1781971075596,"version":"3.54.5"},"reference-count":45,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,10,7]],"date-time":"2020-10-07T00:00:00Z","timestamp":1602028800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002850","name":"Fondo Nacional de Desarrollo Cient\u00edfico y Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["1190720"],"award-info":[{"award-number":["1190720"]}],"id":[{"id":"10.13039\/501100002850","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Gas pipeline systems are one of the largest energy infrastructures in the world and are known to be very efficient and reliable. However, this does not mean they are prone to no risk. Corrosion is a significant problem in gas pipelines that imposes large risks such as ruptures and leakage to the environment and the pipeline system. Therefore, various maintenance actions are performed routinely to ensure the integrity of the pipelines. The costs of the corrosion-related maintenance actions are a significant portion of the pipeline\u2019s operation and maintenance costs, and minimizing this large cost is a highly compelling subject that has been addressed by many studies. In this paper, we investigate the benefits of applying reinforcement learning (RL) techniques to the corrosion-related maintenance management of dry gas pipelines. We first address the rising need for a simulated testbed by proposing a test bench that models corrosion degradation while interacting with the maintenance decision-maker within the RL environment. Second, we propose a condition-based maintenance management approach that leverages a data-driven RL decision-making methodology. An RL maintenance scheduler is applied to the proposed test bench, and the results show that applying the proposed condition-based maintenance management technique can reduce up to 58% of the maintenance costs compared to a periodic maintenance policy while securing pipeline reliability.<\/jats:p>","DOI":"10.3390\/s20195708","type":"journal-article","created":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T08:52:41Z","timestamp":1602147161000},"page":"5708","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":43,"title":["Condition-Based Maintenance with Reinforcement Learning for Dry Gas Pipeline Subject to Internal Corrosion"],"prefix":"10.3390","volume":"20","author":[{"given":"Zahra","family":"Mahmoodzadeh","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering and the B. John Garrick Institute for the Risk Sciences, University of California, Los Angeles\u2013UCLA, Los Angeles, CA 90095, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keo-Yuan","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Materials Science and Engineering and the B. John Garrick Institute for the Risk Sciences, University of California, Los Angeles\u2013UCLA, Los Angeles, CA 90095, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enrique","family":"Lopez Droguett","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Chile, Santiago 8320000, Chile"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Mosleh","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering and the B. John Garrick Institute for the Risk Sciences, University of California, Los Angeles\u2013UCLA, Los Angeles, CA 90095, USA"},{"name":"Department of Materials Science and Engineering and the B. John Garrick Institute for the Risk Sciences, University of California, Los Angeles\u2013UCLA, Los Angeles, CA 90095, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.ress.2016.11.014","article-title":"Corrosion induced failure analysis of subsea pipelines","volume":"159","author":"Yang","year":"2017","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.ijpvp.2010.04.003","article-title":"Review of pipeline integrity management practices","volume":"87","author":"Kishawy","year":"2010","journal-title":"Int. J. Press. Vessel. Pip."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2019\/5415828","article-title":"Next Generation Data Infrastructures: Towards an Extendable Model of the Asset Management Data Infrastructure as Complex Adaptive System","volume":"2019","author":"Brous","year":"2019","journal-title":"Complexity"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lv, Z., Ota, K., Lloret, J., Xiang, W., and Bellavista, P. (2019). Complexity Problems Handled by Big Data Technology. Complexity, 2019.","DOI":"10.1155\/2019\/9090528"},{"key":"ref_5","unstructured":"Szepesvari, C. (2010). Algorithms for Reinforcement Learning, Morgan & Claypool Publishers."},{"key":"ref_6","unstructured":"Sutton, R., and Barto, A. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1109\/ACCESS.2017.2771827","article-title":"Reinforcement learning-based and parametric production-maintenance control policies for a deteriorating manufacturing system","volume":"6","author":"Xanthopoulos","year":"2017","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wei, M., and Qi, C. (2012). Reinforcement Learning Based Maintenance Scheduling for a Two-Machine Flow Line with Deteriorating Quality States. 2012 Third Global Congress on Intelligent Systems, IEEE Computer Society.","DOI":"10.1109\/GCIS.2012.82"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1016\/j.engappai.2009.01.014","article-title":"Dynamic scheduling of maintenance tasks in the petroleum industry: A reinforcement approach","volume":"22","author":"Aissani","year":"2009","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1007\/s10845-016-1237-7","article-title":"Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks","volume":"30","author":"Barde","year":"2019","journal-title":"J. Intell. Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2981","DOI":"10.1007\/s00170-018-2690-6","article-title":"Reinforcement learning-based flow management of gas turbine parts under stochastic failures","volume":"99","author":"Compare","year":"2018","journal-title":"Int. J. Adv. Manuf. Tech."},{"key":"ref_12","first-page":"15","article-title":"Towards Developing a Novel Framework for Practical PHM: A Sequential Decision Problem solved by Reinforcement Learning and Artificial Neural Networks","volume":"10","author":"Bellani","year":"2019","journal-title":"Int. J. Progn. Health Manag."},{"key":"ref_13","unstructured":"Koch, G.H., Brongers, M.P.H., Thompson, N.G., Virmani, Y.P., and Payer, J.H. (2002). Corrosion Cost and Preventive Strategies in the United States, NACE International."},{"key":"ref_14","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv, Available online: http:\/\/arxiv.org\/abs\/1606.01540."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1613\/jair.3912","article-title":"The Arcade Learning Environment: An Evaluation Platform for General Agents","volume":"47","author":"Bellemare","year":"2013","journal-title":"J. Artif. Intell. Res."},{"key":"ref_16","unstructured":"Wang, Z., Bapst, V., Heess, N., Mnih, V., Munos, R., Kavukcuoglu, K., de Freitas, N., and Sample Efficient Actor-Critic with Experience Replay (2020, April 02). Presented at the ICLR 2017, Toulon, France, 24\u201326 April 2017. Available online: http:\/\/arxiv.org\/abs\/1611.01224."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.ejor.2019.06.048","article-title":"Maintenance policy for a system with a weighted linear combination of degradation processes","volume":"280","author":"Wu","year":"2020","journal-title":"Eur. J. Oper. Res."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gurjar, S.S. (2019). Line Inspection ILI Interval for Cross Country Pipelines, Society of Petroleum Engineers.","DOI":"10.2118\/194617-MS"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.ijpvp.2013.06.002","article-title":"System reliability of corroding pipelines considering stochastic process-based models for defect growth and internal pressure","volume":"111","author":"Zhang","year":"2013","journal-title":"Int. J. Press. Vessel. Pip."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102930","DOI":"10.1016\/j.jngse.2019.102930","article-title":"Effect of temporal variability of operating parameters in corrosion modelling for natural gas pipelines subject to uniform corrosion","volume":"69","author":"Wu","year":"2019","journal-title":"J. Nat. Gas Sci. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kale, A., Thacker, B.H., Sridhar, N., and Waldhart, C.J. (2004, January 4\u20138). A probabilistic model for internal corrosion of gas pipelines. Proceedings of IPC 2004 International Pipeline Conference, Calgary, AL, Canada.","DOI":"10.1115\/IPC2004-0483"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/0141-0296(95)92637-N","article-title":"Probabilistic analysis of pipelines subjected to pitting corrosion leaks","volume":"17","author":"Ahammed","year":"1995","journal-title":"Eng. Struct."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"35006","DOI":"10.5006\/1.3360912","article-title":"Model to predict internal pitting corrosion of oil and gas pipelines","volume":"66","author":"Papavinasam","year":"2010","journal-title":"Corrosion"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Modiri, B., Mohammad, M.P., Yazdani, M., Nasirpouri, F., and Salehpour, F. (2014, January 14\u201320). Piping Anti-Corrosion Coating Life Assessment. Proceedings of the ASME 2014 International Mechanical Engineering Congress and Exposition, Montreal, QC, Canada.","DOI":"10.1115\/IMECE2014-36423"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.ijpvp.2010.07.011","article-title":"System reliability of corroding pipelines","volume":"87","author":"Zhou","year":"2010","journal-title":"Int. J. Press. Vessel. Pip."},{"key":"ref_26","unstructured":"ASME B31G, A. S. M. E (1991). Manual for determining the remaining strength of corroded pipelines. ASME B31G-1991."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.ijpvp.2011.09.005","article-title":"Probability assessment of burst limit state due to internal corrosion","volume":"89","author":"Hasan","year":"2012","journal-title":"Int. J. Press. Vessel. Pip."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1108\/00035590510574862","article-title":"Pipeline corrosion risk analysis\u2014An assessment of deterministic and probabilistic methods","volume":"52","author":"Lawson","year":"2005","journal-title":"Anti Corros. Methods Mater."},{"key":"ref_29","unstructured":"(2018). Mitigation of Internal Corrosion in Carbon Steel Gas Pipeline Systems, CAPP."},{"key":"ref_30","unstructured":"Singh, A. (2017). Internal Coating\u2014A Must in Gas Pipelines, Indian Oil Corporation."},{"key":"ref_31","unstructured":"Palmer-Jones, R., and Paisley, D. (, January September). Repairing Internal Corrosion Defects in Pipelines. Proceedings of the 4th International Pipeline Rehabilitation and Maintenance Conference, Prague, Czech Republic."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/S0951-8320(03)00173-X","article-title":"Classes of imperfect repair models based on reduction of failure intensity or virtual age","volume":"84","author":"Doyen","year":"2004","journal-title":"Reliab. Eng. Syst. Safety"},{"key":"ref_33","unstructured":"(2015). Corrosion in the Petrochemical Industry, ASM International. [2nd ed.]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"70832","DOI":"10.1039\/C5RA14406J","article-title":"Corrosion inhibition effect of spiropyrimidinethiones on mild steel in 15% HCl solution: Insight from electrochemical and quantum studies","volume":"5","author":"Yadav","year":"2015","journal-title":"RSC Adv."},{"key":"ref_35","unstructured":"Mitchell, C. (2020, April 02). WOOO-PIG-SOOIE!\u2014The Business Of Pipeline Integrity II. Available online: https:\/\/rbnenergy.com\/wooo-pig-sooie-the-business-of-pipeline-integrity-ii."},{"key":"ref_36","unstructured":"Larsen, K.R. (2020, April 02). Protecting a Pipeline When Its Coating Has AgedTitle. Available online: http:\/\/www.materialsperformance.com\/articles\/coating-linings\/2017\/03\/protecting-a-pipeline-when-its-coating-has-aged."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.cie.2012.02.002","article-title":"An overview of time-based and condition-based maintenance in industrial application","volume":"63","author":"Ahmad","year":"2012","journal-title":"Comp. Ind. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1016\/j.ress.2010.02.016","article-title":"Development of an optimized condition-based maintenance system by data fusion and reliability-centered maintenance","volume":"95","author":"Niu","year":"2010","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2018\/4695890","article-title":"Multiobject Tracking in Videos Based on LSTM and Deep Reinforcement Learning","volume":"2018","author":"Jiang","year":"2018","journal-title":"Complexity"},{"key":"ref_40","unstructured":"Ng, A.Y., and Russell, S.J. Algorithms for Inverse Reinforcement Learning, University of California."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"106483","DOI":"10.1016\/j.ress.2019.04.036","article-title":"Managing engineering systems with large state and action spaces through deep reinforcement learning","volume":"191","author":"Andriotis","year":"2019","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_42","first-page":"279","article-title":"Q-learning Machine","volume":"8","author":"Watkins","year":"1992","journal-title":"Learning"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3294","DOI":"10.1002\/sim.3720","article-title":"Reinforcement learning design for cancer clinical trials","volume":"28","author":"Zhao","year":"2009","journal-title":"Stat. Med."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Nevmyvaka, Y., Feng, Y., and Kearns, M. (2006, January 25\u201329). Reinforcement learning for optimized trade execution. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143929"},{"key":"ref_45","unstructured":"Leibo, J.Z., Zambaldi, V., Lanctot, M., Marecki, J., and Graepel, T. (2017, January 8\u201312). Multi-agent reinforcement learning in sequential social dilemmas. Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, Sao Paulo, Brazil."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5708\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:17:15Z","timestamp":1760177835000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5708"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,7]]},"references-count":45,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195708"],"URL":"https:\/\/doi.org\/10.3390\/s20195708","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,7]]}}}