{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T22:04:09Z","timestamp":1777327449980,"version":"3.51.4"},"reference-count":23,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,4,17]],"date-time":"2021-04-17T00:00:00Z","timestamp":1618617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["769288"],"award-info":[{"award-number":["769288"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Aerospace"],"abstract":"<jats:p>This paper presents a Reinforcement Learning (RL) approach to optimize the long-term scheduling of maintenance for an aircraft fleet. The problem considers fleet status, maintenance capacity, and other maintenance constraints to schedule hangar checks for a specified time horizon. The checks are scheduled within an interval, and the goal is to, schedule them as close as possible to their due date. In doing so, the number of checks is reduced, and the fleet availability increases. A Deep Q-learning algorithm is used to optimize the scheduling policy. The model is validated in a real scenario using maintenance data from 45 aircraft. The maintenance plan that is generated with our approach is compared with a previous study, which presented a Dynamic Programming (DP) based approach and airline estimations for the same period. The results show a reduction in the number of checks scheduled, which indicates the potential of RL in solving this problem. The adaptability of RL is also tested by introducing small disturbances in the initial conditions. After training the model with these simulated scenarios, the results show the robustness of the RL approach and its ability to generate efficient maintenance plans in only a few seconds.<\/jats:p>","DOI":"10.3390\/aerospace8040113","type":"journal-article","created":{"date-parts":[[2021,4,18]],"date-time":"2021-04-18T22:15:13Z","timestamp":1618784113000},"page":"113","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Aircraft Maintenance Check Scheduling Using Reinforcement Learning"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7019-7721","authenticated-orcid":false,"given":"Pedro","family":"Andrade","sequence":"first","affiliation":[{"name":"Department of Informatics Engineering, University of Coimbra, CISUC, 3030-290 Coimbra, Portugal"}]},{"given":"Catarina","family":"Silva","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering, University of Coimbra, CISUC, 3030-290 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9770-7672","authenticated-orcid":false,"given":"Bernardete","family":"Ribeiro","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering, University of Coimbra, CISUC, 3030-290 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9429-6401","authenticated-orcid":false,"given":"Bruno F.","family":"Santos","sequence":"additional","affiliation":[{"name":"Air Transport and Operations, Faculty of Aerospace Engineering, Delft University of Technology, 2629 HS Delft, The Netherlands"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,17]]},"reference":[{"key":"ref_1","unstructured":"Transport Canada (2020, December 07). Canadian Aviation Regulations, Part I\u2014General Provisions, Subpart 1\u2014Interpretation, Available online: https:\/\/web.archive.org\/web\/20121227092905\/http:\/\/www.tc.gc.ca\/eng\/civilaviation\/regserv\/cars\/part1-subpart1-1104.htm."},{"key":"ref_2","unstructured":"Ackert, S.P. (2020, December 12). Basics of Aircraft Maintenance Programs for Financiers: Evaluation & Insights of Commercial Aircraft Maintenance Programs. Available online: http:\/\/www.aircraftmonitor.com\/uploads\/1\/5\/9\/9\/15993320\/basics_of_aircraft_maintenance_programs_for_financiers___v1.pdf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1016\/j.ejor.2019.08.025","article-title":"A practical dynamic programming based methodology for aircraft maintenance check scheduling optimization","volume":"281","author":"Deng","year":"2019","journal-title":"Eur. J. Oper. Res."},{"key":"ref_4","first-page":"177","article-title":"Analysis on Costs for Aircraft Maintenance","volume":"3","author":"PeriyarSelvam","year":"2013","journal-title":"Adv. Aerosp. Sci. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Boere, N.J. (1977). Air Canada Saves with Aircraft Maintenance Scheduling. Interfaces, 7.","DOI":"10.1287\/inte.7.3.1"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chaovalitwongse, W., Furman, K.C., and Pardalos, P. (2009). The Aircraft Maintenance Routing Problem. Optimization and Logistics Challenges in the Enterprise, Springer.","DOI":"10.1007\/978-0-387-88617-6"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1415","DOI":"10.1287\/mnsc.35.12.1415","article-title":"Flight Scheduling and Maintenance Base Planning","volume":"35","author":"Feo","year":"1989","journal-title":"Manage. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1287\/opre.46.2.260","article-title":"The Aircraft Maintenance Routing Problem","volume":"46","author":"Gopalan","year":"1998","journal-title":"Oper. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/S0969-6997(00)00011-9","article-title":"A dynamic approach for aircraft assignment and maintenance scheduling by airlines","volume":"6","author":"Moudani","year":"2000","journal-title":"J. Air. Transp. Manag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1016\/j.ejor.2013.10.066","article-title":"Operational aircraft maintenance routing problem with remaining time consideration","volume":"235","author":"Bilge","year":"2014","journal-title":"Eur. J. Oper. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1850","DOI":"10.1016\/j.ejor.2004.10.033","article-title":"A branch-and-price approach for operational aircraft maintenance routing","volume":"175","author":"Sarac","year":"2006","journal-title":"Eur. J. Oper. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1108\/13552510910997788","article-title":"Building flight planning for an airline company under maintenance constraints","volume":"15","author":"Afsar","year":"2009","journal-title":"J. Qual. Maint. Eng."},{"key":"ref_13","first-page":"128346:1","article-title":"Multiobjective Optimization of Aircraft Maintenance in Thailand Using Goal Programming: A Decision-Support Model","volume":"2012","author":"Pleumpirom","year":"2012","journal-title":"Adv. Decis. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1287\/trsc.2020.0984","article-title":"Dynamic Optimization for Airline Maintenance Operations","volume":"54","author":"Lagos","year":"2020","journal-title":"Transp. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mattila, V., and Virtanen, K. (2011, January 11\u201314). Scheduling fighter aircraft maintenance with reinforcement learning. Proceedings of the 2011 Winter Simulation Conference (WSC), Phoenix, AZ, USA.","DOI":"10.1109\/WSC.2011.6147962"},{"key":"ref_16","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. [2nd ed.]."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_19","unstructured":"van Hasselt, H. (2010, January 6\u20139). Double Q-Learning. Proceedings of the 24th Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_21","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_22","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_23","unstructured":"Deng, Q., Santos, B.F., and Curran, R. (2020, December 28). Aircraft Maintenance Check Scheduling Dataset. Available online: https:\/\/doi.org\/10.4121\/uuid:1630e6fd-9574-46e8-899e-83037c17bcef."}],"container-title":["Aerospace"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2226-4310\/8\/4\/113\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:49:07Z","timestamp":1760161747000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2226-4310\/8\/4\/113"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,17]]},"references-count":23,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["aerospace8040113"],"URL":"https:\/\/doi.org\/10.3390\/aerospace8040113","relation":{},"ISSN":["2226-4310"],"issn-type":[{"value":"2226-4310","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,17]]}}}