{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T01:51:56Z","timestamp":1778637116818,"version":"3.51.4"},"reference-count":75,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2022,3,15]],"date-time":"2022-03-15T00:00:00Z","timestamp":1647302400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,15]],"date-time":"2022-03-15T00:00:00Z","timestamp":1647302400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001773","name":"University of New South Wales","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001773","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ann Oper Res"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm (<jats:inline-formula><jats:alternatives><jats:tex-math>$$\\mathcal {DRLSA}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>DRLSA<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\mathcal {DRLSA}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>DRLSA<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\mathcal {DRLSA}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>DRLSA<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.<\/jats:p>","DOI":"10.1007\/s10479-022-04612-8","type":"journal-article","created":{"date-parts":[[2022,3,15]],"date-time":"2022-03-15T12:03:53Z","timestamp":1647345833000},"page":"79-110","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem"],"prefix":"10.1007","volume":"339","author":[{"given":"Fuat","family":"Kosanoglu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mahir","family":"Atmis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1147-2436","authenticated-orcid":false,"given":"Hasan H\u00fcseyin","family":"Turan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,15]]},"reference":[{"key":"4612_CR1","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1016\/j.cie.2018.09.051","volume":"126","author":"TT Allen","year":"2018","unstructured":"Allen, T. T., Roychowdhury, S., & Liu, E. (2018). Reward-based Monte Carlo-Bayesian reinforcement learning for cyber preventive maintenance. Computers & Industrial Engineering, 126, 578\u2013594.","journal-title":"Computers & Industrial Engineering"},{"key":"4612_CR2","doi-asserted-by":"publisher","first-page":"106483","DOI":"10.1016\/j.ress.2019.04.036","volume":"191","author":"C Andriotis","year":"2019","unstructured":"Andriotis, C., & Papakonstantinou, K. (2019). Managing engineering systems with large state and action spaces through deep reinforcement learning. Reliability Engineering & System Safety, 191, 106483.","journal-title":"Reliability Engineering & System Safety"},{"key":"4612_CR3","doi-asserted-by":"crossref","unstructured":"Andriotis, C.\u00a0P., & Papakonstantinou, K.\u00a0G. (2018). Managing engineering systems with large state and action spaces through deep reinforcement learning. CoRR, arXiv:1811.02052.","DOI":"10.1016\/j.ress.2019.04.036"},{"key":"4612_CR4","unstructured":"Arsenault, R. (2016). Stat of the week: The (rising!) cost of downtime. https:\/\/www.aberdeen.com\/techpro-essentials\/stat-of-the-week-the-rising-cost-of-downtime\/. Accessed: 2021-03-07."},{"key":"4612_CR5","unstructured":"Bello, I., Pham, H., Le, Q.\u00a0V., Norouzi, M., & Bengio, S. (2017). Neural combinatorial optimization with reinforcement learning. arXiv:1611.09940."},{"key":"4612_CR6","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Lodi, A., & Prouvost, A. (2020). Machine learning for combinatorial optimization: a methodological tour d\u2019horizon. arXiv:1811.06128.","DOI":"10.1016\/j.ejor.2020.07.063"},{"key":"4612_CR7","unstructured":"Chen, W., Xu, Y., & Wu, X. (2017). Deep reinforcement learning for multi-resource multi-machine job scheduling. arXiv preprint arXiv:1711.07440, ."},{"key":"4612_CR8","unstructured":"Chen, X., & Tian, Y. (2019). Learning to perform local rewriting for combinatorial optimization. arXiv:1810.00337."},{"key":"4612_CR9","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/0377-2217(90)90301-Q","volume":"46","author":"DT Connolly","year":"1990","unstructured":"Connolly, D. T. (1990). An improved annealing scheme for the QAP. European Journal of Operational Research, 46, 93\u2013100.","journal-title":"European Journal of Operational Research"},{"key":"4612_CR10","doi-asserted-by":"crossref","unstructured":"Deudon, M., Cournut, P., Lacoste, A., Adulyasak, Y., & Rousseau, L.-M. (2018). Learning heuristics for the TSP by policy gradient. In International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research (pp. 170\u2013181). Springer.","DOI":"10.1007\/978-3-319-93031-2_12"},{"key":"4612_CR11","doi-asserted-by":"publisher","unstructured":"Du, K.-L., & Swamy, M. N. S. (2016). Simulated annealing. Search and Optimization by Metaheuristics: Techniques and Algorithms Inspired by Nature (pp. 29\u201336). Cham: Springer International Publishing. https:\/\/doi.org\/10.1007\/978-3-319-41192-7_2","DOI":"10.1007\/978-3-319-41192-7_2"},{"key":"4612_CR12","unstructured":"Duan, L., Hu, H., Qian, Y., Gong, Y., Zhang, X., Xu, Y., & Wei, J. (2019). A multi-task selected learning approach for solving 3D flexible bin packing problem. arXiv:1804.06896."},{"key":"4612_CR13","unstructured":"Emami, P., & Ranka, S. (2018). Learning permutations with sinkhorn policy gradient. arXiv:1805.07010."},{"key":"4612_CR14","doi-asserted-by":"crossref","unstructured":"Etheve, M., Al\u00e8s, Z., Bissuel, C., Juan, O., & Kedad-Sidhoum, S. (2020). Reinforcement learning for variable selection in a branch and bound algorithm. Lecture Notes in Computer Science, (p. 176\u2013185).","DOI":"10.1007\/978-3-030-58942-4_12"},{"key":"4612_CR15","first-page":"219","volume":"11","author":"V Fran\u00e7ois-Lavet","year":"2018","unstructured":"Fran\u00e7ois-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018). An introduction to deep reinforcement learning. Foundations and Trends&#x00AE. Machine Learning, 11, 219\u2013354.","journal-title":"Machine Learning"},{"key":"4612_CR16","doi-asserted-by":"crossref","unstructured":"Gama, R., & Fernandes, H.\u00a0L. (2020). A reinforcement learning approach to the orienteering problem with time windows. arXiv:2011.03647.","DOI":"10.1016\/j.cor.2021.105357"},{"key":"4612_CR17","unstructured":"Hicks, G. (2019). How much is equipment downtime costing your workplace? https:\/\/www.iofficecorp.com\/blog\/equipment-downtime. Accessed: 2021-03-07."},{"key":"4612_CR18","doi-asserted-by":"publisher","unstructured":"Hoong Ong, K.\u00a0S., Niyato, D., & Yuen, C. (2020). Predictive maintenance for edge-based sensor networks: A deep reinforcement learning approach. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT) (pp. 1\u20136). https:\/\/doi.org\/10.1109\/WF-IoT48130.2020.9221098.","DOI":"10.1109\/WF-IoT48130.2020.9221098"},{"key":"4612_CR19","doi-asserted-by":"publisher","unstructured":"Hottung, A., Tanaka, S., & Tierney, K. (2020). Deep learning assisted heuristic tree search for the container pre-marshalling problem. Computers & Operations Research, 113, 104781. https:\/\/doi.org\/10.1016\/j.cor.2019.104781http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0305054819302230.","DOI":"10.1016\/j.cor.2019.104781"},{"key":"4612_CR20","unstructured":"Hu, H., Zhang, X., Yan, X., Wang, L., & Xu, Y. (2017). Solving a new 3D bin packing problem with deep reinforcement learning method. arXiv:1708.05930."},{"key":"4612_CR21","doi-asserted-by":"publisher","first-page":"14413","DOI":"10.1109\/TVT.2020.3034800","volume":"69","author":"J Hu","year":"2020","unstructured":"Hu, J., Niu, H., Carrasco, J., Lennox, B., & Arvin, F. (2020). Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning. IEEE Transactions on Vehicular Technology, 69, 14413\u201314423. https:\/\/doi.org\/10.1109\/TVT.2020.3034800","journal-title":"IEEE Transactions on Vehicular Technology"},{"key":"4612_CR22","doi-asserted-by":"publisher","first-page":"113701","DOI":"10.1016\/j.eswa.2020.113701","volume":"160","author":"J Huang","year":"2020","unstructured":"Huang, J., Chang, Q., & Arinez, J. (2020). Deep reinforcement learning based preventive maintenance policy for serial production lines. Expert Systems with Applications, 160, 113701.","journal-title":"Expert Systems with Applications"},{"key":"4612_CR23","doi-asserted-by":"publisher","first-page":"106982","DOI":"10.1016\/j.compchemeng.2020.106982","volume":"141","author":"CD Hubbs","year":"2020","unstructured":"Hubbs, C. D., Li, C., Sahinidis, N. V., Grossmann, I. E., & Wassick, J. M. (2020). A deep reinforcement learning approach for chemical production scheduling. Computers & Chemical Engineering, 141, 106982.","journal-title":"Computers & Chemical Engineering"},{"key":"4612_CR24","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1287\/mnsc.41.4.577","volume":"41","author":"WC Jordan","year":"1995","unstructured":"Jordan, W. C., & Graves, S. C. (1995). Principles on the benefits of manufacturing process flexibility. Management Science, 41, 577\u2013594.","journal-title":"Management Science"},{"key":"4612_CR25","doi-asserted-by":"publisher","unstructured":"Kandel, I., & Castelli, M. (2020). The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express, 6, 312\u2013315. https:\/\/doi.org\/10.1016\/j.icte.2020.04.010https:\/\/www.sciencedirect.com\/science\/article\/pii\/S2405959519303455.","DOI":"10.1016\/j.icte.2020.04.010"},{"key":"4612_CR26","unstructured":"Kingma, D.P., & Ba, J. (2017). Adam: A method for stochastic optimization. arXiv:1412.6980."},{"key":"4612_CR27","doi-asserted-by":"publisher","first-page":"975","DOI":"10.1007\/BF01009452","volume":"34","author":"S Kirkpatrick","year":"1984","unstructured":"Kirkpatrick, S. (1984). Optimization by simulated annealing: Quantitative studies. Journal of Statistical Physics, 34, 975\u2013986.","journal-title":"Journal of Statistical Physics"},{"key":"4612_CR28","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1126\/science.220.4598.671","volume":"220","author":"S Kirkpatrick","year":"1983","unstructured":"Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671\u2013680.","journal-title":"Science"},{"key":"4612_CR29","unstructured":"Kool, W., van Hoof, H., & Welling, M. (2019). Attention, learn to solve routing problems! arXiv:1803.08475."},{"key":"4612_CR30","unstructured":"Kosanoglu, F., Turan, H.\u00a0H., & Atmis, M. (2018). A simulated annealing algorithm for integrated decisions on spare part inventories and cross-training policies in repairable inventory systems. In Proceedings of International Conference on Computers and Industrial Engineering (pp. 1\u201314)."},{"key":"4612_CR31","doi-asserted-by":"publisher","unstructured":"Krasheninnikova, E., & Garc\u00eda, J., Maestre, R., & Fern\u00e1ndez, F. (2019). Reinforcement learning for pricing strategy optimization in the insurance industry. Engineering Applications of Artificial Intelligence, 80, 8\u201319. https:\/\/doi.org\/10.1016\/j.engappai.2019.01.010http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0952197619300107.","DOI":"10.1016\/j.engappai.2019.01.010"},{"key":"4612_CR32","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.ijpe.2011.03.004","volume":"132","author":"E Levner","year":"2011","unstructured":"Levner, E., Perlman, Y., Cheng, T., & Levner, I. (2011). A network approach to modeling the multi-echelon spare-part inventory system with backorders and interval-valued demand. International Journal of Production Economics, 132, 43\u201351.","journal-title":"International Journal of Production Economics"},{"key":"4612_CR33","doi-asserted-by":"publisher","first-page":"2133","DOI":"10.1016\/j.cja.2019.07.003","volume":"32","author":"Z Li","year":"2019","unstructured":"Li, Z., Zhong, S., & Lin, L. (2019). An aero-engine life-cycle maintenance policy optimization algorithm: Reinforcement learning approach. Chinese Journal of Aeronautics, 32, 2133\u20132150.","journal-title":"Chinese Journal of Aeronautics"},{"key":"4612_CR34","doi-asserted-by":"crossref","unstructured":"Liang, S., Yang, Z., Jin, F., & Chen, Y. (2020). Data centers job scheduling with deep reinforcement learning. In H. W. Lauw, R.C.-W. Wong, A. Ntoulas, E.-P. Lim, S.-K. Ng, & S. J. Pan (Eds.), Advances in Knowledge Discovery and Data Mining (pp. 906\u2013917). Cham: Springer International Publishing.","DOI":"10.1007\/978-3-030-47436-2_68"},{"key":"4612_CR35","unstructured":"Lin, B., Ghaddar, B., & Nathwani, J. (2020). Deep reinforcement learning for electric vehicle routing problem with time windows. arXiv:2010.02068."},{"key":"4612_CR36","doi-asserted-by":"publisher","first-page":"71752","DOI":"10.1109\/ACCESS.2020.2987820","volume":"8","author":"C Liu","year":"2020","unstructured":"Liu, C., Chang, C., & Tseng, C. (2020). Actor-critic deep reinforcement learning for solving job shop scheduling problems. IEEE Access, 8, 71752\u201371762. https:\/\/doi.org\/10.1109\/ACCESS.2020.2987820","journal-title":"IEEE Access"},{"key":"4612_CR37","unstructured":"Ma, Q., Ge, S., He, D., Thaker, D., & Drori, I. (2019). Combinatorial optimization by graph pointer networks and hierarchical reinforcement learning. arXiv:1911.04936."},{"key":"4612_CR38","doi-asserted-by":"publisher","first-page":"5708","DOI":"10.3390\/s20195708","volume":"20","author":"Z Mahmoodzadeh","year":"2020","unstructured":"Mahmoodzadeh, Z., Wu, K.-Y., Droguett, E. L., & Mosleh, A. (2020). Condition-based maintenance with reinforcement learning for dry gas pipeline subject to internal corrosion. Sensors, 20, 5708.","journal-title":"Sensors"},{"key":"4612_CR39","doi-asserted-by":"crossref","unstructured":"Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016). Resource management with deep reinforcement learning. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (pp. 50\u201356).","DOI":"10.1145\/3005745.3005750"},{"key":"4612_CR40","doi-asserted-by":"crossref","unstructured":"Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. (2020). Reinforcement learning for combinatorial optimization: A survey. arXiv:2003.03600.","DOI":"10.1016\/j.cor.2021.105400"},{"key":"4612_CR41","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518, 529\u2013533. https:\/\/doi.org\/10.1038\/nature14236","journal-title":"Nature"},{"key":"4612_CR42","doi-asserted-by":"publisher","first-page":"472","DOI":"10.1287\/mnsc.20.4.472","volume":"20","author":"JA Muckstadt","year":"1973","unstructured":"Muckstadt, J. A. (1973). A model for a multi-item, multi-echelon, multi-indenture inventory system. Management Science, 20, 472\u2013481.","journal-title":"Management Science"},{"key":"4612_CR43","volume-title":"Analysis and algorithms for service parts supply chains","author":"JA Muckstadt","year":"2005","unstructured":"Muckstadt, J. A. (2005). Analysis and algorithms for service parts supply chains. Germany: Springer Science & Business Media."},{"key":"4612_CR44","unstructured":"Nazari, M., Oroojlooy, A., Snyder, L.\u00a0V., & Tak\u00e1\u010d, M. (2018). Reinforcement learning for solving the vehicle routing problem. arXiv:1802.04240."},{"key":"4612_CR45","doi-asserted-by":"crossref","unstructured":"Ong, K. S.\u00a0H., Niyato, D., & Yuen, C. (2020). Predictive maintenance for edge-based sensor networks: A deep reinforcement learning approach. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT) (pp. 1\u20136). IEEE.","DOI":"10.1109\/WF-IoT48130.2020.9221098"},{"key":"4612_CR46","doi-asserted-by":"publisher","first-page":"106649","DOI":"10.1016\/j.compchemeng.2019.106649","volume":"133","author":"P Petsagkourakis","year":"2020","unstructured":"Petsagkourakis, P., Sandoval, I., Bradford, E., Zhang, D., & del Rio-Chanona, E. (2020). Reinforcement learning for batch bioprocess optimization. Computers & Chemical Engineering, 133, 106649. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0098135419304168.","journal-title":"Computers & Chemical Engineering"},{"key":"4612_CR47","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1007\/s10479-017-2594-0","volume":"269","author":"SHA Rahmati","year":"2018","unstructured":"Rahmati, S. H. A., Ahmadi, A., & Govindan, K. (2018). A novel integrated condition-based maintenance and stochastic flexible job shop scheduling problem: simulation-based optimization approach. Annals of Operations Research, 269, 583\u2013621.","journal-title":"Annals of Operations Research"},{"key":"4612_CR48","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/j.apenergy.2019.03.027","volume":"241","author":"R Rocchetta","year":"2019","unstructured":"Rocchetta, R., Bellani, L., Compare, M., Zio, E., & Patelli, E. (2019). A reinforcement learning framework for optimal operation and maintenance of power grids. Applied Energy, 241, 291\u2013301.","journal-title":"Applied Energy"},{"key":"4612_CR49","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1007\/s10479-019-03371-3","volume":"287","author":"N Salari","year":"2020","unstructured":"Salari, N., & Makis, V. (2020). Joint maintenance and just-in-time spare parts provisioning policy for a multi-unit production system. Annals of Operations Research, 287, 351\u2013377.","journal-title":"Annals of Operations Research"},{"key":"4612_CR50","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1007\/s10479-014-1718-z","volume":"226","author":"P Samouei","year":"2015","unstructured":"Samouei, P., Kheirkhah, A. S., & Fattahi, P. (2015). A network approach modeling of multi-echelon spare-part inventory system with backorders and quantity discount. Annals of Operations Research, 226, 551\u2013563.","journal-title":"Annals of Operations Research"},{"key":"4612_CR51","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1287\/opre.16.1.122","volume":"16","author":"CC Sherbrooke","year":"1968","unstructured":"Sherbrooke, C. C. (1968). Metric: A multi-echelon technique for recoverable item control. Operations Research, 16, 122\u2013141.","journal-title":"Operations Research"},{"key":"4612_CR52","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1287\/opre.34.2.311","volume":"34","author":"CC Sherbrooke","year":"1986","unstructured":"Sherbrooke, C. C. (1986). VARI-METRIC: Improved approximations for multi-indenture, multi-echelon availability models. Operations Research, 34, 311\u2013319.","journal-title":"Operations Research"},{"key":"4612_CR53","doi-asserted-by":"publisher","first-page":"106600","DOI":"10.1016\/j.cie.2020.106600","volume":"147","author":"E Skordilis","year":"2020","unstructured":"Skordilis, E., & Moghaddass, R. (2020). A deep reinforcement learning approach for real-time sensor-driven decision making and predictive analytics. Computers & Industrial Engineering, 147, 106600.","journal-title":"Computers & Industrial Engineering"},{"key":"4612_CR54","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.ejor.2018.05.014","volume":"271","author":"A Sleptchenko","year":"2018","unstructured":"Sleptchenko, A., Hanbali, A. A., & Zijm, H. (2018). Joint planning of service engineers and spare parts. European Journal of Operational Research, 271, 97\u2013108.","journal-title":"European Journal of Operational Research"},{"key":"4612_CR55","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.ress.2016.04.006","volume":"153","author":"A Sleptchenko","year":"2016","unstructured":"Sleptchenko, A., & van der Heijden, M. (2016). Joint optimization of redundancy level and spare part inventories. Reliability Engineering & System Safety, 153, 64\u201374.","journal-title":"Reliability Engineering & System Safety"},{"key":"4612_CR56","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1016\/j.ijpe.2017.12.018","volume":"209","author":"A Sleptchenko","year":"2019","unstructured":"Sleptchenko, A., Turan, H. H., Pokharel, S., & ElMekkawy, T. Y. (2019). Cross-training policies for repair shops with spare part inventories. International Journal of Production Economics, 209, 334\u2013345.","journal-title":"International Journal of Production Economics"},{"key":"4612_CR57","doi-asserted-by":"publisher","first-page":"1143","DOI":"10.1057\/palgrave.jors.2602068","volume":"57","author":"B Suman","year":"2006","unstructured":"Suman, B., & Kumar, P. (2006). A survey of simulated annealing as a tool for single and multiobjective optimization. Journal of the Operational Research Society, 57, 1143\u20131160. https:\/\/doi.org\/10.1057\/palgrave.jors.2602068","journal-title":"Journal of the Operational Research Society"},{"key":"4612_CR58","unstructured":"Tang, Y., Agrawal, S., & Faenza, Y. (2020). Reinforcement learning for integer programming: Learning to cut. arXiv:1906.04859."},{"key":"4612_CR59","doi-asserted-by":"publisher","first-page":"107199","DOI":"10.1016\/j.ress.2020.107199","volume":"204","author":"HH Turan","year":"2020","unstructured":"Turan, H. H., Atmis, M., Kosanoglu, F., Elsawah, S., & Ryan, M. J. (2020a). A risk-averse simulation-based approach for a joint optimization of workforce capacity, spare part stocks and scheduling priorities in maintenance planning. Reliability Engineering & System Safety, 204, 107199.","journal-title":"Reliability Engineering & System Safety"},{"key":"4612_CR60","doi-asserted-by":"publisher","unstructured":"Turan, H. H., Kosanoglu, F., & Atmis, M. (2020b). A multi-skilled workforce optimisation in maintenance logistics networks by multi-thread simulated annealing algorithms. International Journal of Production Research, 1\u201323. https:\/\/doi.org\/10.1080\/00207543.2020.1735665","DOI":"10.1080\/00207543.2020.1735665"},{"key":"4612_CR61","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1016\/j.cie.2018.08.032","volume":"125","author":"HH Turan","year":"2018","unstructured":"Turan, H. H., Sleptchenko, A., Pokharel, S., & ElMekkawy, T. Y. (2018). A clustering-based repair shop design for repairable spare part supply systems. Computers & Industrial Engineering, 125, 232\u2013244.","journal-title":"Computers & Industrial Engineering"},{"key":"4612_CR62","doi-asserted-by":"publisher","first-page":"104887","DOI":"10.1016\/j.cor.2020.104887","volume":"117","author":"HH Turan","year":"2020","unstructured":"Turan, H. H., Sleptchenko, A., Pokharel, S., & ElMekkawy, T. Y. (2020c). A sorting based efficient heuristic for pooled repair shop designs. Computers & Operations Research, 117, 104887.","journal-title":"Computers & Operations Research"},{"key":"4612_CR63","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1023\/A:1023209813523","volume":"43","author":"A Van Harten","year":"2003","unstructured":"Van Harten, A., & Sleptchenko, A. (2003). On Markovian multi-class, multi-server queueing. Queueing systems, 43, 307\u2013328.","journal-title":"Queueing systems"},{"key":"4612_CR64","doi-asserted-by":"crossref","unstructured":"Walraven, E., Spaan, M.\u00a0T., & Bakker, B. (2016). Traffic flow optimization: A reinforcement learning approach. Engineering Applications of Artificial Intelligence, 52, 203 \u2013 212. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0952197616000038. https:\/\/doi.org\/10.1016\/j.engappai.2016.01.001.","DOI":"10.1016\/j.engappai.2016.01.001"},{"key":"4612_CR65","doi-asserted-by":"crossref","unstructured":"Wang, Y., & Tang, J. (2020). Optimized skill configuration for the seru production system under an uncertain demand. Annals of Operations Research, (pp. 1\u201321).","DOI":"10.1007\/s10479-020-03805-3"},{"key":"4612_CR66","doi-asserted-by":"publisher","unstructured":"Waschneck, B., Reichstaller, A., Belzner, L., Altenm\u00fcller, T., Bauernhansl, T., Knapp, A., & Kyek, A. (2018a). Deep reinforcement learning for semiconductor production scheduling. In 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC) (pp. 301\u2013306). https:\/\/doi.org\/10.1109\/ASMC.2018.8373191.","DOI":"10.1109\/ASMC.2018.8373191"},{"key":"4612_CR67","doi-asserted-by":"crossref","unstructured":"Waschneck, B., Reichstaller, A., Belzner, L., Altenm\u00fcller, T., Bauernhansl, T., Knapp, A., & Kyek, A. (2018b). Optimization of global production scheduling with deep reinforcement learning. Procedia CIRP, 72, 1264 \u2013 1269. 51st CIRP Conference on Manufacturing Systems.","DOI":"10.1016\/j.procir.2018.03.212"},{"key":"4612_CR68","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279\u2013292. https:\/\/doi.org\/10.1007\/BF00992698","journal-title":"Machine Learning"},{"key":"4612_CR69","doi-asserted-by":"publisher","first-page":"101906","DOI":"10.1016\/j.strusafe.2019.101906","volume":"83","author":"S Wei","year":"2020","unstructured":"Wei, S., Bao, Y., & Li, H. (2020). Optimal policy for structure maintenance: A deep reinforcement learning framework. Structural Safety, 83, 101906.","journal-title":"Structural Safety"},{"key":"4612_CR70","doi-asserted-by":"crossref","unstructured":"Wu, Y., Liu, L., Bae, J., Chow, K.-H., Iyengar, A., Pu, C., Wei, W., Yu, L., & Zhang, Q. (2019). Demystifying learning rate policies for high accuracy training of deep neural networks. arXiv:1908.06477.","DOI":"10.1109\/BigData47090.2019.9006104"},{"key":"4612_CR71","doi-asserted-by":"crossref","unstructured":"Yao, L., Dong, Q., Jiang, J., & Ni, F. (2020). Deep reinforcement learning for long-term pavement maintenance planning. Computer-Aided Civil and Infrastructure Engineering, 35, 1230\u20131245. arXiv:https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/mice.12558.","DOI":"10.1111\/mice.12558"},{"key":"4612_CR72","doi-asserted-by":"publisher","first-page":"3806","DOI":"10.1109\/TITS.2019.2909109","volume":"20","author":"JJQ Yu","year":"2019","unstructured":"Yu, J. J. Q., Yu, W., & Gu, J. (2019). Online vehicle routing with neural combinatorial optimization and deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 20, 3806\u20133817. https:\/\/doi.org\/10.1109\/TITS.2019.2909109","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"4612_CR73","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1007\/978-3-030-10997-4_30","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"C Zhang","year":"2019","unstructured":"Zhang, C., Gupta, C., Farahat, A., Ristovski, K., & Ghosh, D. (2019). Equipment health indicator learning using deep reinforcement learning. In U. Brefeld, E. Curry, E. Daly, B. MacNamee, A. Marascu, F. Pinelli, M. Berlingerio, & N. Hurley (Eds.), Machine Learning and Knowledge Discovery in Databases (pp. 488\u2013504). Cham: Springer International Publishing."},{"key":"4612_CR74","doi-asserted-by":"publisher","first-page":"107094","DOI":"10.1016\/j.ress.2020.107094","volume":"203","author":"N Zhang","year":"2020","unstructured":"Zhang, N., & Si, W. (2020). Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks. Reliability Engineering & System Safety, 203, 107094.","journal-title":"Reliability Engineering & System Safety"},{"key":"4612_CR75","doi-asserted-by":"publisher","unstructured":"Zhao, J., Mao, M., Zhao, X., & Zou, J. (2020). A hybrid of deep reinforcement learning and local search for the vehicle routing problems. IEEE Transactions on Intelligent Transportation Systems (pp. 1\u201311). https:\/\/doi.org\/10.1109\/TITS.2020.3003163","DOI":"10.1109\/TITS.2020.3003163"}],"container-title":["Annals of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-022-04612-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10479-022-04612-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-022-04612-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T18:23:04Z","timestamp":1722363784000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10479-022-04612-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,15]]},"references-count":75,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["4612"],"URL":"https:\/\/doi.org\/10.1007\/s10479-022-04612-8","relation":{},"ISSN":["0254-5330","1572-9338"],"issn-type":[{"value":"0254-5330","type":"print"},{"value":"1572-9338","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,15]]},"assertion":[{"value":"14 February 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 March 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}