{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T19:05:47Z","timestamp":1780081547325,"version":"3.54.0"},"reference-count":47,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,3,31]],"date-time":"2024-03-31T00:00:00Z","timestamp":1711843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010822","name":"Chengdu Science and Technology Project","doi-asserted-by":"publisher","award":["2022-YF05-01058-SN"],"award-info":[{"award-number":["2022-YF05-01058-SN"]}],"id":[{"id":"10.13039\/501100010822","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010822","name":"Chengdu Science and Technology Project","doi-asserted-by":"publisher","award":["2022JDGD0013"],"award-info":[{"award-number":["2022JDGD0013"]}],"id":[{"id":"10.13039\/501100010822","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Sichuan Province Foreign and Overseas High-end Talent Introduction Program","award":["2022-YF05-01058-SN"],"award-info":[{"award-number":["2022-YF05-01058-SN"]}]},{"name":"Sichuan Province Foreign and Overseas High-end Talent Introduction Program","award":["2022JDGD0013"],"award-info":[{"award-number":["2022JDGD0013"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the rapid development of economic globalization and green manufacturing, traditional flexible job shop scheduling has evolved into the low-carbon heterogeneous distributed flexible job shop scheduling problem (LHDFJSP). Additionally, modern smart manufacturing processes encounter complex and diverse contingencies, necessitating the ability to address dynamic events in real-world production activities. To date, there are limited studies that comprehensively address the intricate factors associated with the LHDFJSP, including workshop heterogeneity, job insertions and transfers, and considerations of low-carbon objectives. This paper establishes a multi-objective mathematical model with the goal of minimizing the total weighted tardiness and total energy consumption. To effectively solve this problem, diverse composite scheduling rules are formulated, alongside the application of a deep reinforcement learning (DRL) framework, i.e., Rainbow deep-Q network (Rainbow DQN), to learn the optimal scheduling strategy at each decision point in a dynamic environment. To verify the effectiveness of the proposed method, this paper extends the standard dataset to adapt to the LHDFJSP. Evaluation results confirm the generalization and robustness of the presented Rainbow DQN-based method.<\/jats:p>","DOI":"10.3390\/s24072251","type":"journal-article","created":{"date-parts":[[2024,3,31]],"date-time":"2024-03-31T13:32:56Z","timestamp":1711891976000},"page":"2251","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Dynamic Intelligent Scheduling in Low-Carbon Heterogeneous Distributed Flexible Job Shops with Job Insertions and Transfers"],"prefix":"10.3390","volume":"24","author":[{"given":"Yi","family":"Chen","sequence":"first","affiliation":[{"name":"College of Computer Science and Cyber Security, Chengdu University of Technology, Chengdu 610059, China"},{"name":"Sichuan Engineering Technology Research Center for Industrial Internet Intelligent Monitoring and Application, Chengdu 610059, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6548-2256","authenticated-orcid":false,"given":"Xiaojuan","family":"Liao","sequence":"additional","affiliation":[{"name":"College of Computer Science and Cyber Security, Chengdu University of Technology, Chengdu 610059, China"},{"name":"Sichuan Engineering Technology Research Center for Industrial Internet Intelligent Monitoring and Application, Chengdu 610059, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangzhu","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer Science and Cyber Security, Chengdu University of Technology, Chengdu 610059, China"},{"name":"Sichuan Engineering Technology Research Center for Industrial Internet Intelligent Monitoring and Application, Chengdu 610059, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yingjie","family":"Hou","sequence":"additional","affiliation":[{"name":"College of Computer Science and Cyber Security, Chengdu University of Technology, Chengdu 610059, China"},{"name":"Sichuan Engineering Technology Research Center for Industrial Internet Intelligent Monitoring and Application, Chengdu 610059, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,31]]},"reference":[{"key":"ref_1","unstructured":"International Energy Agency (2023, April 21). Energy Efficiency 2021. Available online: https:\/\/iea.blob.core.windows.net\/assets\/9c30109f-38a7-4a0b-b159-47f00d65e5be\/EnergyEfficiency2021.pdf."},{"key":"ref_2","unstructured":"U.S. Energy Information Administration (2023, April 21). Country Analysis Executive Summary: China, Available online: https:\/\/www.eia.gov\/international\/content\/analysis\/countries_long\/China\/china.pdf."},{"key":"ref_3","unstructured":"U.S. Energy Information Administration (2023, April 21). Annual Energy Review 2020, Available online: https:\/\/www.eia.gov\/totalenergy\/data\/annual\/index.php."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1007\/s00170-013-5354-6","article-title":"A clustering-based modified variable neighborhood search algorithm for a dynamic job shop scheduling problem","volume":"70","author":"Adibi","year":"2014","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"517","DOI":"10.2507\/IJSIMM19-3-CO14","article-title":"A discrete job-shop scheduling algorithm based on improved genetic algorithm","volume":"19","author":"Zhang","year":"2020","journal-title":"Int. J. Simul. Model."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6425","DOI":"10.1109\/TII.2019.2938572","article-title":"Digital-twin-based job shop scheduling toward smart manufacturing","volume":"15","author":"Fang","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.ejor.2009.01.008","article-title":"An improved genetic algorithm for the distributed and flexible job-shop scheduling problem","volume":"200","author":"Pezzella","year":"2010","journal-title":"Eur. J. Oper. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"100861","DOI":"10.1016\/j.swevo.2021.100861","article-title":"A hybrid estimation of distribution algorithm for distributed flexible job shop scheduling with crane transportations","volume":"62","author":"Du","year":"2021","journal-title":"Swarm Evol. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8013","DOI":"10.1109\/TCYB.2023.3280175","article-title":"Surprisingly Popular-Based Adaptive Memetic Algorithm for Energy-Efficient Distributed Flexible Job Shop Scheduling","volume":"53","author":"Li","year":"2023","journal-title":"IEEE Trans. Cybern."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"107082","DOI":"10.1016\/j.cie.2020.107082","article-title":"A cooperative water wave optimization algorithm with reinforcement learning for the distributed assembly no-idle flowshop scheduling problem","volume":"153","author":"Zhao","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"113721","DOI":"10.1016\/j.eswa.2020.113721","article-title":"An efficient memetic algorithm for distributed flexible job shop scheduling problem with transfers","volume":"160","author":"Luo","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"106347","DOI":"10.1016\/j.cie.2020.106347","article-title":"Mixed-integer linear programming and constraint programming formulations for solving distributed flexible job shop scheduling problem","volume":"142","author":"Meng","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"107884","DOI":"10.1016\/j.cie.2021.107884","article-title":"Intelligent factory many-objective distributed flexible job shop collaborative scheduling method","volume":"164","author":"Sang","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"121050","DOI":"10.1016\/j.eswa.2023.121050","article-title":"A Q-learning-based hyper-heuristic evolutionary algorithm for the distributed flexible job-shop scheduling problem with crane transportation","volume":"234","author":"Zhang","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2105","DOI":"10.1007\/s11431-022-2096-6","article-title":"Improved gray wolf optimizer for distributed flexible job shop scheduling problem","volume":"65","author":"Li","year":"2022","journal-title":"Sci. China Technol. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1177\/0954405414553069","article-title":"Energy-aware integrated process planning and scheduling for job shops","volume":"229","author":"Dai","year":"2015","journal-title":"Proc. Inst. Mech. Eng. Part B J. Eng. Manuf."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1177\/0954405414527959","article-title":"Low-carbon scheduling and estimating for a flexible job shop based on carbon footprint and carbon efficiency of multi-job processing","volume":"229","author":"Zhang","year":"2015","journal-title":"Proc. Inst. Mech. Eng. Part B J. Eng. Manuf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"46346","DOI":"10.1109\/ACCESS.2018.2866133","article-title":"Optimizing the low-carbon flexible job shop scheduling problem considering energy consumption","volume":"6","author":"Jiang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_19","first-page":"15","article-title":"A novel mathematical model and multi-objective method for the low-carbon flexible job shop scheduling problem","volume":"13","author":"Yin","year":"2017","journal-title":"Sustain. Comput. Inform. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"106544","DOI":"10.1016\/j.asoc.2020.106544","article-title":"An improved artificial bee colony algorithm for solving multi-objective low-carbon flexible job shop scheduling problem","volume":"95","author":"Li","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"119840","DOI":"10.1016\/j.eswa.2023.119840","article-title":"Dynamic distributed flexible job-shop scheduling problem considering operation inspection","volume":"224","author":"Zhu","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3020","DOI":"10.1080\/00207543.2018.1524165","article-title":"A heuristic model for dynamic flexible job shop scheduling problem considering variable processing times","volume":"57","author":"Katebi","year":"2019","journal-title":"Int. J. Prod. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107211","DOI":"10.1016\/j.cie.2021.107211","article-title":"An effective MCTS-based algorithm for minimizing makespan in dynamic flexible job shop scheduling problem","volume":"155","author":"Li","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1007\/s10951-008-0090-8","article-title":"A survey of dynamic scheduling in manufacturing systems","volume":"12","author":"Ouelhadj","year":"2009","journal-title":"J. Sched."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"102283","DOI":"10.1016\/j.rcim.2021.102283","article-title":"Real-time data-driven dynamic scheduling for flexible job shop with insufficient transportation resources using hybrid deep Q network","volume":"74","author":"Li","year":"2022","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105823","DOI":"10.1016\/j.cor.2022.105823","article-title":"Digital twin-enabled dynamic scheduling with preventive maintenance using a double-layer Q-learning algorithm","volume":"144","author":"Yan","year":"2022","journal-title":"Comput. Oper. Res."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chang, J., Yu, D., Hu, Y., He, W., and Yu, H. (2022). Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival. Processes, 10.","DOI":"10.3390\/pr10040760"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yan, Q., Wu, W., and Wang, H. (2022). Deep Reinforcement Learning for Distributed Flow Shop Scheduling with Flexible Maintenance. Machines, 10.","DOI":"10.3390\/machines10030210"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhang, M., Lu, Y., Hu, Y., Amaitik, N., and Xu, Y. (2022). Dynamic Scheduling Method for Job-Shop Manufacturing Systems by Deep Reinforcement Learning with Proximal Policy Optimization. Sustainability, 14.","DOI":"10.3390\/su14095177"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"116222","DOI":"10.1016\/j.eswa.2021.116222","article-title":"Deep reinforcement learning based scheduling within production plan in semiconductor fabrication","volume":"191","author":"Lee","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1007\/s10845-022-01915-2","article-title":"On reliability of reinforcement learning based production scheduling systems: A comparative survey","volume":"33","author":"Meyes","year":"2022","journal-title":"J. Intell. Manuf."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3020","DOI":"10.1109\/TASE.2021.3104716","article-title":"Real-time scheduling for dynamic partial-no-wait multi-objective flexible job shop by deep reinforcement learning","volume":"19","author":"Luo","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"107489","DOI":"10.1016\/j.cie.2021.107489","article-title":"Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning","volume":"159","author":"Luo","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4049","DOI":"10.1080\/00207543.2022.2058432","article-title":"Deep reinforcement learning for dynamic scheduling of a flexible job shop","volume":"60","author":"Liu","year":"2022","journal-title":"Int. J. Prod. Res."},{"key":"ref_35","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, King\u2019s College]."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon","volume":"38","author":"Tesauro","year":"1995","journal-title":"Commun. ACM"},{"key":"ref_37","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA. No. 1.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_39","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 19\u201324). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_40","unstructured":"Fortunato, M., Azar, M.G., Piot, B., Menick, J., Osband, I., Graves, A., and Legg, S. (2017). Noisy networks for exploration. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"De Asis, K., Hernandez-Garcia, J., Holland, G., and Sutton, R. (2018, January 2\u20137). Multi-step reinforcement learning: A unifying algorithm. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LO, USA. No. 1.","DOI":"10.1609\/aaai.v32i1.11631"},{"key":"ref_43","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., and Silver, D. (2018, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LO, USA. No. 1.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"V\u00e4th, D., and Vu, N.T. (2019, January 11\u201313). To combine or not to combine? A rainbow deep reinforcement learning agent for dialog policies. Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, Stockholm, Sweden.","DOI":"10.18653\/v1\/W19-5908"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/BF02023073","article-title":"Routing and scheduling in a flexible job shop by tabu search","volume":"41","author":"Brandimarte","year":"1993","journal-title":"Ann. Oper. Res."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"117796","DOI":"10.1016\/j.eswa.2022.117796","article-title":"A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem","volume":"205","author":"Lei","year":"2022","journal-title":"Expert Syst. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/7\/2251\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:21:58Z","timestamp":1760106118000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/7\/2251"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,31]]},"references-count":47,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["s24072251"],"URL":"https:\/\/doi.org\/10.3390\/s24072251","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,31]]}}}