{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T07:36:55Z","timestamp":1780990615235,"version":"3.54.1"},"reference-count":32,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T00:00:00Z","timestamp":1740700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>In this paper, a dynamic energy-efficient hybrid flow shop (TDEHFSP) scheduling model is proposed, considering random arrivals of new jobs and transport by transfer vehicles. To simultaneously optimise the maximum completion time and the total energy consumption, a co-evolutionary approach (DDQCE) using a double deep Q-network (DDQN) is introduced, where global and local search tasks are assigned to different populations to optimise the use of computational resources. In addition, a multi-objective NEW heuristic strategy is implemented to generate an initial population with enhanced convergence and diversity. The DDQCE incorporates an energy-efficient strategy based on time interval \u2018left shift\u2019 and turn-on\/off mechanisms, alongside a rescheduling model to manage dynamic disturbances. In addition, 36 test instances of varying sizes, simplified from the excavator boom manufacturing process, are designed for comparative experiments with traditional algorithms. The experimental results demonstrate that DDQCE achieves 40% more Pareto-optimal solutions compared to NSGA-II and MOEA\/D while requiring 10% less computational time, confirming that this algorithm efficiently solves the TDEHFSP problem.<\/jats:p>","DOI":"10.3390\/systems13030170","type":"journal-article","created":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T04:28:18Z","timestamp":1740716898000},"page":"170","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Double Deep Q-Network-Based Solution to a Dynamic, Energy-Efficient Hybrid Flow Shop Scheduling System with the Transport Process"],"prefix":"10.3390","volume":"13","author":[{"given":"Qinglei","family":"Zhang","sequence":"first","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huaqiang","family":"Si","sequence":"additional","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiyun","family":"Qin","sequence":"additional","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianguo","family":"Duan","sequence":"additional","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Zhou","sequence":"additional","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huaixia","family":"Shi","sequence":"additional","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liang","family":"Nie","sequence":"additional","affiliation":[{"name":"China Institute of FTZ Supply Chain, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1016\/j.ejor.2015.05.019","article-title":"Carbon-efficient scheduling of flow shops by multi-objective optimization","volume":"248","author":"Ding","year":"2016","journal-title":"Eur. J. Oper. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"119077","DOI":"10.1016\/j.eswa.2022.119077","article-title":"Efficient multi-objective meta-heuristic algorithms for energy-aware non-permutation flow-shop scheduling problem","volume":"213","author":"Goli","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"105977","DOI":"10.1016\/j.engappai.2023.105977","article-title":"Solving energy-efficient fuzzy hybrid flow-shop scheduling problem at a variable machine speed using an extended NSGA-II","volume":"121","author":"Wang","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4247","DOI":"10.1080\/00207540701450013","article-title":"Operational methods for minimization of energy consumption of manufacturing equipment","volume":"45","author":"Mouzon","year":"2007","journal-title":"Int. J. Prod. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1080\/19397030802257236","article-title":"A framework to minimise total energy consumption and total tardiness on a single machine","volume":"1","author":"Mouzon","year":"2008","journal-title":"Int. J. Sustain. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1016\/j.rcim.2013.04.001","article-title":"Energy-efficient scheduling for a flexible flow shop using an improved genetic-simulated annealing algorithm","volume":"29","author":"Dai","year":"2013","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.ijpe.2013.01.028","article-title":"Hybrid flow shop scheduling considering machine electricity consumption cost","volume":"146","author":"Luo","year":"2013","journal-title":"Int. J. Prod. Econ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"105788","DOI":"10.1016\/j.cor.2022.105788","article-title":"Exact method to optimize the total electricity cost in two-machine permutation flow shop scheduling problem under Time-of-use tariff","volume":"144","author":"Ho","year":"2022","journal-title":"Comput. Oper. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"121309","DOI":"10.1016\/j.eswa.2023.121309","article-title":"Multi-objective optimization of energy-efficient remanufacturing system scheduling problem with lot-streaming production mode","volume":"237","author":"Tian","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1007\/s10845-021-01837-5","article-title":"Energy-time tradeoffs for remanufacturing system scheduling using an invasive weed optimization algorithm","volume":"34","author":"Wang","year":"2023","journal-title":"J. Intell. Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s10845-020-01580-3","article-title":"A discrete differential evolution algorithm for flow shop group scheduling problem with sequence-dependent setup and transportation times","volume":"32","author":"Yuan","year":"2021","journal-title":"J. Intell. Manuf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1080\/0951192X.2014.880805","article-title":"An artificial bee colony algorithm for a two-stage hybrid flowshop scheduling problem with multilevel product structures and requirement operations","volume":"28","author":"Kheirandish","year":"2015","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"109717","DOI":"10.1016\/j.cie.2023.109717","article-title":"Modelling and optimization of distributed assembly hybrid flowshop scheduling problem with transportation resource scheduling","volume":"186","author":"Luo","year":"2023","journal-title":"Comput. Ind. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4984","DOI":"10.1109\/TSMC.2019.2916088","article-title":"A three-stage multiobjective approach based on decomposition for an energy-efficient hybrid flow shop scheduling problem","volume":"50","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"117489","DOI":"10.1016\/j.eswa.2022.117489","article-title":"Robust scheduling for flexible machining job shop subject to machine breakdowns and new job arrivals considering system reusability and task recurrence","volume":"203","author":"Duan","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/j.jpdc.2018.07.022","article-title":"GPU based parallel genetic algorithm for solving an energy-efficient dynamic flexible flow shop scheduling problem","volume":"133","author":"Luo","year":"2019","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.jmsy.2021.09.018","article-title":"Real-time integrated production-scheduling and maintenance-planning in a flexible job shop with machine deterioration and condition-based maintenance","volume":"61","author":"Ghaleb","year":"2021","journal-title":"J. Manuf. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"120837","DOI":"10.1016\/j.eswa.2023.120837","article-title":"Q-learning driven multi-population memetic algorithm for distributed three-stage assembly hybrid flow shop scheduling with flexible preventive maintenance","volume":"232","author":"Jia","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"106360","DOI":"10.1016\/j.cor.2023.106360","article-title":"Multi-objective energy-efficient hybrid flow shop scheduling using Q-learning and GVNS driven NSGA-II","volume":"159","author":"Li","year":"2023","journal-title":"Comput. Oper. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TCCN.2021.3063525","article-title":"Joint buffer-aided hybrid-duplex relay selection and power allocation for secure cognitive networks with double deep Q-network","volume":"7","author":"Huang","year":"2021","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7848","DOI":"10.1109\/TVT.2022.3169804","article-title":"Intelligent handover algorithm for vehicle-to-network communications with double-deep Q-learning","volume":"71","author":"Tan","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6615","DOI":"10.1109\/TCOMM.2022.3204846","article-title":"Delay-aware and energy-efficient carrier aggregation in 5G using double deep Q-networks","volume":"70","author":"Khoramnejad","year":"2022","journal-title":"IEEE Trans. Commun."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1109\/TCCN.2021.3116251","article-title":"On joint offloading and resource allocation: A double deep q-network approach","volume":"7","author":"Khoramnejad","year":"2021","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"7363","DOI":"10.1109\/TSMC.2020.2967936","article-title":"Deep Q-learning with Q-matrix transfer learning for novel fire evacuation environment","volume":"51","author":"Sharma","year":"2020","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"135738","DOI":"10.1016\/j.jclepro.2022.135738","article-title":"An improved multi-objective firefly algorithm for energy-efficient hybrid flowshop rescheduling problem","volume":"385","author":"Wang","year":"2023","journal-title":"J. Clean. Prod."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.omega.2013.12.004","article-title":"A novel discrete artificial bee colony algorithm for the hybrid flowshop scheduling problem with makespan minimisation","volume":"45","author":"Pan","year":"2014","journal-title":"Omega"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3818","DOI":"10.1109\/TCYB.2022.3164165","article-title":"Solving biobjective distributed flow-shop scheduling problems with lot-streaming using an improved Jaya algorithm","volume":"53","author":"Pan","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2854","DOI":"10.1080\/00207543.2022.2070786","article-title":"A reinforcement learning-driven brain storm optimisation algorithm for multi-objective energy-efficient distributed assembly no-wait flow shop scheduling problem","volume":"61","author":"Zhao","year":"2023","journal-title":"Int. J. Prod. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"121570","DOI":"10.1016\/j.eswa.2023.121570","article-title":"A multiobjective memetic algorithm with particle swarm optimization and Q-learning-based local search for energy-efficient distributed heterogeneous hybrid flow-shop scheduling problem","volume":"237","author":"Zhang","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"100858","DOI":"10.1016\/j.swevo.2021.100858","article-title":"Energy-efficient distributed heterogeneous welding flow shop scheduling problem using a modified MOEA\/D","volume":"62","author":"Wang","year":"2021","journal-title":"Swarm Evol. Comput."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/3\/170\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:44:21Z","timestamp":1760028261000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/3\/170"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,28]]},"references-count":32,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["systems13030170"],"URL":"https:\/\/doi.org\/10.3390\/systems13030170","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,28]]}}}