{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T12:42:04Z","timestamp":1784205724843,"version":"3.55.0"},"reference-count":50,"publisher":"ASME International","issue":"5","license":[{"start":{"date-parts":[[2023,5,9]],"date-time":"2023-05-09T00:00:00Z","timestamp":1683590400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Practical manufacturing system operates in highly dynamic and uncertain environments, where stochastic disturbances disrupt the execution of the production schedule as originally developed. Previous dynamic scheduling mainly focuses on the constructing predictive models for machine unavailability, with little studies on the adaptive and self-learning capacities for changing scheduling environments. Therefore, a digital twin (DT) driven scheduling with a dynamic feedback mechanism is proposed, in which a reinforcement learning (RL) based adaptive scheduling is developed in DT to make corrective decisions for the disturbances during production runs. In the proposed architecture, the happening disturbance is first detected in the virtual layer by the status continuously updating in accordance with the physical workshop. Furthermore, the reschedule triggering condition is determined in real-time through the calculation of the progress deviations resulting from disturbances. For the scheduling approach, the distributed RL (DRL) based adaptive scheduling method is built to perceive the dynamic production status from virtual environment and implement corrective strategies to hedge against the occurred disturbances. Finally, the proposed method is verified by a practical job shop case and the corresponding DT system is developed to show the effectiveness and advantages after a practical implementation.<\/jats:p>","DOI":"10.1115\/1.4062349","type":"journal-article","created":{"date-parts":[[2023,4,19]],"date-time":"2023-04-19T02:10:28Z","timestamp":1681870228000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":23,"title":["An Adaptive Job Shop Scheduling Mechanism for Disturbances by Running Reinforcement Learning in Digital Twin Environment"],"prefix":"10.1115","volume":"23","author":[{"given":"Weiguang","family":"Fang","sequence":"first","affiliation":[{"name":"China Academy of Launch Vehicle Technology , Beijing 100076 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , Nanjing 210016 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiwei","family":"Qian","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , Nanjing 210016 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Guo","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , Nanjing 210016 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaoxun","family":"Li","sequence":"additional","affiliation":[{"name":"China Academy of Launch Vehicle Technology , Beijing 100076 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zeqing","family":"Liu","sequence":"additional","affiliation":[{"name":"China Academy of Launch Vehicle Technology , Beijing 100076 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenning","family":"Liu","sequence":"additional","affiliation":[{"name":"China Academy of Launch Vehicle Technology , Beijing 100076 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongpao","family":"Hong","sequence":"additional","affiliation":[{"name":"China Academy of Launch Vehicle Technology , Beijing 100076 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2023,5,9]]},"reference":[{"issue":"1","key":"2023050914234563900_CIT0001","first-page":"35","article-title":"Scheduling: Theory, Algorithms and Systems Development","volume":"1","author":"Hadavi","year":"1991","journal-title":"Oper. Res. Proc."},{"issue":"1","key":"2023050914234563900_CIT0002","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1016\/j.jmsy.2017.03.008","article-title":"Big Data Analytics Based Fault Prediction for Shop Floor Scheduling","volume":"43","author":"Ji","year":"2017","journal-title":"J. Manuf. Syst."},{"issue":"12","key":"2023050914234563900_CIT0003","doi-asserted-by":"publisher","first-page":"3802","DOI":"10.1080\/00207543.2018.1504248","article-title":"Industry 4.0: Smart Scheduling","volume":"57","author":"Rossit","year":"2019","journal-title":"Int. J. Prod. Res."},{"issue":"4","key":"2023050914234563900_CIT0004","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1007\/s10951-008-0090-8","article-title":"A Survey of Dynamic Scheduling in Manufacturing Systems","volume":"12","author":"Ouelhadj","year":"2009","journal-title":"J. Schedul."},{"issue":"1","key":"2023050914234563900_CIT0005","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.engappai.2014.09.015","article-title":"A Discrete Teaching-Learning-Based Optimisation Algorithm for Realistic Flowshop Rescheduling Problems","volume":"37","author":"Li","year":"2015","journal-title":"Eng. Appl. Artificial Intell."},{"issue":"1","key":"2023050914234563900_CIT0006","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/j.jmsy.2019.12.004","article-title":"An Approximate Nondominated Sorting Genetic Algorithm to Integrate Optimization of Production Scheduling and Accurate Maintenance Based on Reliability Intervals","volume":"54","author":"Chen","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"1","key":"2023050914234563900_CIT0007","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/j.jmsy.2019.05.003","article-title":"Fog Computing and Convolutional Neural Network Enabled Prognosis for Machining Process Optimization","volume":"52","author":"Liang","year":"2019","journal-title":"J. Manuf. Syst."},{"issue":"7","key":"2023050914234563900_CIT0008","doi-asserted-by":"publisher","first-page":"1485","DOI":"10.1007\/s10845-016-1194-1","article-title":"Manifold Learning Based Rescheduling Decision Mechanism for Recessive Disturbances in RFID-Driven Job Shops","volume":"29","author":"Wang","year":"2016","journal-title":"J. Intell. Manuf."},{"issue":"1","key":"2023050914234563900_CIT0009","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1007\/s12559-018-9595-4","article-title":"Extracting New Dispatching Rules for Multi-objective Dynamic Flexible Job Shop Scheduling With Limited Buffer Spaces","volume":"12","author":"Teymourifar","year":"2020","journal-title":"Cogn. Comput."},{"issue":"2","key":"2023050914234563900_CIT0010","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1007\/s10845-018-1454-3","article-title":"Adaptive Job Shop Scheduling Strategy Based on Weighted Q-Learning Algorithm","volume":"31","author":"Wang","year":"2018","journal-title":"J. Intell. Manuf."},{"issue":"1","key":"2023050914234563900_CIT0011","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.procir.2018.03.103","article-title":"Digital Twin Service Towards Smart Manufacturing","volume":"72","author":"Qi","year":"2018","journal-title":"Proc. CIRP"},{"issue":"1","key":"2023050914234563900_CIT0012","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.jmsy.2020.04.013","article-title":"Digital Twin-Based Smart Assembly Process Design and Application Framework for Complex Products and Its Case Study","volume":"58","author":"Yi","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"3","key":"2023050914234563900_CIT0013","doi-asserted-by":"publisher","first-page":"030802","DOI":"10.1115\/1.4050244","article-title":"Digital Twins: Review and Challenges","volume":"21","author":"Juarez","year":"2021","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2023050914234563900_CIT0014","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1016\/j.eng.2019.01.014","article-title":"Digital Twins and Cyber-Physical Systems Toward Smart Manufacturing and Industry 4.0: Correlation and Comparison","volume":"5","author":"Tao","year":"2019","journal-title":"Engineering"},{"issue":"12","key":"2023050914234563900_CIT0015","doi-asserted-by":"publisher","first-page":"3935","DOI":"10.1080\/00207543.2018.1443229","article-title":"Digital Twin-Driven Product Design Framework","volume":"57","author":"Tao","year":"2019","journal-title":"Int. J. Prod. Res."},{"issue":"1","key":"2023050914234563900_CIT0016","doi-asserted-by":"publisher","first-page":"20418","DOI":"10.1109\/ACCESS.2017.2756069","article-title":"Digital Twin Shop-Floor: A New Shop-Floor Paradigm Towards Smart Manufacturing","volume":"5","author":"Tao","year":"2017","journal-title":"IEEE Access"},{"issue":"1","key":"2023050914234563900_CIT0017","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/j.cirp.2018.04.055","article-title":"Digital Twin Driven Prognostics and Health Management for Complex Equipment","volume":"67","author":"Tao","year":"2018","journal-title":"CIRP Ann."},{"issue":"1","key":"2023050914234563900_CIT0018","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.mfglet.2018.02.006","article-title":"Digital Twin-Proof of Concept","volume":"15","author":"Haag","year":"2018","journal-title":"Manuf. Lett."},{"issue":"4","key":"2023050914234563900_CIT0019","doi-asserted-by":"publisher","first-page":"1034","DOI":"10.1080\/00207543.2019.1607978","article-title":"Knowledge-Driven Digital Twin Manufacturing Cell Towards Intelligent Manufacturing","volume":"58","author":"Zhou","year":"2020","journal-title":"Int. J. Prod. Res."},{"issue":"1","key":"2023050914234563900_CIT0020","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1016\/j.jmsy.2020.04.015","article-title":"Modeling and Implementation of a Digital Twin of Material Flows Based on Physics Simulation","volume":"58","author":"Glatt","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"1","key":"2023050914234563900_CIT0021","doi-asserted-by":"publisher","first-page":"031005","DOI":"10.1115\/1.4049634","article-title":"Multidimensional Data Modeling and Model Validation for Digital Twin Workshop","volume":"21","author":"Qian","year":"2021","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"1","key":"2023050914234563900_CIT0022","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.jmsy.2020.06.002","article-title":"Consistency Retention Method for CNC Machine Tool Digital Twin Model","volume":"58","author":"Wei","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"1","key":"2023050914234563900_CIT0023","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1016\/j.jmsy.2020.04.008","article-title":"Digital Twin Enhanced Dynamic Job-Shop Scheduling","volume":"58","author":"Zhang","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"12","key":"2023050914234563900_CIT0024","doi-asserted-by":"publisher","first-page":"6425","DOI":"10.1109\/TII.2019.2938572","article-title":"Digital-Twin-Based Job Shop Scheduling Toward Smart Manufacturing","volume":"15","author":"Fang","year":"2019","journal-title":"IEEE Trans. Indust. Inform."},{"issue":"1","key":"2023050914234563900_CIT0025","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.jmsy.2021.05.007","article-title":"Bi-Level Dynamic Scheduling Architecture Based on Service Unit Digital Twin Agents","volume":"60","author":"Zhang","year":"2021","journal-title":"J. Manuf. Syst."},{"issue":"1","key":"2023050914234563900_CIT0026","doi-asserted-by":"publisher","first-page":"102309","DOI":"10.1016\/j.rcim.2021.102309","article-title":"Rapid Construction Method of Equipment Model for Discrete Manufacturing Digital Twin Workshop System","volume":"75","author":"Zhang","year":"2022","journal-title":"Rob. Comput. Integr. Manuf."},{"issue":"1","key":"2023050914234563900_CIT0027","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1109\/TNN.2004.842673","article-title":"Reinforcement Learning: An Introduction","volume":"16","author":"Sutton","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"issue":"5","key":"2023050914234563900_CIT0028","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement Learning: An Introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"issue":"7","key":"2023050914234563900_CIT0029","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1016\/j.cor.2011.07.019","article-title":"Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling With Reinforcement Learning","volume":"39","author":"Zhang","year":"2012","journal-title":"Comput. Oper. Res."},{"issue":"2","key":"2023050914234563900_CIT0030","doi-asserted-by":"publisher","first-page":"021010","DOI":"10.1115\/1.4052800","article-title":"Knowledge Acquisition of Self-Organizing Systems With Deep Multiagent Reinforcement Learning","volume":"22","author":"Ji","year":"2022","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"7","key":"2023050914234563900_CIT0031","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1016\/j.cie.2018.03.039","article-title":"Real-Time Scheduling for a Smart Factory Using a Reinforcement Learning Approach","volume":"125","author":"Shiue","year":"2018","journal-title":"Comput. Ind. Eng."},{"issue":"7","key":"2023050914234563900_CIT0032","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.cie.2017.05.026","article-title":"A Reinforcement Learning Approach to Parameter Estimation in Dynamic Job Shop Scheduling","volume":"110","author":"Shahrabi","year":"2017","journal-title":"Comput. Ind. Eng."},{"issue":"1","key":"2023050914234563900_CIT0033","doi-asserted-by":"publisher","first-page":"14699","DOI":"10.1109\/ACCESS.2018.2800641","article-title":"Gantry Work Cell Scheduling Through Reinforcement Learning With Knowledge-Guided Reward Setting","volume":"6","author":"Ou","year":"2018","journal-title":"IEEE Access"},{"issue":"1","key":"2023050914234563900_CIT0034","doi-asserted-by":"publisher","first-page":"9987","DOI":"10.1109\/ACCESS.2020.2964955","article-title":"A Deep-Reinforcement-Learning-Based Optimization Approach for Real-Time Scheduling in Cloud Manufacturing","volume":"8","author":"Zhu","year":"2020","journal-title":"IEEE Access"},{"issue":"1","key":"2023050914234563900_CIT0035","doi-asserted-by":"publisher","first-page":"106749","DOI":"10.1016\/j.cie.2020.106749","article-title":"Deep Reinforcement Learning Based AGVs Real-Time Scheduling With Mixed Rule for Flexible Shop Floor in Industry 4.0","volume":"149","author":"Hu","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"2023050914234563900_CIT0036","first-page":"1264","article-title":"Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling With Reinforcement Learning","author":"Waschneck","year":"2018"},{"key":"2023050914234563900_CIT0037","doi-asserted-by":"publisher","DOI":"10.1080\/00207543.2021.1975057","article-title":"Multi-Resource Constrained Dynamic Workshop Scheduling Based on Proximal Policy Optimisation","author":"Luo","year":"2021","journal-title":"Int. J. Prod. Res"},{"issue":"7","key":"2023050914234563900_CIT0038","doi-asserted-by":"publisher","first-page":"102198","DOI":"10.1016\/j.rcim.2021.102198","article-title":"Research on Flexible Job Shop Scheduling Under Finite Transportation Conditions for Digital Twin Workshop","volume":"72","author":"Yan","year":"2021","journal-title":"Rob. Comput.-Integr. Manuf."},{"issue":"10","key":"2023050914234563900_CIT0039","doi-asserted-by":"publisher","first-page":"1485","DOI":"10.3901\/JME.2009.10.137","article-title":"Dynamic Rescheduling Optimization of Job-Shop Under Uncertain Conditions","volume":"45","author":"Liu","year":"2009","journal-title":"J. Mech. Eng."},{"issue":"9","key":"2023050914234563900_CIT0040","doi-asserted-by":"publisher","first-page":"2751","DOI":"10.1080\/00207543.2019.1602744","article-title":"Big Data Driven Jobs Remaining Time Prediction in Discrete Manufacturing System: A Deep Learning-Based Approach","volume":"58","author":"Fang","year":"2020","journal-title":"Int. J. Prod. Res."},{"issue":"4","key":"2023050914234563900_CIT0041","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1016\/S0278-6125(01)80005-4","article-title":"Predicting the Performance of Rescheduling Strategies for Parallel Machine Systems","volume":"19","author":"Vieira","year":"2000","journal-title":"J. Manuf. Syst."},{"issue":"1","key":"2023050914234563900_CIT0042","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/S0377-2217(97)00396-2","article-title":"Match-Up Scheduling Under a Machine Break-Down","volume":"112","author":"Akturk","year":"1999","journal-title":"Eur. J. Oper. Res."},{"issue":"6","key":"2023050914234563900_CIT0043","doi-asserted-by":"publisher","first-page":"10627","DOI":"10.1109\/JIOT.2019.2940131","article-title":"A Two-Stage Transfer Learning Based Deep Learning Approach for Production Progress Prediction in IoT-Enabled Manufacturing","volume":"6","author":"Huang","year":"2019","journal-title":"IEEE Internet Thing J."},{"issue":"4","key":"2023050914234563900_CIT0044","doi-asserted-by":"publisher","first-page":"1809","DOI":"10.1007\/s10845-017-1350-2","article-title":"Review of Job Shop Scheduling Research and Its New Perspectives Under Industry 4.0","volume":"30","author":"Zhang","year":"2017","journal-title":"J. Intell. Manuf."},{"issue":"7","key":"2023050914234563900_CIT0045","doi-asserted-by":"publisher","first-page":"106246","DOI":"10.1016\/j.cie.2019.106246","article-title":"A Parallel Gated Recurrent Units (P-GRUs) Network for the Shifting Lateness Bottleneck Prediction in Make-to-Order Production System","volume":"140","author":"Fang","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"2023050914234563900_CIT0046","doi-asserted-by":"publisher","DOI":"10.13196\/j.cims.2021.01.018","article-title":"Study on Non-Permutation Flow Shop Scheduling Problem Based on Deep Reinforcement Learning","author":"Xiao","year":"2019","journal-title":"Comput. Integr. Manuf. Syst"},{"issue":"9","key":"2023050914234563900_CIT0047","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1007\/s00170-006-0662-8","article-title":"Dynamic Parallel Machine Scheduling With Mean Weighted Tardiness Objective by Q-Learning","volume":"34","author":"Zhang","year":"2007","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"2023050914234563900_CIT0048","first-page":"425","article-title":"A Study of Count-Based Exploration and Bonus for Reinforcement Learning","author":"Xu","year":"2017"},{"key":"2023050914234563900_CIT0049","first-page":"205","article-title":"OPC-UA and DPWS Interoperability for Factory Floor Monitoring Using Complex Event Processing","author":"Izaguirre","year":"2011"},{"issue":"4","key":"2023050914234563900_CIT0050","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1109\/TASE.2017.2784826","article-title":"Real-Time Manufacturing Machine and System Performance Monitoring Using Internet of Things","volume":"15","author":"Saez","year":"2018","journal-title":"IEEE Trans. Autom. Sci. Eng."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/23\/5\/051013\/7011818\/jcise_23_5_051013.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/23\/5\/051013\/7011818\/jcise_23_5_051013.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,9]],"date-time":"2023-05-09T18:58:23Z","timestamp":1683658703000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/23\/5\/051013\/1162999\/An-Adaptive-Job-Shop-Scheduling-Mechanism-for"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,9]]},"references-count":50,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4062349","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,9]]},"article-number":"051013"}}