{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T14:36:46Z","timestamp":1781534206066,"version":"3.54.5"},"reference-count":38,"publisher":"ASME International","issue":"9","license":[{"start":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T00:00:00Z","timestamp":1778803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"funder":[{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2022M721597"],"award-info":[{"award-number":["2022M721597"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52105522"],"award-info":[{"award-number":["52105522"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>As a critical stage in helicopter manufacturing, the assembly process relies on effective scheduling to ensure both efficiency and quality. Traditional, experience-based scheduling methods are often inadequate for complex shop-floor environments, especially given the heterogeneity of worker skills and complex technological constraints. Therefore, this article proposes a deep reinforcement learning approach based on proximal policy optimization with an attention mechanism (PPO-AM) to solve the helicopter assembly workshop scheduling problem (HASP) with consideration for worker skill proficiency. First, an assembly time prediction model was established that integrates workers' skill levels, task criticality, and the dynamic evolution of proficiency. This model provides a precise foundation for task-time estimation in assembly workshop scheduling. Based on this foundation, a shop scheduling model incorporating multiple process constraints was constructed. Furthermore, the state space, action space, and reward function were dynamically adjusted according to production progress, thereby formulating the problem as a Markov decision process (MDP). Within this framework, a PPO-AM method incorporating a self-attention mechanism was proposed, and an assembly shop scheduling agent was developed based on this method to enable flexible and efficient worker allocation in practical scenarios. The PPO-AM method leverages the self-attention mechanism to assess the importance of state features, enabling the agent to adaptively focus on critical information, thereby enhancing its state awareness and policy generalization capability. Experiments were conducted on assembly shop cases of various scales as well as in real-world engineering scenarios, and the results demonstrated that PPO-AM exhibits superior performance and practical value in complex assembly scheduling tasks.<\/jats:p>","DOI":"10.1115\/1.4071848","type":"journal-article","created":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T16:46:42Z","timestamp":1777999602000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":0,"title":["Deep Reinforcement Learning for Helicopter Assembly Workshop Scheduling Considering the Workers' Operational Proficiency"],"prefix":"10.1115","volume":"26","author":[{"given":"Zihao","family":"Zhou","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01scyh794","id-type":"ROR","asserted-by":"publisher"}],"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Guo","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaohua","family":"Huang","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lijun","family":"Ma","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01scyh794","id-type":"ROR","asserted-by":"publisher"}],"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai","family":"Geng","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01scyh794","id-type":"ROR","asserted-by":"publisher"}],"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shengbo","family":"Wang","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics College of Mechanical and Electrical Engineering, , , \u00a0 ,","place":["Nanjing, Jiangsu, China, 210016"]},{"id":[{"id":"https:\/\/ror.org\/0030zas98","id-type":"ROR","asserted-by":"publisher"}],"name":"The Hong Kong Polytechnic University Department of Industrial and Systems Engineering, , \u00a0 ,","place":["Hong Kong, China, 999077"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiwei","family":"Qian","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/037dym702","id-type":"ROR","asserted-by":"publisher"}],"name":"Ningbo University of Technology Robotics Institute, , , \u00a0 ,","place":["Ningbo, Zhejiang, China, 315211"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2026,5,15]]},"reference":[{"issue":"8","key":"2026051523040552000_CIT0001","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1016\/j.jmsy.2022.05.003","article-title":"Multi-Objective Complex Product Assembly Scheduling Problem Considering Parallel Team and Worker Skills","volume":"63","author":"Liu","year":"2022","journal-title":"J. Manuf. Syst."},{"issue":"11","key":"2026051523040552000_CIT0002","doi-asserted-by":"publisher","first-page":"4973","DOI":"10.1007\/s00170-023-11707-4","article-title":"Multi-Objective Production Scheduling Optimization and Management Control System of Complex Aerospace Components: A Review","volume":"127","author":"Ma","year":"2023","journal-title":"Int. J. Adv. Manuf. Technol."},{"issue":"1","key":"2026051523040552000_CIT0003","doi-asserted-by":"publisher","first-page":"012009","DOI":"10.1088\/1757-899X\/638\/1\/012009","article-title":"Research on Automatic Assembly Technology for Final Assembly of Helicopter Fuselage","volume":"638","author":"Xiao","year":",","journal-title":"IOP Conf. Ser.: Mater. Sci. Eng."},{"issue":"1","key":"2026051523040552000_CIT0004","doi-asserted-by":"publisher","first-page":"35","DOI":"10.3390\/engproc2024080035","article-title":"Intelligent Process Design System for Human\u2013Robot Collaboration in Helicopter Assembly","volume":"80","author":"Zhang","year":"2025","journal-title":"Eng. Proc."},{"key":"2026051523040552000_CIT0005","doi-asserted-by":"publisher","first-page":"106929","DOI":"10.1016\/j.cor.2024.106929","article-title":"A Literature Review of Reinforcement Learning Methods Applied to Job-Shop Scheduling Problems","volume":"175","author":"Zhang","year":"2025","journal-title":"Comput. Oper. Res."},{"key":"2026051523040552000_CIT0006","doi-asserted-by":"publisher","first-page":"4574","DOI":"10.1109\/TEM.2022.3208431","article-title":"A Multi-MILP Model Collaborative Optimization Method for Integrated Process Planning and Scheduling Problem","volume":"71","author":"Liu","year":"2022","journal-title":"IEEE Trans. Eng. Manage."},{"issue":"3","key":"2026051523040552000_CIT0007","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1016\/j.ejor.2022.01.034","article-title":"An Algorithm Selection Approach for the Flexible Job Shop Scheduling Problem: Choosing Constraint Programming Solvers Through Machine Learning","volume":"302","author":"M\u00fcller","year":"2022","journal-title":"Eur. J. Oper. Res."},{"key":"2026051523040552000_CIT0008","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1016\/j.jmsy.2023.12.008","article-title":"Automated Guided Vehicle Dispatching and Routing Integration via Digital Twin With Deep Reinforcement Learning","volume":"72","author":"Zhang","year":"2024","journal-title":"J. Manuf. Syst."},{"issue":"2","key":"2026051523040552000_CIT0009","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1080\/00207543.2019.1696487","article-title":"Ranking Dispatching Rules in Multi-Objective Dynamic Flow Shop Scheduling: A Multi-Faceted Perspective","volume":"59","author":"Oukil","year":"2021","journal-title":"Int. J. Prod. Res."},{"issue":"5","key":"2026051523040552000_CIT0010","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1080\/0951192X.2012.731612","article-title":"Heuristics to Schedule Uniform Parallel Batch Processing Machines With Dynamic Job Arrivals","volume":"26","author":"Li","year":"2013","journal-title":"Int. J. Comput. Integr. Manuf."},{"issue":"7","key":"2026051523040552000_CIT0011","doi-asserted-by":"publisher","first-page":"709","DOI":"10.1080\/0951192X.2015.1067907","article-title":"A Hybrid Meta-Heuristic Algorithm for Flowshop Robust Scheduling Under Machine Breakdown Uncertainty","volume":"29","author":"Fazayeli","year":"2016","journal-title":"Int. J. Comput. Integr. Manuf."},{"issue":"5","key":"2026051523040552000_CIT0012","doi-asserted-by":"publisher","first-page":"1303","DOI":"10.1007\/s00170-015-7987-0","article-title":"A Genetic Algorithm for Energy-Efficiency in Job-Shop Scheduling","volume":"85","author":"Salido","year":"2016","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"2026051523040552000_CIT0013","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/00207543.2024.2383785","article-title":"Distributed Assembly Shop Scheduling Problem for Complex Products Considering Multiskilled Worker Assignment and Transportation Time","author":"Gao","year":"2024","journal-title":"Int. J. Prod. Res."},{"key":"2026051523040552000_CIT0014","doi-asserted-by":"publisher","first-page":"114282","DOI":"10.1016\/j.eswa.2020.114282","article-title":"Multi-Objective Evolutionary Algorithms With Heuristic Decoding for Hybrid Flow Shop Scheduling Problem With Worker Constraint","volume":"168","author":"Han","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"2026051523040552000_CIT0015","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1016\/j.ijpe.2015.05.038","article-title":"Workforce Minimization for a Mixed-Model Assembly Line in the Automotive Industry","volume":"170","author":"Batta\u00efa","year":"2015","journal-title":"Int. J. Prod. Econ."},{"key":"2026051523040552000_CIT0016","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1016\/j.procir.2021.05.100","article-title":"Skill-Based Worker Assignment in a Manual Assembly Line","volume":"100","author":"Gr\u00e4\u00dfler","year":"2021","journal-title":"Procedia CIRP"},{"issue":"3","key":"2026051523040552000_CIT0017","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1016\/j.ejor.2016.11.027","article-title":"Traveling Worker Assembly Line (Re) Balancing Problem: Model, Reduction Techniques, and Real Case Studies","volume":"259","author":"Sikora","year":"2017","journal-title":"Eur. J. Oper. Res."},{"issue":"11","key":"2026051523040552000_CIT0018","doi-asserted-by":"publisher","first-page":"7468","DOI":"10.1109\/TII.2021.3051896","article-title":"Flexible Worker Allocation in Aircraft Final Assembly Line Using Multiobjective Evolutionary Algorithms","volume":"17","author":"Fang","year":"2021","journal-title":"IEEE Trans. Ind. Inf."},{"issue":"4","key":"2026051523040552000_CIT0019","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1108\/AA-06-2015-051","article-title":"An Adaptive BPSO Algorithm for Multi-Skilled Workers Assignment Problem in Aircraft Assembly Lines","volume":"35","author":"Xin","year":"2015","journal-title":"Assem. Autom."},{"issue":"10","key":"2026051523040552000_CIT0020","doi-asserted-by":"publisher","first-page":"2926","DOI":"10.1080\/00207543.2018.1550269","article-title":"Flow Shop Scheduling Problems With Assembly Operations: A Review and New Trends","volume":"57","author":"Komaki","year":"2019","journal-title":"Int. J. Prod. Res."},{"issue":"1","key":"2026051523040552000_CIT0021","doi-asserted-by":"publisher","first-page":"011004","DOI":"10.1115\/1.4070035","article-title":"Robust Scheduling Based on Deep Reinforcement Learning for Flexible Job Shop With Machine Breakdown and New Job Arrival","volume":"26","author":"Gan","year":"2026","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2026051523040552000_CIT0022","doi-asserted-by":"publisher","first-page":"122752","DOI":"10.1016\/j.eswa.2023.122752","article-title":"Comparison Study of Dispatching Rules and Heuristics for Online Scheduling of Single Machine Scheduling Problem With Predicted Release Time Jobs","volume":"243","author":"Xiong","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"2026051523040552000_CIT0023","doi-asserted-by":"publisher","first-page":"100664","DOI":"10.1016\/j.swevo.2020.100664","article-title":"An Improved Genetic Algorithm for the Flexible Job Shop Scheduling Problem With Multiple Time Constraints","volume":"54","author":"Zhang","year":"2020","journal-title":"Swarm Evol. Comput."},{"key":"2026051523040552000_CIT0024","doi-asserted-by":"publisher","first-page":"101485","DOI":"10.1016\/j.swevo.2024.101485","article-title":"A Genetic Algorithm With Critical Path-Based Variable Neighborhood Search for Distributed Assembly Job Shop Scheduling Problem","volume":"85","author":"Tian","year":"2024","journal-title":"Swarm Evol. Comput."},{"key":"2026051523040552000_CIT0025","doi-asserted-by":"publisher","first-page":"584","DOI":"10.1016\/j.jclepro.2018.02.004","article-title":"Efficient Multi-Objective Optimization Algorithm for Hybrid Flow Shop Scheduling Problems With Setup Energy Consumptions","volume":"181","author":"Li","year":"2018","journal-title":"J. Cleaner Prod."},{"key":"2026051523040552000_CIT0026","doi-asserted-by":"publisher","first-page":"102786","DOI":"10.1016\/j.rcim.2024.102786","article-title":"Digital Twin-Driven Dynamic Scheduling for the Assembly Workshop of Complex Products With Workers Allocation","volume":"89","author":"Gao","year":"2024","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"2026051523040552000_CIT0027","doi-asserted-by":"publisher","first-page":"122995","DOI":"10.1109\/ACCESS.2021.3110242","article-title":"Dynamic Jobshop Scheduling Algorithm Based on Deep Q Network","volume":"9","author":"Zhao","year":"2021","journal-title":"IEEE Access"},{"key":"2026051523040552000_CIT0028","doi-asserted-by":"publisher","first-page":"101776","DOI":"10.1016\/j.aei.2022.101776","article-title":"Real-Time Scheduling for Distributed Permutation Flowshops With Dynamic Job Arrivals Using Deep Reinforcement Learning","volume":"54","author":"Yang","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"2026051523040552000_CIT0029","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.jmsy.2023.09.009","article-title":"Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling Problem Considering Variable Processing Times","volume":"71","author":"Zhang","year":"2023","journal-title":"J. Manuf. Syst."},{"issue":"2","key":"2026051523040552000_CIT0030","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.cja.2023.10.011","article-title":"Efficient and Fair PPO-Based Integrated Scheduling Method for Multiple Tasks of SATech-01 Satellite","volume":"37","author":"Qi","year":"2024","journal-title":"Chin. J. Aeronaut."},{"key":"2026051523040552000_CIT0031","doi-asserted-by":"publisher","first-page":"102834","DOI":"10.1016\/j.rcim.2024.102834","article-title":"Multi-Agent Deep Reinforcement Learning for Dynamic Reconfigurable Shop Scheduling Considering Batch Processing and Worker Cooperation","volume":"91","author":"Li","year":"2025","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"2026051523040552000_CIT0032","doi-asserted-by":"publisher","first-page":"109241","DOI":"10.1016\/j.asoc.2022.109241","article-title":"Reward Criteria Impact on the Performance of Reinforcement Learning Agent for Autonomous Navigation","volume":"126","author":"Dayal","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"2026051523040552000_CIT0033","doi-asserted-by":"publisher","first-page":"694","DOI":"10.1016\/j.jmsy.2022.11.001","article-title":"Independent Double DQN-Based Multi-Agent Reinforcement Learning Approach for Online Two-Stage Hybrid Flow Shop Scheduling With Batch Machines","volume":"65","author":"Wang","year":"2022","journal-title":"J. Manuf. Syst."},{"issue":"3","key":"2026051523040552000_CIT0034","doi-asserted-by":"publisher","first-page":"2787","DOI":"10.1016\/j.aej.2021.01.030","article-title":"Solving Flow-Shop Scheduling Problem With a Reinforcement Learning Algorithm That Generalizes the Value Function With Neural Network","volume":"60","author":"Ren","year":"2021","journal-title":"Alexandria Eng. J."},{"issue":"5","key":"2026051523040552000_CIT0035","doi-asserted-by":"publisher","first-page":"051013","DOI":"10.1115\/1.4062349","article-title":"An Adaptive Job Shop Scheduling Mechanism for Disturbances by Running Reinforcement Learning in Digital Twin Environment","volume":"23","author":"Fang","year":"2023","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"05","key":"2026051523040552000_CIT0036","doi-asserted-by":"publisher","first-page":"1350014","DOI":"10.1142\/S0217595913500140","article-title":"Flow Shop Scheduling With Reinforcement Learning","volume":"30","author":"Zhang","year":"2013","journal-title":"Asia-Pac. J. Oper. Res."},{"key":"2026051523040552000_CIT0037","doi-asserted-by":"publisher","first-page":"5998","DOI":"10.65215\/ctdc8e75","article-title":"Attention Is All You Need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"2026051523040552000_CIT0038","doi-asserted-by":"publisher","first-page":"186474","DOI":"10.1109\/ACCESS.2020.3029868","article-title":"Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN","volume":"8","author":"Han","year":"2020","journal-title":"IEEE Access"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/26\/9\/091004\/7611543\/jcise-25-1693.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/26\/9\/091004\/7611543\/jcise-25-1693.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T03:04:15Z","timestamp":1778900655000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/26\/9\/091004\/1232873\/Deep-Reinforcement-Learning-for-Helicopter"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,15]]},"references-count":38,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2026,9,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4071848","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,15]]},"article-number":"091004"}}