{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:38:23Z","timestamp":1779295103359,"version":"3.51.4"},"reference-count":34,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T00:00:00Z","timestamp":1710806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSFC","award":["61903229"],"award-info":[{"award-number":["61903229"]}]},{"name":"NSFC","award":["XLYC1907166"],"award-info":[{"award-number":["XLYC1907166"]}]},{"name":"NSFC","award":["ZR2019BF004"],"award-info":[{"award-number":["ZR2019BF004"]}]},{"name":"NSFC","award":["2021-B-004"],"award-info":[{"award-number":["2021-B-004"]}]},{"name":"Liaoning Revitalization Talents Program","award":["61903229"],"award-info":[{"award-number":["61903229"]}]},{"name":"Liaoning Revitalization Talents Program","award":["XLYC1907166"],"award-info":[{"award-number":["XLYC1907166"]}]},{"name":"Liaoning Revitalization Talents Program","award":["ZR2019BF004"],"award-info":[{"award-number":["ZR2019BF004"]}]},{"name":"Liaoning Revitalization Talents Program","award":["2021-B-004"],"award-info":[{"award-number":["2021-B-004"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["61903229"],"award-info":[{"award-number":["61903229"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["XLYC1907166"],"award-info":[{"award-number":["XLYC1907166"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["ZR2019BF004"],"award-info":[{"award-number":["ZR2019BF004"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["2021-B-004"],"award-info":[{"award-number":["2021-B-004"]}]},{"name":"Archival Science and Technology Project of Liaoning Province","award":["61903229"],"award-info":[{"award-number":["61903229"]}]},{"name":"Archival Science and Technology Project of Liaoning Province","award":["XLYC1907166"],"award-info":[{"award-number":["XLYC1907166"]}]},{"name":"Archival Science and Technology Project of Liaoning Province","award":["ZR2019BF004"],"award-info":[{"award-number":["ZR2019BF004"]}]},{"name":"Archival Science and Technology Project of Liaoning Province","award":["2021-B-004"],"award-info":[{"award-number":["2021-B-004"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The scheduling of disassembly lines is of great importance to achieve optimized productivity. In this paper, we address the Hybrid Disassembly Line Balancing Problem that combines linear disassembly lines and U-shaped disassembly lines, considering multi-skilled workers, and targeting profit and carbon emissions. In contrast to common approaches in reinforcement learning that typically employ weighting strategies to solve multi-objective problems, our approach innovatively incorporates non-dominated ranking directly into the reward function. The exploration of Pareto frontier solutions or better solutions is moderated by comparing performance between solutions and dynamically adjusting rewards based on the occurrence of repeated solutions. The experimental results show that the multi-objective Advantage Actor-Critic algorithm based on Pareto optimization exhibits superior performance in terms of metrics superiority in the comparison of six experimental cases of different scales, with an excellent metrics comparison rate of 70%. In some of the experimental cases in this paper, the solutions produced by the multi-objective Advantage Actor-Critic algorithm show some advantages over other popular algorithms such as the Deep Deterministic Policy Gradient Algorithm, the Soft Actor-Critic Algorithm, and the Non-Dominated Sorting Genetic Algorithm II. This further corroborates the effectiveness of our proposed solution.<\/jats:p>","DOI":"10.3390\/info15030168","type":"journal-article","created":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T09:39:31Z","timestamp":1710841171000},"page":"168","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Multi-Objective Advantage Actor-Critic Algorithm for Hybrid Disassembly Line Balancing with Multi-Skilled Workers"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4176-3947","authenticated-orcid":false,"given":"Jiacun","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Software Engineering, Monmouth University, West Long Branch, NJ 07764, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guipeng","family":"Xi","sequence":"additional","affiliation":[{"name":"College of Information and Control Engineering, Liaoning Petrochemical University, Fushun 113001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9142-1251","authenticated-orcid":false,"given":"Xiwang","family":"Guo","sequence":"additional","affiliation":[{"name":"College of Information and Control Engineering, Liaoning Petrochemical University, Fushun 113001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4578-2726","authenticated-orcid":false,"given":"Shujin","family":"Qin","sequence":"additional","affiliation":[{"name":"College of Information and Control Engineering, Liaoning Petrochemical University, Fushun 113001, China"},{"name":"Research Center of the Economic and Social Development of Henan East Provincial Joint, Shangqiu Normal University, Shangqiu 476000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henry","family":"Han","sequence":"additional","affiliation":[{"name":"School of Engineering & Computer Science, Baylor University, Waco, TX 76798, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1849","DOI":"10.1080\/00207543.2014.965348","article-title":"A network-based shortest route model for parallel disassembly line balancing problem","volume":"53","author":"Hezer","year":"2015","journal-title":"Int. J. Prod. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1109\/TAI.2023.3266187","article-title":"Multi-objective multi-verse optimizer for multi-robotic u-shaped disassembly line balancing problems","volume":"5","author":"Qin","year":"2023","journal-title":"IEEE Trans. Artif. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"125418","DOI":"10.1016\/j.jclepro.2020.125418","article-title":"Minimizing energy consumption in multi-objective two-sided disassembly line balancing problem with complex execution constraints using dual-individual simulated annealing algorithm","volume":"284","author":"Liang","year":"2021","journal-title":"J. Clean. Prod."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1016\/j.jmsy.2020.06.013","article-title":"Multi-objective partial parallel disassembly line balancing problem using hybrid group neighborhood search algorithm","volume":"56","author":"Zhu","year":"2020","journal-title":"J. Manuf. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1115\/1.1329875","article-title":"Metrics for quality assessment of a multiobjective design optimization solution set","volume":"123","author":"Wu","year":"2001","journal-title":"J. Mech. Des."},{"key":"ref_6","unstructured":"Li, S., Wu, Y., Cui, X., Dong, H., Fang, F., and Russell, S. (February, January 27). Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_7","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning. PMLR, Stockholm, Sweden."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Deb, K., Agrawal, S., Pratap, A., and Meyarivan, T. (2000, January 18\u201320). A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Proceedings of the Parallel Problem Solving from Nature PPSN VI: 6th International Conference, Paris, France. Proceedings 6.","DOI":"10.1007\/3-540-45356-3_83"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"127145","DOI":"10.1016\/j.neucom.2023.127145","article-title":"Reinforcement learning for Hybrid Disassembly Line Balancing Problems","volume":"569","author":"Wang","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.jclepro.2018.11.114","article-title":"Modeling and optimization of multi-objective partial disassembly line balancing problem considering hazard and profit","volume":"211","author":"Wang","year":"2019","journal-title":"J. Clean. Prod."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"115431","DOI":"10.1016\/j.eswa.2021.115431","article-title":"Modelling and solving profit-oriented U-shaped partial disassembly line balancing problem","volume":"183","author":"Li","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7354","DOI":"10.1080\/00207543.2018.1471238","article-title":"A Pareto firefly algorithm for multi-objective disassembly line balancing problems with hazard evaluation","volume":"56","author":"Zhu","year":"2018","journal-title":"Int. J. Prod. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/s40684-019-00140-2","article-title":"Disassembly line balancing optimization method for high efficiency and low carbon emission","volume":"8","author":"Zhang","year":"2021","journal-title":"Int. J. Precis. Eng. Manuf.-Green Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1465","DOI":"10.1016\/j.jclepro.2019.06.035","article-title":"Multi-objective low-carbon disassembly line balancing for agricultural machinery using MDFOA and fuzzy AHP","volume":"233","author":"Yang","year":"2019","journal-title":"J. Clean. Prod."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1016\/j.ejor.2005.03.055","article-title":"A balancing method and genetic algorithm for disassembly line balancing","volume":"179","author":"McGovern","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.asoc.2018.02.032","article-title":"A variable neighborhood search heuristic algorithm for production routing problems","volume":"66","author":"Qiu","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"105400","DOI":"10.1016\/j.cor.2021.105400","article-title":"Reinforcement learning for combinatorial optimization: A survey","volume":"134","author":"Mazyavkina","year":"2021","journal-title":"Comput. Oper. Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7807","DOI":"10.1109\/LRA.2021.3098248","article-title":"Reinforcement Learning-Based Selective Disassembly Sequence Planning for the End-of-Life Products With Structure Uncertainty","volume":"6","author":"Zhao","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","first-page":"1","article-title":"Reinforcement learning for disassembly system optimization problems: A survey","volume":"2","author":"Guo","year":"2023","journal-title":"Int. J. Netw. Dyn. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2760","DOI":"10.1109\/JBHI.2022.3187950","article-title":"Multi-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events","volume":"27","author":"Shuvo","year":"2022","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Amidzadeh, M. (2023). A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysis. arXiv.","DOI":"10.1109\/CDC49753.2023.10383419"},{"key":"ref_22","first-page":"392","article-title":"Multi-objective U-shaped demolition line equilibrium problem with uncertain worker physical exertion","volume":"29","author":"Zheng","year":"2023","journal-title":"Comput.-Integr. Manuf. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2061","DOI":"10.1109\/TCSS.2022.3217101","article-title":"Multiobjective U-shaped disassembly line balancing problem considering human fatigue index and an efficient solution","volume":"10","author":"Guo","year":"2022","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.procir.2019.02.140","article-title":"Multi-robotic disassembly line balancing with uncertain processing time","volume":"83","author":"Ming","year":"2019","journal-title":"Procedia CIRP"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"122323","DOI":"10.1016\/j.eswa.2023.122323","article-title":"Two sided disassembly line balancing problem with rest time of works: A constraint programming model and an improved NSGA II algorithm","volume":"239","author":"Xu","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xu, S., Guo, X., Liu, S., Qi, L., Qin, S., Zhao, Z., and Tang, Y. (2021, January 17\u201320). Multi-objective Optimizer with Collaborative Resource Allocation Strategy for U-shaped Stochastic Disassembly Line Balancing Problem. Proceedings of the 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, Melbourne, Australia.","DOI":"10.1109\/SMC52423.2021.9659105"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3093","DOI":"10.1007\/s12555-019-0278-z","article-title":"Balance control for the first-order inverted pendulum based on the advantage actor-critic algorithm","volume":"18","author":"Zheng","year":"2020","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_28","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"934","DOI":"10.1109\/JAS.2018.7511165","article-title":"A dynamic road incident information delivery strategy to reduce urban traffic congestion","volume":"5","author":"Qi","year":"2018","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ishibuchi, H., Masuda, H., Tanigaki, Y., and Nojima, Y. (April, January 29). Modified distance calculation in generational distance and inverted generational distance. Proceedings of the Evolutionary Multi-Criterion Optimization: 8th International Conference, EMO 2015, Guimar\u00e3es, Portugal. Proceedings, Part II 8.","DOI":"10.1007\/978-3-319-15892-1_8"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/2331554","article-title":"The probable error of a mean","volume":"6","author":"Gosset","year":"1908","journal-title":"Biometrika"},{"key":"ref_32","first-page":"52","article-title":"2-stage optimal design and analysis for disassembly system with environmental and economic parts selection using the recyclability evaluation method","volume":"13","author":"Igarashi","year":"2014","journal-title":"Ind. Eng. Manag. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.wasman.2022.04.015","article-title":"Multi-objective optimisation for cell-level disassembly of waste power battery modules in human-machine hybrid mode","volume":"144","author":"Wu","year":"2022","journal-title":"Waste Manag."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1051","DOI":"10.1109\/TETCI.2022.3146882","article-title":"Deep reinforcement learning based adaptive operator selection for evolutionary multi-objective optimization","volume":"7","author":"Tian","year":"2022","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/3\/168\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:16:01Z","timestamp":1760105761000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/3\/168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,19]]},"references-count":34,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["info15030168"],"URL":"https:\/\/doi.org\/10.3390\/info15030168","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,19]]}}}