{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T04:20:14Z","timestamp":1787545214506,"version":"build-2736575974"},"reference-count":56,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T00:00:00Z","timestamp":1787270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>Human\u2013robot collaborative assembly line balancing and scheduling constitutes an NP-hard combinatorial optimization problem involving the simultaneous optimization of task assignment, resource allocation, processing mode selection, station-level scheduling, and ergonomic constraints. This study proposes a Q-learning-based hyper-heuristic genetic algorithm (QLHH-GA) to solve the cost-oriented ergonomic mixed-model human\u2013robot collaborative assembly line balancing and scheduling problem. The proposed approach integrates bio-inspired evolutionary mechanisms of population variation and selection with adaptive, Q-learning-guided low-level heuristic selection. The Q-learning layer uses performance feedback to adapt the search strategy to different solution states while maintaining solution feasibility. A mixed-integer linear programming (MILP) model is also developed to minimize the total operating cost, including station opening, labor, robot operation, and energy consumption costs, while enforcing station-level energy expenditure (EE) limits. Computational experiments conducted using benchmark instances of varying sizes and a literature-based industrial case study demonstrate that QLHH-GA produces solutions comparable to those obtained by the MILP model on small-scale instances and maintains strong solution quality on larger instances, for which exact optimization becomes computationally prohibitive. These findings demonstrate the scalability and effectiveness of reinforcement-learning-guided hyper-heuristic search for designing cost-efficient and ergonomically constrained human\u2013robot collaborative assembly lines.<\/jats:p>","DOI":"10.3390\/biomimetics11080600","type":"journal-article","created":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T15:51:05Z","timestamp":1787327465000},"page":"600","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Q-Learning-Based Hyper-Heuristic Genetic Algorithm for Optimizing Human\u2013Robot Collaborative Assembly Lines"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3432-0099","authenticated-orcid":false,"given":"Se\u00e7il","family":"Kula\u00e7","sequence":"first","affiliation":[{"name":"Quality Coordination Office, Bursa Technical University, 16350 Bursa, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"785","DOI":"10.5267\/j.ijiec.2023.7.001","article-title":"Assembly line balancing with cobots: An extensive review and critiques","volume":"14","author":"Chutima","year":"2023","journal-title":"Int. J. Ind. Eng. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"110795","DOI":"10.1016\/j.cie.2024.110795","article-title":"Assembly line rebalancing problem with human\u2013robot collaboration and a hyper-matheuristic solution approach","volume":"200","year":"2025","journal-title":"Comput. Ind. Eng."},{"key":"ref_3","first-page":"18","article-title":"The assembly line balancing problem","volume":"6","author":"Salveson","year":"1955","journal-title":"J. Ind. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1016\/j.ifacol.2015.06.145","article-title":"Assembly line balancing with ergonomics paradigms: Two alternative methods","volume":"48","author":"Battini","year":"2015","journal-title":"IFAC-PapersOnLine"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e1548","DOI":"10.1002\/widm.1548","article-title":"Advancements in Q-learning meta-heuristic optimization algorithms: A survey","volume":"14","author":"Yang","year":"2024","journal-title":"WIREs Data Min. Knowl. Discov."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1016\/j.ejor.2004.07.023","article-title":"A survey on problems and methods in generalized assembly line balancing","volume":"168","author":"Becker","year":"2006","journal-title":"Eur. J. Oper. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.ijpe.2012.10.020","article-title":"A taxonomy of line balancing problems and their solution approaches","volume":"142","author":"Dolgui","year":"2013","journal-title":"Int. J. Prod. Econ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1007\/s10696-022-09477-4","article-title":"Balancing and sequencing of mixed-model assembly line considering preventive maintenance scenarios: Mathematical model and a migrating birds optimization algorithm","volume":"35","author":"Meng","year":"2023","journal-title":"Flex. Serv. Manuf. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"109737","DOI":"10.1016\/j.cie.2023.109737","article-title":"Human\u2013robot collaboration in assembly line balancing problems: Review and research gaps","volume":"186","author":"Kheirabadi","year":"2023","journal-title":"Comput. Ind. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"109775","DOI":"10.1016\/j.cie.2023.109775","article-title":"Balancing and scheduling human\u2013robot collaborated assembly lines with layout and objective consideration","volume":"187","author":"Nourmohammadi","year":"2024","journal-title":"Comput. Ind. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2618","DOI":"10.1080\/00207543.2021.1906461","article-title":"A two-level optimisation-simulation method for production planning and scheduling: The industrial case of a human\u2013robot collaborative assembly line","volume":"60","author":"Vieira","year":"2022","journal-title":"Int. J. Prod. Res."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shan, H., Zou, C., Qin, M., Meng, Z., and Peng, P. (2021, January 26\u201328). Research on man-robot serial cooperative two-sided assembly line balancing based on NSGA-II. Proceedings of the 2021 4th World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM 2021), Shanghai, China.","DOI":"10.1109\/WCMEIM54377.2021.00082"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"109776","DOI":"10.1016\/j.cie.2023.109776","article-title":"Solving human\u2013robot collaborative mixed-model two-sided assembly line balancing using multi-objective discrete artificial bee colony algorithm","volume":"187","author":"Huang","year":"2024","journal-title":"Comput. Ind. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1016\/j.ifacol.2019.11.473","article-title":"Assembly line balancing with collaborative robots under consideration of ergonomics: A cost-oriented approach","volume":"52","author":"Weckenborg","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.cirpj.2019.08.004","article-title":"Optimizing ergonomics in assembly lines: A multi-objective genetic algorithm","volume":"27","author":"Dini","year":"2019","journal-title":"CIRP J. Manuf. Sci. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yaphiar, S., Nugraha, C., and Ma\u2019ruf, A. (2020). Mixed model assembly line balancing for human\u2013robot shared tasks. iMEC-APCOMS 2019, Springer. Lecture Notes in Mechanical Engineering.","DOI":"10.1007\/978-981-15-0950-6_38"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"8575","DOI":"10.1007\/s00521-020-05610-2","article-title":"Multi-objective migrating bird optimization algorithm for cost-oriented assembly line balancing problem with collaborative robots","volume":"33","author":"Li","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"K\u00f6khan, S., and Bayko\u00e7, \u00d6.F. (2021). Parallel assembly lines with heterogeneous workforce: A cost-driven mathematical model and simulated annealing approach. Proceedings of the Fifteenth International Conference on Management Science and Engineering Management, Springer. Lecture Notes on Data Engineering and Communications Technologies.","DOI":"10.1007\/978-3-030-79203-9_9"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yilmaz, O., Aydin, N., and Kucukkoc, I. (2025). Integrated model assignment and multi-line balancing in human\u2013robot collaborative mixed-model assembly lines. Flex. Serv. Manuf. J., Early access.","DOI":"10.1007\/s10696-025-09635-4"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"116876","DOI":"10.1016\/j.cam.2025.116876","article-title":"Bi-objective optimization of human\u2013robot collaborative mixed-model multiple assembly lines considering model assignment and energy consumption","volume":"473","author":"Yilmaz","year":"2026","journal-title":"J. Comput. Appl. Math."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1007\/s00170-025-17076-4","article-title":"Multi-objective mixed-model assembly line balancing with human\u2013robot collaboration and fuzzy processing time","volume":"142","author":"Kang","year":"2026","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1080\/00207543.2015.1074299","article-title":"Ergonomics in assembly line balancing based on energy expenditure: A multi-objective model","volume":"54","author":"Battini","year":"2016","journal-title":"Int. J. Prod. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1080\/00207543.2021.1989077","article-title":"Balancing collaborative human\u2013robot assembly lines to optimise cycle time and ergonomic risk","volume":"60","author":"Stecke","year":"2022","journal-title":"Int. J. Prod. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.ifacol.2022.09.383","article-title":"Considering physical workload and workforce diversity in a collaborative assembly line balancing (C-ALB) optimization model","volume":"55","author":"Keshvarparast","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"ref_25","first-page":"461","article-title":"Cost-based assembly line balancing and worker-cobot assignment problem under ergonomic constraints","volume":"39","author":"Bekdemir","year":"2024","journal-title":"J. Fac. Eng. Archit. Gazi Univ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4749","DOI":"10.1080\/00207543.2024.2442546","article-title":"Task allocation of human\u2013robot collaborative assembly line considering assembly complexity and workload balance","volume":"63","author":"Cai","year":"2025","journal-title":"Int. J. Prod. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"116711","DOI":"10.1016\/j.apm.2025.116711","article-title":"Incorporating energy expenditure and rest allowance in a human\u2013robot collaborative learning-forgetting process","volume":"154","author":"Keshvarparast","year":"2026","journal-title":"Appl. Math. Model."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kula\u00e7, S., and Kiraz, A. (2026). Collaborative robot integration in mixed-model assembly line balancing with ergonomic risk considerations. Flex. Serv. Manuf. J., Early access.","DOI":"10.1007\/s10696-026-09662-9"},{"key":"ref_29","first-page":"2453","article-title":"An artificial bee colony based-hyper heuristic algorithm with local search for the assembly line balancing problems","volume":"40","author":"Aydin","year":"2024","journal-title":"Eng. Comput."},{"key":"ref_30","first-page":"577","article-title":"A hyper-heuristic approach for stochastic parallel assembly line balancing problems with equipment costs","volume":"22","year":"2022","journal-title":"Oper. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Rauf, M., Mumtaz, J., Adeel, R., Minhas, K.A., and Usman, M. (2025). An efficient Q-learning-based multi-objective intelligent hybrid genetic algorithm for mixed-model assembly line efficiency. Symmetry, 17.","DOI":"10.3390\/sym17060811"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"113110","DOI":"10.1016\/j.engappai.2025.113110","article-title":"Solving assembly line balancing problems with reinforcement learning","volume":"163","author":"Kayhan","year":"2026","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.apm.2019.02.019","article-title":"Developing optimization & robust models for a mixed-model assembly line balancing problem with semi-automated operations","volume":"72","author":"Samouei","year":"2019","journal-title":"Appl. Math. Model."},{"key":"ref_34","first-page":"1","article-title":"Designing assembly lines with humans and collaborative robots: A genetic approach","volume":"68","author":"Dini","year":"2019","journal-title":"CIRP Ann."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Abdous, M.A., Delorme, X., and Battini, D. (2020). Cobotic assembly line design problem with ergonomics. IFIP Advances in Information and Communication Technology, Springer.","DOI":"10.1007\/978-3-030-62412-5_47"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"199","DOI":"10.3901\/JME.2020.09.199","article-title":"CE-GA co-evolutionary algorithm for solving U-shaped assembly line balancing problem with man-robot cooperation","volume":"56","author":"Yifan","year":"2020","journal-title":"J. Mech. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"012021","DOI":"10.1088\/1742-6596\/1858\/1\/012021","article-title":"A mixed-integer linear programming formulation for assembly line balancing problem with human-robot shared tasks","volume":"1858","author":"Nugraha","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2901","DOI":"10.1007\/s00170-021-08068-1","article-title":"Job rotation and human\u2013robot collaboration for enhancing ergonomics in assembly lines by a genetic algorithm","volume":"118","author":"Dini","year":"2022","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1016\/j.jmsy.2022.02.005","article-title":"Harmonizing ergonomics and economics of assembly lines using collaborative robots and exoskeletons","volume":"62","author":"Weckenborg","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.cirpj.2022.11.005","article-title":"Improving ergonomics in mixed-model assembly lines balancing noise exposure and energy expenditure","volume":"40","author":"Dini","year":"2023","journal-title":"CIRP J. Manuf. Sci. Technol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"111091","DOI":"10.1016\/j.cie.2025.111091","article-title":"Research on the balancing problem of human\u2013robot collaborative assembly line in SMEs considering ergonomic risk and cost","volume":"204","author":"Yin","year":"2025","journal-title":"Comput. Ind. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kee, D. (2022). Systematic comparison of OWAS, RULA, and REBA based on a literature review. Int. J. Environ. Res. Public Health, 19.","DOI":"10.3390\/ijerph19010595"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.ifacol.2016.07.594","article-title":"The integrated assembly line balancing and parts feeding problem with ergonomics considerations","volume":"49","author":"Battini","year":"2016","journal-title":"IFAC-PapersOnLine"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Jak\u0161i\u0107, Z., Devi, S., Jak\u0161i\u0107, O., and Guha, K. (2023). A comprehensive review of bio-inspired optimization algorithms including applications in microelectronics and nanophotonics. Biomimetics, 8.","DOI":"10.3390\/biomimetics8030278"},{"key":"ref_45","first-page":"453","article-title":"A classification of hyper-heuristic approaches: Revisited","volume":"272","author":"Burke","year":"2019","journal-title":"Int. Ser. Oper. Res. Manag. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1016\/j.ejor.2019.07.073","article-title":"Recent advances in selection hyper-heuristics","volume":"285","author":"Drake","year":"2020","journal-title":"Eur. J. Oper. Res."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"112915","DOI":"10.1016\/j.eswa.2019.112915","article-title":"A genetic programming hyper-heuristic approach for the multi-skill resource constrained project scheduling problem","volume":"140","author":"Lin","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"110695","DOI":"10.1016\/j.asoc.2023.110695","article-title":"Q-learning-based hyper-heuristic evolutionary algorithm for the distributed assembly blocking flowshop scheduling problem","volume":"146","author":"Zhang","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_49","unstructured":"Perkins, D., Escobar, O.J., and Green, L. (2025). DQN performance with epsilon greedy policies and prioritized experience replay. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s40685-019-0101-y","article-title":"Balancing of assembly lines with collaborative robots","volume":"13","author":"Weckenborg","year":"2020","journal-title":"Bus. Res."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"120737","DOI":"10.1016\/j.eswa.2023.120737","article-title":"Energy aware semi-automatic assembly line balancing problem considering ergonomic risk and uncertain processing time","volume":"231","author":"Rahman","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"9095","DOI":"10.1007\/s00521-018-3957-4","article-title":"Mathematical models and migrating birds optimization for robotic U-shaped assembly line balancing problem","volume":"31","author":"Li","year":"2019","journal-title":"Neural Comput. Appl."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1444","DOI":"10.1016\/j.procir.2022.05.172","article-title":"A genetic algorithm for heterogeneous human\u2013robot collaboration assembly line balancing problems","volume":"107","author":"Nourmohammadi","year":"2022","journal-title":"Procedia CIRP"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Taguchi, G., Chowdhury, S., and Wu, Y. (2004). Taguchi\u2019s Quality Engineering Handbook, John Wiley & Sons.","DOI":"10.1002\/9780470258354"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"105674","DOI":"10.1016\/j.cor.2021.105674","article-title":"Balancing and scheduling assembly lines with human\u2013robot collaboration tasks","volume":"140","author":"Nourmohammadi","year":"2022","journal-title":"Comput. Oper. Res."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"101686","DOI":"10.1016\/j.swevo.2024.101686","article-title":"A Q-learning based artificial bee colony algorithm for solving surgery scheduling problems with setup time","volume":"90","author":"Zhang","year":"2024","journal-title":"Swarm Evol. Comput."}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/8\/600\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T04:13:55Z","timestamp":1787544835000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/8\/600"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,21]]},"references-count":56,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["biomimetics11080600"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics11080600","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,21]]}}}