{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T20:20:36Z","timestamp":1774297236527,"version":"3.50.1"},"reference-count":67,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T00:00:00Z","timestamp":1708646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The manufacturing industry often faces challenges related to customer satisfaction, system degradation, product sustainability, inventory, and operation management. If not addressed, these challenges can be substantially harmful and costly for the sustainability of manufacturing plants. Paradigms, e.g., Industry 4.0 and smart manufacturing, provide effective and innovative solutions, aiming at managing manufacturing operations, and controlling the quality of completed goods offered to the customers. Aiming at that end, this paper endeavors to mitigate the described challenges in a multi-stage degrading manufacturing\/remanufacturing system through the implementation of an intelligent machine learning-based decision-making mechanism. To carry out decision-making, reinforcement learning is coupled with lean green manufacturing. The scope of this implementation is the creation of a smart lean and sustainable production environment that has a minimal environmental impact. Considering the latter, this effort is made to reduce material consumption and extend the lifecycle of manufactured products using pull production, predictive maintenance, and circular economy strategies. To validate this, a well-defined experimental analysis meticulously investigates the behavior and performance of the proposed mechanism. Results obtained by this analysis support the presented reinforcement learning\/ad hoc control mechanism\u2019s capability and competence achieving both high system sustainability and enhanced material reuse.<\/jats:p>","DOI":"10.3390\/a17030098","type":"journal-article","created":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T10:47:30Z","timestamp":1708685250000},"page":"98","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Reinforcement Learning-Based Optimization for Sustainable and Lean Production within the Context of Industry 4.0"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0979-9643","authenticated-orcid":false,"given":"Panagiotis D.","family":"Paraschos","sequence":"first","affiliation":[{"name":"Department of Production and Management Engineering, Democritus University of Thrace, 67100 Xanthi, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0323-6954","authenticated-orcid":false,"given":"Georgios K.","family":"Koulinas","sequence":"additional","affiliation":[{"name":"Department of Production and Management Engineering, Democritus University of Thrace, 67100 Xanthi, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0194-0654","authenticated-orcid":false,"given":"Dimitrios E.","family":"Koulouriotis","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, National Technical University of Athens, 15772 Athens, Greece"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.procir.2013.05.002","article-title":"Evolving Paradigms of Manufacturing: From Mass Production to Mass Customization and Personalization","volume":"7","author":"Hu","year":"2013","journal-title":"Procedia CIRP"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"101007","DOI":"10.1016\/j.infoecopol.2022.101007","article-title":"Impact of digitalization and environmental regulation on total factor productivity","volume":"61","author":"Wen","year":"2022","journal-title":"Inf. Econ. Policy"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1016\/j.wasman.2016.11.038","article-title":"From waste to sustainable materials management: Three case studies of the transition journey","volume":"61","author":"Silva","year":"2017","journal-title":"Waste Manag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3255","DOI":"10.1002\/bse.2801","article-title":"Enabling the Circular Economy transition: A sustainable lean manufacturing recipe for Industry 4.0","volume":"30","author":"Ciliberto","year":"2021","journal-title":"Bus. Strateg. Environ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"103298","DOI":"10.1016\/j.compind.2020.103298","article-title":"Machine learning and reasoning for predictive maintenance in Industry 4.0: Current status and challenges","volume":"123","author":"Dalzochio","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.ijpe.2018.08.019","article-title":"The expected contribution of Industry 4.0 technologies for industrial performance","volume":"204","author":"Dalenogare","year":"2018","journal-title":"Int. J. Prod. Econ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6505","DOI":"10.1109\/ACCESS.2017.2783682","article-title":"Smart Factory of Industry 4.0: Key Technologies, Application Case, and Challenges","volume":"6","author":"Chen","year":"2018","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ijpe.2019.01.004","article-title":"Industry 4.0 technologies: Implementation patterns in manufacturing companies","volume":"210","author":"Frank","year":"2019","journal-title":"Int. J. Prod. Econ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"103688","DOI":"10.1016\/j.compind.2022.103688","article-title":"Literature review and methodological framework for integration of IoT and PLM in manufacturing industry","volume":"140","author":"Barrios","year":"2022","journal-title":"Comput. Ind."},{"key":"ref_10","first-page":"69","article-title":"A data-driven scheduling approach to smart manufacturing","volume":"15","author":"Rossit","year":"2019","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Demertzi, V., Demertzis, S., and Demertzis, K. (2023). An Overview of Privacy Dimensions on the Industrial Internet of Things (IIoT). Algorithms, 16.","DOI":"10.3390\/a16080378"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"103529","DOI":"10.1016\/j.compind.2021.103529","article-title":"Coupling digital simulation and machine learning metamodel through an active learning approach in Industry 4.0 context","volume":"133","author":"Chabanet","year":"2021","journal-title":"Comput. Ind."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103390","DOI":"10.1016\/j.compind.2020.103390","article-title":"Technology enablers for the implementation of Industry 4.0 to traditional manufacturing sectors: A review","volume":"125","author":"Azariadis","year":"2021","journal-title":"Comput. Ind."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1108\/BIJ-12-2018-0444","article-title":"Industry 4.0: Defining the research agenda","volume":"28","year":"2021","journal-title":"Benchmarking Int. J."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Queiroz, G.A., Alves Junior, P.N., and Costa Melo, I. (2022). Digitalization as an Enabler to SMEs Implementing Lean-Green? A Systematic Review through the Topic Modelling Approach. Sustainability, 14.","DOI":"10.3390\/su142114089"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1080\/19397038.2021.1970855","article-title":"Integral measures and framework for green lean six sigma implementation in manufacturing environment","volume":"14","author":"Yadav","year":"2021","journal-title":"Int. J. Sustain. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1016\/j.proeng.2014.12.341","article-title":"A Review on Lean Manufacturing Implementation Techniques","volume":"97","author":"Sundar","year":"2014","journal-title":"Procedia Eng."},{"key":"ref_18","first-page":"44","article-title":"A framework for lean manufacturing implementation","volume":"1","author":"Mostafa","year":"2013","journal-title":"Prod. Manuf. Res."},{"key":"ref_19","first-page":"241","article-title":"A literature review of lean manufacturing","volume":"8","author":"Gupta","year":"2013","journal-title":"Int. J. Manag. Sci. Eng. Manag."},{"key":"ref_20","first-page":"45","article-title":"A framework to improve construction processes: Integrating Lean, Green and Six Sigma","volume":"14","author":"Banawi","year":"2014","journal-title":"Int. J. Constr. Manag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"131130","DOI":"10.1016\/j.jclepro.2022.131130","article-title":"Green Lean Six Sigma for improving manufacturing sustainability: Framework development and validation","volume":"345","author":"Rathi","year":"2022","journal-title":"J. Clean. Prod."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"128691","DOI":"10.1016\/j.jclepro.2021.128691","article-title":"An integrated smart, green, resilient, and lean manufacturing framework: A literature review and future research directions","volume":"319","author":"Touriki","year":"2021","journal-title":"J. Clean. Prod."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1002\/bse.2897","article-title":"Integration of green and lean practices for sustainable business management","volume":"31","author":"Singh","year":"2022","journal-title":"Bus. Strateg. Environ."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s41660-019-00082-x","article-title":"Lean and Green Manufacturing\u2014A Review on its Applications and Impacts","volume":"3","author":"Leong","year":"2019","journal-title":"Process Integr. Optim. Sustain."},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.]."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1016\/j.jmsy.2020.07.004","article-title":"Reinforcement learning for combined production-maintenance and quality control of a manufacturing system with deterioration failures","volume":"56","author":"Paraschos","year":"2020","journal-title":"J. Manuf. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Paraschos, P.D., Koulinas, G.K., and Koulouriotis, D.E. (2023). A reinforcement learning\/ad-hoc planning and scheduling mechanism for flexible and sustainable manufacturing systems. Flex. Serv. Manuf. J.","DOI":"10.1007\/s10696-023-09496-9"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1108\/JMTM-12-2018-0446","article-title":"Industry 4.0 and Lean Manufacturing","volume":"32","author":"Pagliosa","year":"2019","journal-title":"J. Manuf. Technol. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.promfg.2021.10.059","article-title":"A machine learning-based framework for data mining and optimization of a production system","volume":"55","author":"Koulinas","year":"2021","journal-title":"Procedia Manuf."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1016\/j.promfg.2021.10.055","article-title":"Parametric and reinforcement learning control for degrading multi-stage systems","volume":"55","author":"Paraschos","year":"2021","journal-title":"Procedia Manuf."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1108\/TQM-05-2022-0164","article-title":"Integrating Industry 4.0 and Total Productive Maintenance for global sustainability","volume":"36","author":"Samadhiya","year":"2022","journal-title":"TQM J."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1507","DOI":"10.1007\/s00170-014-6052-8","article-title":"Multi-objective optimization of production control mechanisms for multi-stage serial manufacturing-inventory systems","volume":"74","author":"Xanthopoulos","year":"2014","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1439","DOI":"10.1016\/j.promfg.2020.10.200","article-title":"A Decision Trees-based knowledge mining approach for controlling a complex production system","volume":"51","author":"Koulinas","year":"2020","journal-title":"Procedia Manuf."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"128847","DOI":"10.1016\/j.jclepro.2021.128847","article-title":"Smart circular product design strategies towards eco-effective production systems: A lean eco-design industry 4.0 framework","volume":"320","author":"Dahmani","year":"2021","journal-title":"J. Clean. Prod."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1016\/j.spc.2021.01.001","article-title":"Leveraging Optimized and Cleaner Production through Industry 4.0","volume":"26","author":"Amjad","year":"2021","journal-title":"Sustain. Prod. Consum."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cirpj.2010.03.006","article-title":"Sustainable manufacturing: Modeling and optimization challenges at the product, process and system levels","volume":"2","author":"Jayal","year":"2010","journal-title":"CIRP J. Manuf. Sci. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Aruanno, B. (2024). EcoPrintAnalyzer: Assessing Sustainability in Material Extrusion Additive Manufacturing for Informed Decision-Making. Sustainability, 16.","DOI":"10.3390\/su16020615"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1016\/j.psep.2018.05.009","article-title":"Sustainable Industry 4.0 framework: A systematic literature review identifying the current trends and future perspectives","volume":"117","author":"Kamble","year":"2018","journal-title":"Process Saf. Environ. Prot."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"7046","DOI":"10.1080\/00207543.2019.1574042","article-title":"Social sustainable supplier evaluation and selection: A group decision-support approach","volume":"57","author":"Bai","year":"2019","journal-title":"Int. J. Prod. Res."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1108\/13598541211227126","article-title":"Green supply chain management practices: Impact on performance","volume":"17","author":"Green","year":"2012","journal-title":"Supply Chain Manag."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"744","DOI":"10.1016\/j.jclepro.2017.08.006","article-title":"The concept of sustainable manufacturing and its definitions: A content-analysis based literature review","volume":"166","author":"Moldavska","year":"2017","journal-title":"J. Clean. Prod."},{"key":"ref_42","first-page":"100107","article-title":"Deep learning for manufacturing sustainability: Models, applications in Industry 4.0 and implications","volume":"2","author":"Jamwal","year":"2022","journal-title":"Int. J. Inf. Manag. Data Insights"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"112128","DOI":"10.1016\/j.rser.2022.112128","article-title":"Data-driven probabilistic machine learning in sustainable smart energy\/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm","volume":"160","author":"Ahmad","year":"2022","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1007\/s10479-022-04783-4","article-title":"The capacitated lot-sizing and energy efficient single machine scheduling problem with sequence dependent setup times and costs in a closed-loop supply chain network","volume":"321","author":"Roshani","year":"2023","journal-title":"Ann. Oper. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jmsy.2022.02.002","article-title":"Extending the lean value stream mapping to the context of Industry 4.0: An agent-based technology approach","volume":"63","author":"Ferreira","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"108916","DOI":"10.1016\/j.ijpe.2023.108916","article-title":"Implications of using Industry 4.0 base technologies for lean and agile supply chains and performance","volume":"262","author":"Carvalho","year":"2023","journal-title":"Int. J. Prod. Econ."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"135731","DOI":"10.1016\/j.jclepro.2022.135731","article-title":"Development of an advanced application process of Lean Manufacturing approach based on a new integrated MCDM method under Pythagorean fuzzy environment","volume":"386","author":"Soltani","year":"2023","journal-title":"J. Clean. Prod."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"129813","DOI":"10.1016\/j.jclepro.2021.129813","article-title":"Environmental and economic sustainability through innovative green products by remanufacturing","volume":"332","author":"Sarkar","year":"2022","journal-title":"J. Clean. Prod."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Liu, B., and De Giovanni, P. (2019). Green process innovation through Industry 4.0 technologies and supply chain coordination. Ann. Oper. Res., 1\u201336.","DOI":"10.1007\/s10479-019-03498-3"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"123223","DOI":"10.1016\/j.jclepro.2020.123223","article-title":"Green lean operationalisation of the circular economy concept on production shop floor level","volume":"278","author":"Kurdve","year":"2021","journal-title":"J. Clean. Prod."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Tripathi, V., Chattopadhyaya, S., Mukhopadhyay, A.K., Sharma, S., Singh, J., Pimenov, D.Y., and Giasin, K. (2021). An Innovative Agile Model of Smart Lean\u2013Green Approach for Sustainability Enhancement in Industry 4.0. J. Open Innov. Technol. Mark. Complex., 7.","DOI":"10.3390\/joitmc7040215"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Duarte, S., and Cruz-Machado, V. (2017, January 24\u201325). An investigation of lean and green supply chain in the Industry 4.0. Proceedings of the 2017 International Symposium on Industrial Engineering and Operations Management (IEOM), Bristol, UK.","DOI":"10.1007\/978-3-319-59280-0_103"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"102321","DOI":"10.1016\/j.rcim.2022.102321","article-title":"AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop","volume":"76","author":"Li","year":"2022","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_54","first-page":"282","article-title":"Reinforcement Learning Approach for Multi-period Inventory with Stochastic Demand","volume":"Volume 646","author":"Shakya","year":"2022","journal-title":"AIAI 2022: Artificial Intelligence Applications and Innovations"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Matrenin, P.V. (2022). Improvement of Ant Colony Algorithm Performance for the Job-Shop Scheduling Problem Using Evolutionary Adaptation and Software Realization Heuristics. Algorithms, 16.","DOI":"10.3390\/a16010015"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"905","DOI":"10.1007\/s10845-021-01847-3","article-title":"Reinforcement learning applications to machine scheduling problems: A comprehensive literature review","volume":"34","author":"Kayhan","year":"2023","journal-title":"J. Intell. Manuf."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"105823","DOI":"10.1016\/j.cor.2022.105823","article-title":"Digital twin-enabled dynamic scheduling with preventive maintenance using a double-layer Q-learning algorithm","volume":"144","author":"Yan","year":"2022","journal-title":"Comput. Oper. Res."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"117380","DOI":"10.1016\/j.eswa.2022.117380","article-title":"A reinforcement learning based RMOEA\/D for bi-objective fuzzy flexible job shop scheduling","volume":"203","author":"Li","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"108406","DOI":"10.1016\/j.cie.2022.108406","article-title":"Using real-time manufacturing data to schedule a smart factory via reinforcement learning","volume":"171","author":"Gu","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"107971","DOI":"10.1016\/j.cie.2022.107971","article-title":"Machine learning integrated design and operation management for resilient circular manufacturing systems","volume":"167","author":"Paraschos","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1504\/IJMTM.2010.032894","article-title":"An investigation of the influence of coefficient of variation in the demand distribution on the performance of several lean production control strategies","volume":"20","author":"Geraghty","year":"2010","journal-title":"Int. J. Manuf. Technol. Manag."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Axs\u00e4ter, S. (2015). Inventory Control, Springer International Publishing. [3rd ed.].","DOI":"10.1007\/978-3-319-15729-0"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1018919806139","article-title":"Comparison among three pull control policies: Kanban, base stock, and generalized kanban","volume":"93","author":"Duri","year":"2000","journal-title":"Ann. Oper. Res."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1080\/07408170008963914","article-title":"Extended kanban control system: Combining kanban and base stock","volume":"32","author":"Dallery","year":"2000","journal-title":"IIE Trans."},{"key":"ref_65","first-page":"2258","article-title":"Efficient Average Reward Reinforcement Learning Using Constant Shifting Values","volume":"30","author":"Yang","year":"2016","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_66","first-page":"313","article-title":"Reinforcement learning-based adaptive production control of pull manufacturing systems","volume":"36","author":"Xanthopoulos","year":"2019","journal-title":"J. Ind. Prod. Eng."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.jclepro.2015.04.064","article-title":"Lean and green\u2014A systematic review of the state of the art literature","volume":"102","year":"2015","journal-title":"J. Clean. Prod."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/98\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:04:03Z","timestamp":1760105043000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/98"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,23]]},"references-count":67,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["a17030098"],"URL":"https:\/\/doi.org\/10.3390\/a17030098","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,23]]}}}