{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:18:17Z","timestamp":1760231897218,"version":"build-2065373602"},"reference-count":53,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T00:00:00Z","timestamp":1665100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>We developed a self-optimizing decision system that dynamically minimizes the overall energy consumption of an industrial process. Our model is based on a deep reinforcement learning (DRL) framework, adopting three reinforcement learning methods, namely: deep Q-network (DQN), proximal policy optimization (PPO), and advantage actor\u2013critic (A2C) algorithms, combined with a self-predicting random forest model. This smart decision system is a physics-informed DRL that sets the key industrial input parameters to optimize energy consumption while ensuring the product quality based on desired output parameters. The system is self-improving and can increase its performances without further human assistance. We applied the approach to the process of heating tempered glass. Indeed, the identification and control of tempered glass parameters is a challenging task requiring expertise. In addition, optimizing energy consumption while dealing with this issue is of great value-added. The evaluation of the decision system under the three configurations has been performed and consequently, outcomes and conclusions have been explained in this paper. Our intelligent decision system provides an optimized set of parameters for the heating process within the acceptance limits while minimizing overall energy consumption. This work provides the necessary foundations to address energy optimization issues related to process parameterization from theory to practice and providing real industrial application; further research opens a new horizon towards intelligent and sustainable manufacturing.<\/jats:p>","DOI":"10.3390\/systems10050180","type":"journal-article","created":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T03:32:56Z","timestamp":1665459176000},"page":"180","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["A Learning-Based Decision Tool towards Smart Energy Optimization in the Manufacturing Process"],"prefix":"10.3390","volume":"10","author":[{"given":"Choumicha","family":"El Mazgualdi","sequence":"first","affiliation":[{"name":"Laboratory of Mathematical Modeling, Simulation and Smart Systems (L2M3S), Artificial Intelligence for Engineering Sciences Team (IASI), Department of Mathematics and Computer Science, ENSAM-Meknes, Moulay Ismail University, Meknes 50050, Morocco"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7761-6300","authenticated-orcid":false,"given":"Tawfik","family":"Masrour","sequence":"additional","affiliation":[{"name":"Laboratory of Mathematical Modeling, Simulation and Smart Systems (L2M3S), Artificial Intelligence for Engineering Sciences Team (IASI), Department of Mathematics and Computer Science, ENSAM-Meknes, Moulay Ismail University, Meknes 50050, Morocco"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2330-8326","authenticated-orcid":false,"given":"Noureddine","family":"Barka","sequence":"additional","affiliation":[{"name":"Mathematics, Computer Science and Engineering Department, Universit\u00e9 du Qu\u00e9bec \u00e0 Rimouski, 300 All\u00e9e des Ursulines, Rimouski, QC G5L 3A1, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4073-6904","authenticated-orcid":false,"given":"Ibtissam","family":"El Hassani","sequence":"additional","affiliation":[{"name":"Laboratory of Mathematical Modeling, Simulation and Smart Systems (L2M3S), Artificial Intelligence for Engineering Sciences Team (IASI), Department of Mathematics and Computer Science, ENSAM-Meknes, Moulay Ismail University, Meknes 50050, Morocco"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,7]]},"reference":[{"key":"ref_1","first-page":"99","article-title":"Optimum design of a CCHP system based on Economical, energy and environmental considerations using GA and PSO","volume":"9","author":"Rabbani","year":"2018","journal-title":"Int. J. Ind. Eng. Comput."},{"key":"ref_2","unstructured":"Thiede, S., and Herrmann, C. (2010, January 19\u201321). Simulation-based energy flow evaluation for sustainable manufacturing systems. Proceedings of the 17th CIRP International Conference on Life Cycle Engineering, LCE 2010, Hefei, China."},{"key":"ref_3","unstructured":"Seefeldt, F., Marco, W., and Schlesinger, M. (2007). The Future Role of Coal in Europe, EUROCOAL."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.engappai.2018.11.007","article-title":"Industry 4.0: A bibliometric analysis and detailed overview","volume":"78","author":"Muhuri","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.jmsy.2020.06.010","article-title":"Smart manufacturing process and system automation-a critical review of the standards and envisioned scenarios","volume":"56","author":"Lu","year":"2020","journal-title":"J. Manuf. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"104232","DOI":"10.1016\/j.engappai.2021.104232","article-title":"A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications","volume":"101","author":"Zobeiry","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_7","unstructured":"Chakraborty, S. (2020). Simulation free reliability analysis: A physics-informed deep learning based approach. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_9","first-page":"1","article-title":"Algorithms for reinforcement learning","volume":"4","year":"2010","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_11","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1002\/9780470310533.ch9","article-title":"Fundamentals of tempered glass","volume":"10","author":"Mcmaster","year":"1989","journal-title":"Ceram. Eng. Sci. Proc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.engstruct.2014.06.014","article-title":"Experimental investigation of monolithic tempered glass fragment characteristics subjected to blast loads","volume":"75","author":"Zhang","year":"2014","journal-title":"Eng. Struct."},{"key":"ref_14","unstructured":"Uhlmann, D.R., and Kreidl, N.J. (2014). Thermal tempering of glass. Glass Science and Technology, Elsevier."},{"key":"ref_15","unstructured":"Rantala, M. (2015). Heat Transfer Phenomena in Float Glass Heat Treatment Processes. [Doctoral Thesis, Tampere University of Technology]."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mazgualdi, C.E., Masrour, T., Hassani, E., and Khdoudi, A. (2020, January 19\u201320). A Deep Reinforcement Learning (DRL) Decision Model for Heating Process Parameters Identification in Automotive Glass Manufacturing. Proceedings of the International Conference on Artificial Intelligence & Industrial Applications, Meknes, Morocco.","DOI":"10.1007\/978-3-030-51186-9_6"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rcim.2019.01.011","article-title":"Energy-Efficient machining process analysis and optimisation based on BS EN24T alloy steel as case studies","volume":"58","author":"Moreira","year":"2019","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3648","DOI":"10.1016\/j.asoc.2012.06.006","article-title":"Soft computing based multi-objective optimization of steam cycle power plant using NSGA-II and ANN","volume":"12","author":"Hajabdollahi","year":"2012","journal-title":"Appl. Soft Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"115077","DOI":"10.1016\/j.apenergy.2020.115077","article-title":"Optimal demand response operation of electric boosting glass furnaces","volume":"269","author":"Seo","year":"2020","journal-title":"Appl. Energy"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"116851","DOI":"10.1016\/j.energy.2019.116851","article-title":"Energy optimization and prediction modeling of petrochemical industries: An improved convolutional neural network based on cross-feature","volume":"194","author":"Geng","year":"2020","journal-title":"Energy"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Su, Y. (2020). Multi-objective optimization of cutting parameters in turning AISI 304 austenitic stainless steel. Metals, 10.","DOI":"10.3390\/met10020217"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.procir.2019.01.022","article-title":"Multi-objective optimization for energy efficient machining with high productivity and quality for a turning process","volume":"80","author":"Sangwan","year":"2019","journal-title":"Procedia CIRP"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"118714","DOI":"10.1016\/j.jclepro.2019.118714","article-title":"Dual-objective program and improved artificial bee colony for the optimization of energy-conscious milling parameters subject to multiple constraints","volume":"245","author":"Wang","year":"2020","journal-title":"J. Clean. Prod."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.promfg.2018.07.046","article-title":"Trade-off analysis of tool wear, machining quality and energy efficiency of alloy cast iron milling process","volume":"26","author":"Luan","year":"2018","journal-title":"Procedia Manuf."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.enconman.2018.03.049","article-title":"Energy management and optimization modeling based on a novel fuzzy extreme learning machine: Case study of complex petrochemical industries","volume":"165","author":"Han","year":"2018","journal-title":"Energy Convers. Manag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.compchemeng.2017.11.020","article-title":"Support vector regression modelling and optimization of energy consumption in carbon fiber production line","volume":"109","author":"Golkarnarenji","year":"2018","journal-title":"Comput. Chem. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.jmsy.2021.08.009","article-title":"Machine learning-based real-time monitoring system for smart connected worker to improve energy efficiency","volume":"61","author":"Bian","year":"2021","journal-title":"J. Manuf. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"112963","DOI":"10.1016\/j.eswa.2019.112963","article-title":"Application of deep reinforcement learning to intrusion detection for supervised problems","volume":"141","author":"Carro","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.engappai.2018.11.006","article-title":"Reinforcement learning based compensation methods for robot manipulators","volume":"78","author":"Pane","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_30","first-page":"1","article-title":"Vnf service chain deployment algorithm in 5g communication based on reinforcement learning","volume":"48","author":"Xuan","year":"2020","journal-title":"IAENG Int. J. Comput. Sci."},{"key":"ref_31","first-page":"1","article-title":"Smart Autonomous Vehicles in High Dimensional Warehouses Using Deep Reinforcement Learning Approach","volume":"29","author":"Rhazzaf","year":"2021","journal-title":"Eng. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wiering, M., and Otterlo, M. (2012). Reinforcement learning and Markov decision processes. Reinforcement Learning: State-Of-The-Art, Springer.","DOI":"10.1007\/978-3-642-27645-3"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0377-2217(89)90348-2","article-title":"Markov decision processes","volume":"39","author":"White","year":"1989","journal-title":"Eur. J. Operational Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1126\/science.153.3731.34","article-title":"Dynamic programming","volume":"153","author":"Bellman","year":"1966","journal-title":"Science"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_36","unstructured":"Werner, H., and Ehn, G. (2018). Reinforcement Learning for Planning of a Simulated Production Line. [Master\u2019s Theses, Lund Central Station]."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/2.485891","article-title":"Artificial neural networks: A tutorial","volume":"29","author":"Jain","year":"1996","journal-title":"Computer"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Khdoudi, A., Masrour, T., and Mazgualdi, C. (2019, January 8\u201311). Using Machine Learning Algorithms for the Prediction of Industrial Process Parameters Based on Product Design. Proceedings of the International Conference on Advanced Intelligent Systems for Sustainable Development, Marrakech, Morocco.","DOI":"10.1007\/978-3-030-36671-1_67"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_40","first-page":"1057","article-title":"Policy gradient methods for reinforcement learning with function approximation","volume":"12","author":"Sutton","year":"1999","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_41","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6\u201311). Trust region policy optimization. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_42","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_43","first-page":"1","article-title":"Enhancing A Stock Timing Strategy by Reinforcement Learning","volume":"48","author":"Li","year":"2021","journal-title":"IAENG Int. J. Comput. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1109\/TSMCC.2012.2218595","article-title":"A survey of actor\u2013critic reinforcement learning: Standard and natural policy gradients","volume":"42","author":"Grondman","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_45","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_46","unstructured":"Lee, H.H. (2018). Finite Element Simulations with ANSYS Workbench 18, SDC Publications."},{"key":"ref_47","unstructured":"Bergman, T.L., Lavine, A.S., and Incropera, F.P. (2011). Introduction to Heat Transfer, John Wiley & Sons."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1146\/annurev.fluid.30.1.365","article-title":"Boiling heat transfer","volume":"30","author":"Dhir","year":"1998","journal-title":"Annu. Rev. Fluid Mech."},{"key":"ref_49","unstructured":"Mills, A.F. (1992). Heat Transfer, CRC Press."},{"key":"ref_50","first-page":"272","article-title":"Random forests and decision trees","volume":"9","author":"Ali","year":"2012","journal-title":"Int. J. Comput. Sci. Issues"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1007\/s00500-020-05348-y","article-title":"Machine learning for KPIs prediction: A case study of the overall equipment effectiveness within the automotive industry","volume":"25","author":"Mazgualdi","year":"2021","journal-title":"Soft Comput."},{"key":"ref_52","unstructured":"Dhariwal, P., Hesse, C., Klimov, O., Nichol, A., Plappert, M., Radford, A., Schulman, J., Sidor, S., Wu, Y., and Zhokhov, P. (2022, September 05). Openai Baselines: High-Quality Implementations of Reinforcement Learning Algorithms. Available online: https:\/\/github.com\/openai\/baselines."},{"key":"ref_53","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/10\/5\/180\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:47:42Z","timestamp":1760143662000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/10\/5\/180"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,7]]},"references-count":53,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["systems10050180"],"URL":"https:\/\/doi.org\/10.3390\/systems10050180","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2022,10,7]]}}}