{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T17:36:40Z","timestamp":1771695400735,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2021,7,2]],"date-time":"2021-07-02T00:00:00Z","timestamp":1625184000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100014188","name":"Ministry of Science and ICT, South Korea","doi-asserted-by":"publisher","award":["IITP-2021-2020-0-01821"],"award-info":[{"award-number":["IITP-2021-2020-0-01821"]}],"id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this study, based on multi-access edge computing (MEC), we provided the possibility of cooperating manufacturing processes. We tried to solve the job shop scheduling problem by applying DQN (deep Q-network), a reinforcement learning model, to this method. Here, to alleviate the overload of computing resources, an efficient DQN was used for the experiments using transfer learning data. Additionally, we conducted scheduling studies in the edge computing ecosystem of our manufacturing processes without the help of cloud centers. Cloud computing, an environment in which scheduling processing is performed, has issues sensitive to the manufacturing process in general, such as security issues and communication delay time, and research is being conducted in various fields, such as the introduction of an edge computing system that can replace them. We proposed a method of independently performing scheduling at the edge of the network through cooperative scheduling between edge devices within a multi-access edge computing structure. The proposed framework was evaluated, analyzed, and compared with existing frameworks in terms of providing solutions and services.<\/jats:p>","DOI":"10.3390\/s21134553","type":"journal-article","created":{"date-parts":[[2021,7,2]],"date-time":"2021-07-02T10:06:34Z","timestamp":1625220394000},"page":"4553","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A Novel Approach to the Job Shop Scheduling Problem Based on the Deep Q-Network in a Cooperative Multi-Access Edge Computing Ecosystem"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2280-8209","authenticated-orcid":false,"given":"Junhyung","family":"Moon","sequence":"first","affiliation":[{"name":"Department of Smart Factory Convergence, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon 16419, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0476-7730","authenticated-orcid":false,"given":"Minyeol","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Smart Factory Convergence, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon 16419, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4061-9532","authenticated-orcid":false,"given":"Jongpil","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Smart Factory Convergence, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon 16419, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1809","DOI":"10.1007\/s10845-017-1350-2","article-title":"Review of job shop scheduling research and its new perspectives under Industry 4.0","volume":"30","author":"Zhang","year":"2019","journal-title":"J. Intell. Manuf."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"106208","DOI":"10.1016\/j.asoc.2020.106208","article-title":"Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning","volume":"91","author":"Luo","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6900","DOI":"10.1109\/ACCESS.2017.2778504","article-title":"A survey on the edge computing for the Internet of Things","volume":"6","author":"Yu","year":"2017","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Angelopoulos, A., Michailidis, E.T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S., and Zahariadis, T. (2020). Tackling faults in the industry 4.0 era\u2014A survey of machine-learning solutions and key aspects. Sensors, 20.","DOI":"10.3390\/s20010109"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.procir.2019.03.041","article-title":"Design, implementation and evaluation of reinforcement learning for an adaptive order dispatching in job shop manufacturing systems","volume":"81","author":"Kuhnle","year":"2019","journal-title":"Procedia CIRP"},{"key":"ref_6","unstructured":"Gu, S., Lillicrap, T., Sutskever, I., and Levine, S. (2016, January 20\u201322). Continuous deep q-learning with model-based acceleration. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"55112","DOI":"10.1109\/ACCESS.2018.2872674","article-title":"DRL-scheduling: An intelligent QoS-aware job scheduling framework for applications in clouds","volume":"6","author":"Wei","year":"2018","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1109\/MCOM.2018.1700822","article-title":"SoftEdgeNet: SDN based energy-efficient distributed network architecture for edge computing","volume":"56","author":"Sharma","year":"2018","journal-title":"IEEE Commun. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4276","DOI":"10.1109\/TII.2019.2908210","article-title":"Smart Manufacturing Scheduling With Edge Computing Using Multiclass Deep Q Network","volume":"15","author":"Lin","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"6133","DOI":"10.1109\/TII.2020.2963910","article-title":"A trust-based team formation framework for mobile intelligence in smart factories","volume":"16","author":"Fortino","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Moon, J., and Jeong, J. (2021, January 4\u20136). Smart Manufacturing Scheduling System: DQN based on Cooperative Edge Computing. Proceedings of the 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM), Seoul, Korea.","DOI":"10.1109\/IMCOM51814.2021.9377434"},{"key":"ref_13","first-page":"94","article-title":"A scalable and quick-response software defined vehicular network assisted by mobile edge computing","volume":"55","author":"Liu","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1109\/MC.2016.302","article-title":"Responsive data architecture for the Internet of Things","volume":"49","author":"Linthicum","year":"2016","journal-title":"Computer"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1109\/JIOT.2017.2683200","article-title":"A survey on internet of things: Architecture, enabling technologies, security and privacy, and applications","volume":"4","author":"Lin","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_16","first-page":"645","article-title":"Using smart city data in 5G self-organizing networks","volume":"5","author":"Mason","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1109\/SURV.2012.010912.00035","article-title":"A survey on cyber security for smart grid communications","volume":"14","author":"Yan","year":"2012","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.future.2016.06.024","article-title":"Recovery for overloaded mobile edge computing","volume":"70","author":"Satria","year":"2017","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/cpe.5465","article-title":"EASE: Energy-efficient task scheduling for edge computing under uncertain runtime and unstable communication conditions","volume":"33","author":"Yan","year":"2021","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/JIOT.2016.2579198","article-title":"Edge computing: Vision and challenges","volume":"3","author":"Shi","year":"2016","journal-title":"IEEE Internet Things J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/MCE.2016.2590158","article-title":"Mobile-Edge Computing versus The Internet?: Looking beyond the literal meaning of MEC","volume":"5","author":"Frankston","year":"2016","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2322","DOI":"10.1109\/COMST.2017.2745201","article-title":"A survey on mobile edge computing: The communication perspective","volume":"19","author":"Mao","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1016\/j.comcom.2020.01.018","article-title":"Enhanced resource allocation in mobile edge computing using reinforcement learning based MOACO algorithm for IIOT","volume":"151","author":"Vimal","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/MVT.2013.2269187","article-title":"5G on the horizon: Key challenges for the radio-access network","volume":"8","author":"Demestichas","year":"2013","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.future.2019.02.050","article-title":"Edge computing: A survey","volume":"97","author":"Khan","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Charyyev, B., Arslan, E., and Gunes, M.H. (2020, January 7\u201311). Latency Comparison of Cloud Datacenters and Edge Servers. Proceedings of the IEEE Global Communications Conference (Globecom), Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322406"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bonomi, F., Milito, R., Natarajan, P., and Zhu, J. (2014). Fog computing: A platform for internet of things and analytics. Big Data and Internet of Things: A Roadmap for Smart Environments, Springer.","DOI":"10.1007\/978-3-319-05029-4_7"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/MCC.2016.91","article-title":"Internet of Things and edge cloud computing roadmap for manufacturing","volume":"3","author":"Georgakopoulos","year":"2016","journal-title":"IEEE Cloud Comput."},{"key":"ref_29","first-page":"1","article-title":"The internet of things: How the next evolution of the internet is changing everything","volume":"1","author":"Evans","year":"2011","journal-title":"CISCO White Pap."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yang, X., Wang, T., Ren, X., and Yu, W. (2017). Survey on improving data utility in differentially private sequential data publishing. IEEE Trans. Big Data.","DOI":"10.1109\/TBDATA.2017.2715334"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, J., Mao, Y., Zhang, J., and Letaief, K.B. (2016, January 10\u201315). Delay-optimal computation task scheduling for mobile-edge computing systems. Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541539"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MCE.2016.2590118","article-title":"Mobile-edge computing architecture: The role of MEC in the Internet of Things","volume":"5","author":"Sabella","year":"2016","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/COMST.2017.2705720","article-title":"On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration","volume":"19","author":"Taleb","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_34","unstructured":"Morris, I. (2016). ETSI Drops Mobile from MEC, Light Reading."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"107969","DOI":"10.1016\/j.comnet.2021.107969","article-title":"Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning","volume":"190","author":"Wang","year":"2021","journal-title":"Comput. Netw."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Waschneck, B., Reichstaller, A., Belzner, L., Altenm\u00fcller, T., Bauernhansl, T., Knapp, A., and Kyek, A. (May, January 30). Deep reinforcement learning for semiconductor production scheduling. Proceedings of the 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA.","DOI":"10.1109\/ASMC.2018.8373191"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Moon, J., Park, G., and Jeong, J. (2021). POP-ON: Prediction of Process Using One-Way Language Model Based on NLP Approach. Appl. Sci., 11.","DOI":"10.3390\/app11020864"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2291","DOI":"10.1016\/j.cor.2011.12.005","article-title":"A new hybrid genetic algorithm for job shop scheduling problem","volume":"39","author":"Wang","year":"2012","journal-title":"Comput. Oper. Res."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1109\/TSMCA.2007.914753","article-title":"An effective PSO and AIS-based hybrid intelligent algorithm for job-shop scheduling","volume":"38","author":"Ge","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Tseng, S.P., Tsai, C.W., Chen, J.L., Chiang, M.C., and Yang, C.S. (2011, January 27\u201330). Job shop scheduling based on ACO with a hybrid solution construction strategy. Proceedings of the 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), Taipei, Taiwan.","DOI":"10.1109\/FUZZY.2011.6007565"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1287\/opre.40.1.113","article-title":"Job shop scheduling by simulated annealing","volume":"40","author":"Aarts","year":"1992","journal-title":"Oper. Res."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_43","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2951","DOI":"10.1109\/TCYB.2016.2562674","article-title":"Surrogate-assisted genetic programming with simplified models for automated design of dispatching rules","volume":"47","author":"Nguyen","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1016\/j.cie.2018.03.039","article-title":"Real-time scheduling for a smart factory using a reinforcement learning approach","volume":"125","author":"Shiue","year":"2018","journal-title":"Comput. Ind. Eng."},{"key":"ref_46","unstructured":"Parisotto, E., Ba, J.L., and Salakhutdinov, R. (2015). Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv."},{"key":"ref_47","unstructured":"(2021, July 02). Jobshop Instance. Available online: http:\/\/jobshop.jjvh.nl\/index.php."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/MNET.001.1800510","article-title":"AI-enhanced offloading in edge computing: When machine learning meets industrial IoT","volume":"33","author":"Sun","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ding, S., and Lin, D. (2020, January 7\u201311). Dynamic Task Allocation for Cost-Efficient Edge Cloud Computing. Proceedings of the 2020 IEEE International Conference on Services Computing (SCC), Beijing, China.","DOI":"10.1109\/SCC49832.2020.00036"},{"key":"ref_50","unstructured":"Nath, S.B., Gupta, H., Chakraborty, S., and Ghosh, S.K. (2018). A survey of fog computing and communication: Current researches and future directions. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.mfglet.2018.01.005","article-title":"A fog computing industrial cyber-physical system for embedded low-latency machine learning Industry 4.0 applications","volume":"15","author":"Gallagher","year":"2018","journal-title":"Manuf. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3391196","article-title":"An overview of service placement problem in Fog and Edge Computing","volume":"53","author":"Salaht","year":"2020","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"4235","DOI":"10.1109\/TII.2019.2902878","article-title":"Artificial intelligence-driven mechanism for edge-computing-based industrial applications","volume":"15","author":"Sodhro","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/IJACI.2017070101","article-title":"Configuring a trusted cloud service model for smart city exploration using hybrid intelligence","volume":"8","author":"Sarkar","year":"2017","journal-title":"Int. J. Ambient. Comput. Intell. (IJACI)"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4553\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:25:20Z","timestamp":1760163920000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4553"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,2]]},"references-count":54,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["s21134553"],"URL":"https:\/\/doi.org\/10.3390\/s21134553","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,2]]}}}