{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T10:14:48Z","timestamp":1781518488582,"version":"3.54.1"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T00:00:00Z","timestamp":1641600000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T00:00:00Z","timestamp":1641600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"national natural science foundation of china","doi-asserted-by":"publisher","award":["62072146"],"award-info":[{"award-number":["62072146"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"zhejiang provincial key technology research and development program","award":["2019C01059"],"award-info":[{"award-number":["2019C01059"]}]},{"name":"national key technology research and development program of china","award":["2019YFB2102100"],"award-info":[{"award-number":["2019YFB2102100"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Edge computing is a new paradigm for providing cloud computing capacities at the edge of network near mobile users. It offers an effective solution to help mobile devices with computation-intensive and delay-sensitive tasks. However, the edge of network presents a dynamic environment with large number of devices, high mobility of users, heterogeneous applications and intermittent traffic. In such environment, edge computing often suffers from unbalance resource allocation, which leads to task failure and affects system performance. To tackle this problem, we proposed a deep reinforcement learning(DRL)-based workload scheduling approach with the goal of balancing the workload, reducing the service time and the failed task rate. Meanwhile, We adopt Deep-Q-Network(DQN) algorithms to solve the complexity and high dimension of workload scheduling problem. Simulation results show that our proposed approach achieves the best performance in aspects of service time, virtual machine(VM) utilization, and failed tasks rate compared with other approaches. Our DRL-based approach can provide an efficient solution to the workload scheduling problem in edge computing.<\/jats:p>","DOI":"10.1186\/s13677-021-00276-0","type":"journal-article","created":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T21:25:50Z","timestamp":1641677150000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":99,"title":["Deep Reinforcement Learning-Based Workload Scheduling for Edge Computing"],"prefix":"10.1186","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5363-8137","authenticated-orcid":false,"given":"Tao","family":"Zheng","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Wan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jilin","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Congfeng","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,1,8]]},"reference":[{"issue":"99","key":"276_CR1","first-page":"1","volume":"PP","author":"Y Mao","year":"2017","unstructured":"Mao Y, et al. (2017) A Survey on Mobile Edge Computing: The Communication Perspective. IEEE Commun Surv Tutor PP(99):1\u20131.","journal-title":"IEEE Commun Surv Tutor"},{"issue":"1","key":"276_CR2","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/s11276-016-1322-z","volume":"24","author":"X Guo","year":"2018","unstructured":"Guo X, et al. (2018) Data offloading and task allocation for cloudlet-assisted ad hoc mobile clouds. Wirel Netw 24(1):79\u201388.","journal-title":"Wirel Netw"},{"issue":"1","key":"276_CR3","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1016\/j.ins.2014.06.022","volume":"283","author":"W Fang","year":"2014","unstructured":"Fang W, Li Y, Zhang H, Xiong N, Lai J, Vasilakos AV (2014) On the throughput-energy tradeoff for data transmission between cloud and mobile devices. Inf Sci 283(1):79\u201393.","journal-title":"Inf Sci"},{"issue":"1","key":"276_CR4","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1109\/SURV.2013.050113.00090","volume":"16","author":"Z Sanaei","year":"2014","unstructured":"Sanaei Z, et al. (2014) Heterogeneity in Mobile Cloud Computing: Taxonomy and Open Challenges. IEEE Commun Surv Tutor 16(1):369\u2013392.","journal-title":"IEEE Commun Surv Tutor"},{"issue":"1","key":"276_CR5","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/MC.2017.9","volume":"50","author":"Satyanarayanan","year":"2017","unstructured":"Satyanarayanan, Mahadev (2017) The Emergence of Edge Computing. Computer 50(1):30\u201339.","journal-title":"Computer"},{"key":"276_CR6","unstructured":"Cisco (2018) Cisco global cloud index: Forecast and methodology, 2016-2021. White Paper.pp 1\u201344."},{"issue":"6","key":"276_CR7","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1109\/JIOT.2019.2935543","volume":"6","author":"L Lei","year":"2019","unstructured":"Lei L, Xu H, Xiong X, Zheng K, Xiang W, Wang X (2019) Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing. IEEE Internet Things J 6(6):9\u201310133. https:\/\/doi.org\/10.1109\/JIOT.2019.2935543.","journal-title":"IEEE Internet Things J"},{"issue":"2","key":"276_CR8","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1109\/TNSM.2019.2901346","volume":"16","author":"C Sonmez","year":"2019","unstructured":"Sonmez C, Ozgovde A, Ersoy C (2019) Fuzzy Workload Orchestration for Edge Computing. IEEE Trans Netw Serv Manag 16(2):769\u2013782. https:\/\/doi.org\/10.1109\/TNSM.2019.2901346.","journal-title":"IEEE Trans Netw Serv Manag"},{"key":"276_CR9","doi-asserted-by":"publisher","unstructured":"Flores H, Su X, Kostakos V, Ding AY, Nurmi P, Tarkoma S, Hui P, Li Y (2017) Large-scale offloading in the Internet of Things, 479\u2013484. https:\/\/doi.org\/10.1109\/PERCOMW.2017.7917610.","DOI":"10.1109\/PERCOMW.2017.7917610"},{"key":"276_CR10","doi-asserted-by":"publisher","unstructured":"Sonmez C, Ozgovde A, Ersoy C (2017) EdgeCloudSim: An environment for performance evaluation of Edge Computing systems In: 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC), 39\u201344. https:\/\/doi.org\/10.1109\/FMEC.2017.7946405.","DOI":"10.1109\/FMEC.2017.7946405"},{"key":"276_CR11","doi-asserted-by":"publisher","unstructured":"Santoro D, Zozin D, Pizzolli D, De Pellegrini F, Cretti S (2017) Foggy: A Platform for Workload Orchestration in a Fog Computing Environment In: 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), 231\u2013234. https:\/\/doi.org\/10.1109\/CloudCom.2017.62.","DOI":"10.1109\/CloudCom.2017.62"},{"key":"276_CR12","doi-asserted-by":"publisher","unstructured":"Anas A, Sharma M, Abozariba R, Asaduzzaman M, Benkhelifa E, Patwary MN (2017) Autonomous Workload Balancing in Cloud Federation Environments with Different Access Restrictions In: 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), 636\u2013641. https:\/\/doi.org\/10.1109\/MASS.2017.68.","DOI":"10.1109\/MASS.2017.68"},{"key":"276_CR13","doi-asserted-by":"publisher","unstructured":"Ma X, Zhou A, Zhang S, Wang S (2020) Cooperative Service Caching and Workload Scheduling in Mobile Edge Computing In: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, 2076\u20132085. https:\/\/doi.org\/10.1109\/INFOCOM41043.2020.9155455.","DOI":"10.1109\/INFOCOM41043.2020.9155455"},{"key":"276_CR14","doi-asserted-by":"publisher","unstructured":"Nascimento A, Olimpio V, Silva V, Paes A, de Oliveira D (2019) A Reinforcement Learning Scheduling Strategy for Parallel Cloud-Based Workflows In: 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 817\u2013824. https:\/\/doi.org\/10.1109\/IPDPSW.2019.00134.","DOI":"10.1109\/IPDPSW.2019.00134"},{"key":"276_CR15","doi-asserted-by":"publisher","unstructured":"Liu Z, Zhang H, Rao B, Wang L (2018) A Reinforcement Learning Based Resource Management Approach for Time-critical Workloads in Distributed Computing Environment In: 2018 IEEE International Conference on Big Data (Big Data), 252\u2013261. https:\/\/doi.org\/10.1109\/BigData.2018.8622393.","DOI":"10.1109\/BigData.2018.8622393"},{"issue":"99","key":"276_CR16","first-page":"1","volume":"PP","author":"Y Hao","year":"2020","unstructured":"Hao Y, et al. (2020) Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System. IEEE Trans Ind Inf PP(99):1\u20131.","journal-title":"IEEE Trans Ind Inf"},{"key":"276_CR17","doi-asserted-by":"crossref","unstructured":"Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning In: Proceedings of the AAAI conference on artificial intelligence, Vol 30, No 1.","DOI":"10.1609\/aaai.v30i1.10295"},{"issue":"5","key":"276_CR18","doi-asserted-by":"publisher","first-page":"8577","DOI":"10.1109\/JIOT.2019.2921159","volume":"6","author":"C Qiu","year":"2019","unstructured":"Qiu C, et al. (2019) Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications. IEEE Internet Things J 6(5):8577\u20138588.","journal-title":"IEEE Internet Things J"},{"key":"276_CR19","unstructured":"Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347."},{"issue":"99","key":"276_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TCOMM.2020.3036046","volume":"PP","author":"L Liu","year":"2020","unstructured":"Liu L, Mitra U (2020) On Sampled Reinforcement Learning in Wireless Networks: Exploitation of Policy Structures. IEEE Trans Commun PP(99):1\u20131.","journal-title":"IEEE Trans Commun"},{"issue":"1","key":"276_CR21","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1002\/spe.995","volume":"41","author":"RN Calheiros","year":"2011","unstructured":"Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R (2011) CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23\u201350.","journal-title":"Softw Pract Exp"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-021-00276-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-021-00276-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-021-00276-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,22]],"date-time":"2023-01-22T09:27:44Z","timestamp":1674379664000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-021-00276-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,8]]},"references-count":21,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["276"],"URL":"https:\/\/doi.org\/10.1186\/s13677-021-00276-0","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,8]]},"assertion":[{"value":"22 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"3"}}