{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:41:35Z","timestamp":1774539695128,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T00:00:00Z","timestamp":1690502400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T00:00:00Z","timestamp":1690502400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Cloud task scheduling and resource allocation (TSRA) constitute a core issue in cloud computing. Batch submission is a common user task deployment mode in cloud computing systems. In this mode, it has been a challenge for cloud systems to balance the quality of user service and the revenue of cloud service provider (CSP). To this end, with multi-objective optimization (MOO) of minimizing task latency and energy consumption, we propose a cloud TSRA framework based on deep learning (DL). The system solves the TSRA problems of multiple task queues and virtual machine (VM) clusters by uniting multiple deep neural networks (DNNs) as task scheduler of cloud system. The DNNs are divided into exploration part and exploitation part. At each scheduling time step, the model saves the best outputs of all scheduling policies from each DNN to the experienced sample memory pool (SMP), and periodically selects random training samples from SMP to train each DNN of exploitation part. We designed a united deep learning (UDL) algorithm based on this framework. Experimental results show that the UDL algorithm can effectively solve the MOO problem of TSRA for cloud tasks, and performs better than benchmark algorithms such as heterogeneous distributed deep learning (HDDL) in terms of task scheduling performance.<\/jats:p>","DOI":"10.1186\/s13677-023-00490-y","type":"journal-article","created":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T08:02:02Z","timestamp":1690531322000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["UDL: a cloud task scheduling framework based on multiple deep neural networks"],"prefix":"10.1186","volume":"12","author":[{"given":"Qirui","family":"Li","sequence":"first","affiliation":[]},{"given":"Zhiping","family":"Peng","sequence":"additional","affiliation":[]},{"given":"Delong","family":"Cui","sequence":"additional","affiliation":[]},{"given":"Jianpeng","family":"Lin","sequence":"additional","affiliation":[]},{"given":"Hao","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,7,28]]},"reference":[{"issue":"3","key":"490_CR1","doi-asserted-by":"publisher","first-page":"762","DOI":"10.1109\/TETC.2018.2794328","volume":"8","author":"J Panneerselvam","year":"2020","unstructured":"Panneerselvam J, Liu L, Antonopoulos N (2020) An approach to optimise resource provision with energy-awareness in datacentres by combating task heterogeneity. IEEE Trans Emerg Top Comput 8(3):762\u2013780","journal-title":"IEEE Trans Emerg Top Comput"},{"issue":"1","key":"490_CR2","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1109\/TCC.2014.2306427","volume":"2","author":"Q Zhang","year":"2014","unstructured":"Zhang Q, Zhani MF, Boutaba R, Hellerstein JL (2014) Dynamic heterogeneity-aware resource provisioning in the cloud. IEEE Trans Cloud Comput 2(1):14\u201328","journal-title":"IEEE Trans Cloud Comput"},{"issue":"2","key":"490_CR3","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1007\/s10619-019-07273-y","volume":"38","author":"N Mansouri","year":"2020","unstructured":"Mansouri N, Javidi MM (2020) Cost-based job scheduling strategy in cloud computing environments. Distrib Parallel Databases 38(2):365\u2013400","journal-title":"Distrib Parallel Databases"},{"key":"490_CR4","doi-asserted-by":"publisher","first-page":"2489","DOI":"10.1007\/s10586-016-0684-4","volume":"20","author":"SHH Madni","year":"2016","unstructured":"Madni SHH, Latiff MSAA, Coulibaly Y, Abdulhamid SM (2016) Recent advancements in resource allocation techniques for cloud computing environment: a systematic review. Clust Comput 20:2489\u20132533","journal-title":"Clust Comput"},{"key":"490_CR5","doi-asserted-by":"crossref","unstructured":"Mathew T, Sekaran KC, Jose J (2014) Study and analysis of various task scheduling algorithms in the cloud computing environment. 2014 International Conference on Advances in Computing, Communications and Informatics(ICACCI), Pune, India, pp 658-664","DOI":"10.1109\/ICACCI.2014.6968517"},{"key":"490_CR6","doi-asserted-by":"crossref","unstructured":"Patil N, Aeloor D (2017) A review - different scheduling algorithms in cloud computing environment. 2017 11th International Conference on Intelligent Systems and Control(ISCO), Coimbatore, India, pp 182-185","DOI":"10.1109\/ISCO.2017.7855977"},{"key":"490_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.parco.2017.01.002","volume":"62","author":"A Verma","year":"2017","unstructured":"Verma A, Kaushal S (2017) A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling. Parallel Comput 62:1\u201319","journal-title":"Parallel Comput"},{"key":"490_CR8","doi-asserted-by":"publisher","first-page":"2687","DOI":"10.1109\/ACCESS.2015.2508940","volume":"3","author":"L Zuo","year":"2015","unstructured":"Zuo L, Shu L, Dong S, Zhu C, Hara T (2015) A multi-objective optimization scheduling method based on the ant colony algorithm in cloud computing. IEEE Access 3:2687\u20132699","journal-title":"IEEE Access"},{"key":"490_CR9","doi-asserted-by":"crossref","unstructured":"Alkayal ES, Jennings NR, Abulkhair MF (2016) Efficient task scheduling multi-objective particle swarm optimization in cloud computing. 2016 IEEE 41st Conference on Local Computer Networks Workshops(LCN Workshops), Dubai, United Arab Emirates, pp 17-24","DOI":"10.1109\/LCN.2016.024"},{"key":"490_CR10","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1016\/j.future.2016.02.016","volume":"74","author":"H Duan","year":"2017","unstructured":"Duan H, Chen C, Min G, Wu Y (2017) Energy-aware scheduling of virtual machines in heterogeneous cloud computing systems. Futur Gener Comput Syst 74:142\u2013150","journal-title":"Futur Gener Comput Syst"},{"issue":"2","key":"490_CR11","first-page":"210","volume":"3","author":"S Srichandan","year":"2018","unstructured":"Srichandan S, Kumar TA, Bibhudatta S (2018) Task scheduling for cloud computing using multi-objective hybrid bacteria foraging algorithm. Futur Comput Inf J 3(2):210\u201323","journal-title":"Futur Comput Inf J"},{"key":"490_CR12","doi-asserted-by":"publisher","first-page":"1595","DOI":"10.1007\/s10586-015-0484-2","volume":"18","author":"Z Peng","year":"2015","unstructured":"Peng Z, Cui D, Zuo J, Li Q, Xu B (2015) Random task scheduling scheme based on reinforcement learning in cloud computing. Clust Comput 18:1595\u20131607","journal-title":"Clust Comput"},{"key":"490_CR13","first-page":"1","volume":"2015","author":"Z Peng","year":"2015","unstructured":"Peng Z, Cui D, Zuo J, Lin W (2015) Research on cloud computing resources provisioning based on reinforcement learning. Math Probl Eng 2015:1\u201312","journal-title":"Math Probl Eng"},{"key":"490_CR14","doi-asserted-by":"publisher","first-page":"1030","DOI":"10.1109\/TCC.2017.2773078","volume":"4","author":"D Cui","year":"2020","unstructured":"Cui D, Peng Z, Xiong J, Xu B, Lin W (2020) A reinforcement learning-based mixed job scheduler scheme for Grid or IaaS cloud. IEEE Trans Cloud Comput 4:1030\u20131039","journal-title":"IEEE Trans Cloud Comput"},{"issue":"10","key":"490_CR15","first-page":"1319","volume":"32","author":"T Thein","year":"2020","unstructured":"Thein T, Myo MM, Parvin S, Gawanmeh A (2020) Reinforcement learning based methodology for energy-efficient resource allocation in cloud data centers. J King Saud Univ - Comput Inf Sci 32(10):1319\u20131578","journal-title":"J King Saud Univ - Comput Inf Sci"},{"key":"490_CR16","doi-asserted-by":"crossref","unstructured":"Wei Y, Daniel K, Liu S, Li P, WU L, Meng X, (2019) A reinforcement learning based auto-scaling approach for SaaS providers in dynamic cloud environment. Math Probl Eng 2019:1\u201311","DOI":"10.1155\/2019\/5080647"},{"issue":"7","key":"490_CR17","doi-asserted-by":"publisher","first-page":"4957","DOI":"10.1109\/TII.2020.3019386","volume":"17","author":"H Liang","year":"2021","unstructured":"Liang H, Zhang X, Hong X, Zhang Z, Li M, Hu G, Hou F (2021) Reinforcement learning enabled dynamic resource allocation in the internet of vehicles. IEEE Trans Ind Inform 17(7):4957\u20134967","journal-title":"IEEE Trans Ind Inform"},{"issue":"5","key":"490_CR18","doi-asserted-by":"publisher","first-page":"3576","DOI":"10.1109\/JIOT.2020.3025015","volume":"8","author":"W Guo","year":"2021","unstructured":"Guo W, Tian W, Ye Y, Xu L, Wu K (2021) Cloud resource scheduling with deep reinforcement learning and imitation learning. IEEE Internet Things J 8(5):3576\u20133586","journal-title":"IEEE Internet Things J"},{"key":"490_CR19","doi-asserted-by":"crossref","unstructured":"Chudasama V, Bhavsar M (2020) A dynamic prediction for elastic resource allocation in hybrid cloud environment. Scalable Comput: Pract Experience 21(4 SI):661-672","DOI":"10.12694\/scpe.v21i4.1805"},{"issue":"7","key":"490_CR20","doi-asserted-by":"publisher","first-page":"1883122","DOI":"10.1080\/17517575.2021.1883122","volume":"16","author":"A Lakhan","year":"2022","unstructured":"Lakhan A, Mastoi Q, Elhoseny M, Memon MS, Mohammed MA (2022) Deep neural network-based application partitioning and scheduling for hospitals and medical enterprises using IoT assisted mobile fog cloud. Enterp Inf Syst 16(7):1883122","journal-title":"Enterp Inf Syst"},{"key":"490_CR21","doi-asserted-by":"crossref","unstructured":"Rangra A, Sehgal VK, Shukla S (2019)A novel approach of cloud based scheduling using deep-learning approach in E-Commerce domain. Int J Inf Syst Model Des 10(3 SI):59-75","DOI":"10.4018\/IJISMD.2019070104"},{"key":"490_CR22","doi-asserted-by":"publisher","first-page":"197863","DOI":"10.1109\/ACCESS.2020.3033557","volume":"8","author":"J Lin","year":"2020","unstructured":"Lin J, Cui D, Peng Z, Li Q, He J (2020) A two-stage framework for the multi-user multi-data center job scheduling and resource allocation. IEEE Access 8:197863\u2013197874","journal-title":"IEEE Access"},{"issue":"1","key":"490_CR23","first-page":"1","volume":"41","author":"Q Liu","year":"2018","unstructured":"Liu Q, Zhai J, Zhang Z, Zhong S, Zhou Q, Zhang P, Xu J (2018) A survey on deep reinforcement learning. Jisuanji Xuebao\/Chin J Comput 41(1):1\u201327","journal-title":"Jisuanji Xuebao\/Chin J Comput"},{"key":"490_CR24","doi-asserted-by":"crossref","unstructured":"Sharma AR, Kaushik P (2017) Literature survey of statistical, deep and reinforcement learning in natural language processing. 2017 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India, pp 350-354","DOI":"10.1109\/CCAA.2017.8229841"},{"issue":"7540","key":"490_CR25","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Veness J (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529\u201333","journal-title":"Nature"},{"key":"490_CR26","doi-asserted-by":"crossref","unstructured":"Phaniteja S, Dewangan P, Guhan P, Sarkar A, Krishna KM (2017) A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots. 2017 IEEE International Conference on Robotics and Biomimetics(ROBIO), Macau, Macao, pp 1818-1823","DOI":"10.1109\/ROBIO.2017.8324682"},{"issue":"4","key":"490_CR27","doi-asserted-by":"publisher","first-page":"2753","DOI":"10.1007\/s10586-019-03042-9","volume":"23","author":"Z Peng","year":"2020","unstructured":"Peng Z, Lin J, Cui D, Li Q, He J (2020) A multi-objective trade-off framework for cloud resource scheduling based on the Deep Q-network algorithm. Clust Comput 23(4):2753\u20132767","journal-title":"Clust Comput"},{"key":"490_CR28","doi-asserted-by":"crossref","unstructured":"Lin J, Peng Z, Cui D (2018) Deep reinforcement learning for multi-resource cloud job scheduling. 2018 25th International Conference on Neural Information Processing, Siem Reap, Cambodia, pp 289-302","DOI":"10.1007\/978-3-030-04182-3_26"},{"key":"490_CR29","doi-asserted-by":"crossref","unstructured":"Bitsakos C, Konstantinou I, Koziris N (2018) DERP: a deep reinforcement learning cloud system for elastic resource provisioning. 2018 IEEE International Conference on Cloud Computing Technology and Science(CloudCom), Nicosia, Cyprus, 21-29","DOI":"10.1109\/CloudCom2018.2018.00020"},{"key":"490_CR30","doi-asserted-by":"crossref","unstructured":"Zhang C, Liu Z, Gu B, Yamori K, Tanaka Y (2018) A deep reinforcement learning based approach for cost- and energy-aware multi-flow mobile data offloading. IEICE Trans Commun E101.B:1625-1634","DOI":"10.1587\/transcom.2017CQP0014"},{"issue":"6","key":"490_CR31","doi-asserted-by":"publisher","first-page":"1446","DOI":"10.3390\/s19061446","volume":"19","author":"H Liang","year":"2019","unstructured":"Liang H, Feng L, Zhang L, Qian Y (2019) Multi-server multi-user multi-task computation offloading for mobile edge computing networks. Sensors 19(6):1446","journal-title":"Sensors"},{"key":"490_CR32","doi-asserted-by":"publisher","first-page":"2699","DOI":"10.1007\/s10586-021-03525-8","volume":"25","author":"Q Li","year":"2022","unstructured":"Li Q, Peng Z, Cui D, Lin J, He J (2022) Two-stage selection of distributed data centers based on deep reinforcement learning. Clust Comput 25:2699\u20132714","journal-title":"Clust Comput"},{"key":"490_CR33","unstructured":"Li K, Zhang T, Wang R, Qin W, He Hi, Huang H (2021) Research reviews of combinatorial optimization methods based on deep reinforcement learning. Acta Automatica Sinica 47(11):2521\u20132537"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00490-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00490-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00490-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T08:03:48Z","timestamp":1690531428000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00490-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,28]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["490"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00490-y","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,28]]},"assertion":[{"value":"15 February 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This material is the authors\u2019 own original work, which has not been previously published elsewhere. The paper is not currently being considered for publication elsewhere.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"114"}}