{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T16:17:22Z","timestamp":1781108242073,"version":"3.54.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,5,10]],"date-time":"2023-05-10T00:00:00Z","timestamp":1683676800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,10]],"date-time":"2023-05-10T00:00:00Z","timestamp":1683676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Guangdong Provincial Natural Science Foundation of China","award":["2018B030312002"],"award-info":[{"award-number":["2018B030312002"]}]},{"name":"Guangdong Provincial Natural Science Foundation of China","award":["2018B030312002"],"award-info":[{"award-number":["2018B030312002"]}]},{"name":"Guangdong Provincial Natural Science Foundation of China","award":["2018B030312002"],"award-info":[{"award-number":["2018B030312002"]}]},{"name":"Guangdong Provincial Natural Science Foundation of China","award":["2018B030312002"],"award-info":[{"award-number":["2018B030312002"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U1801266"],"award-info":[{"award-number":["U1801266"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U1801266"],"award-info":[{"award-number":["U1801266"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U1801266"],"award-info":[{"award-number":["U1801266"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U1801266"],"award-info":[{"award-number":["U1801266"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Task scheduling is a complex problem in cloud computing, and attracts many researchers\u2019 interests. Recently, many deep reinforcement learning (DRL)-based methods have been proposed to learn the scheduling policy through interacting with the environment. However, most DRL methods focus on a specific environment, which may lead to a weak adaptability to new environments because they have low sample efficiency and require full retraining to learn updated policies for new environments. To overcome the weakness and reduce the time consumption of adapting to new environment, we propose a task scheduling method based on meta reinforcement learning called MRLCC. Through comparing MRLCC and baseline algorithms on the performance of shortening makespan in different environments, we can find that MRLCC is able to adapt to different environments quickly and has a high sample efficiency. Besides, the experimental results demonstrate that MRLCC can maintain a high utilization rate over all baseline algorithms after a few steps of gradient update.<\/jats:p>","DOI":"10.1186\/s13677-023-00440-8","type":"journal-article","created":{"date-parts":[[2023,5,10]],"date-time":"2023-05-10T10:40:28Z","timestamp":1683715228000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["MRLCC: an adaptive cloud task scheduling method based on meta reinforcement learning"],"prefix":"10.1186","volume":"12","author":[{"given":"Xi","family":"Xiu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jialun","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yujie","family":"Long","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weigang","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,5,10]]},"reference":[{"key":"440_CR1","doi-asserted-by":"crossref","unstructured":"Ullah A, Nawi NM, Ouhame S (2022) Recent advancement in VM task allocation system for cloud computing: review from 2015 to 2021. Artif Intell Rev 55:1\u201345","DOI":"10.1007\/s10462-021-10071-7"},{"issue":"6","key":"440_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3243929","volume":"51","author":"AJ Ferrer","year":"2019","unstructured":"Ferrer AJ, Marqu\u00e8s JM, Jorba J (2019) Towards the decentralised cloud: Survey on approaches and challenges for mobile, ad hoc, and edge computing. ACM Comput Surv (CSUR) 51(6):1\u201336","journal-title":"ACM Comput Surv (CSUR)"},{"key":"440_CR3","doi-asserted-by":"crossref","unstructured":"Nazir R, Ahmed Z, Ahmad Z, Shaikh N, Laghari A, Kumar K (2020) Cloud computing applications: a review. EAI Endorsed Trans Cloud Syst 6(17):e5","DOI":"10.4108\/eai.22-5-2020.164667"},{"issue":"1","key":"440_CR4","first-page":"1","volume":"12","author":"V Vinothina","year":"2022","unstructured":"Vinothina V, Rajagopal S et al (2022) Review on mapping of tasks to resources in cloud computing. Int J Cloud Appl Comput (IJCAC) 12(1):1\u201317","journal-title":"Int J Cloud Appl Comput (IJCAC)"},{"key":"440_CR5","doi-asserted-by":"publisher","first-page":"110934","DOI":"10.1016\/j.jss.2021.110934","volume":"176","author":"B Zheng","year":"2021","unstructured":"Zheng B, Pan L, Liu S (2021) Market-oriented online bi-objective service scheduling for pleasingly parallel jobs with variable resources in cloud environments. J Syst Softw 176:110934","journal-title":"J Syst Softw"},{"issue":"11","key":"440_CR6","doi-asserted-by":"publisher","first-page":"2647","DOI":"10.1109\/TC.2013.148","volume":"63","author":"W Song","year":"2014","unstructured":"Song W, Xiao Z, Chen Q, Luo H (2014) Adaptive resource provisioning for the cloud using online bin packing. IEEE Trans Comput 63(11):2647\u20132660. https:\/\/doi.org\/10.1109\/TC.2013.148","journal-title":"IEEE Trans Comput"},{"issue":"4","key":"440_CR7","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1145\/2740070.2626334","volume":"44","author":"R Grandl","year":"2014","unstructured":"Grandl R, Ananthanarayanan G, Kandula S, Rao S, Akella A (2014) Multi-resource packing for cluster schedulers. ACM SIGCOMM Comput Commun Rev 44(4):455\u2013466","journal-title":"ACM SIGCOMM Comput Commun Rev"},{"key":"440_CR8","unstructured":"Ghodsi A, Zaharia M, Hindman B, Konwinski A, Shenker S, Stoica I (2011) Dominant resource fairness: Fair allocation of multiple resource types. In: Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11).\u00a0USENIX Association,\u00a0Boston"},{"key":"440_CR9","doi-asserted-by":"crossref","unstructured":"Xie Y, Sheng Y, Qiu M, Gui F (2022) An adaptive decoding biased random key genetic algorithm for cloud workflow scheduling. Eng Appl Artif Intell 112(104):879","DOI":"10.1016\/j.engappai.2022.104879"},{"issue":"107","key":"440_CR10","first-page":"419","volume":"95","author":"MS Ajmal","year":"2021","unstructured":"Ajmal MS, Iqbal Z, Khan FZ, Ahmad M, Ahmad I, Gupta BB (2021) Hybrid ant genetic algorithm for efficient task scheduling in cloud data centers. Comput Electr Eng 95(107):419","journal-title":"Comput Electr Eng"},{"key":"440_CR11","doi-asserted-by":"crossref","unstructured":"Mao H, Alizadeh M, Menache I, Kandula S (2016) Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM workshop on hot topics in networks. Association for Computing Machinery,\u00a0New York, pp 50\u201356","DOI":"10.1145\/3005745.3005750"},{"key":"440_CR12","doi-asserted-by":"publisher","first-page":"158548","DOI":"10.1109\/ACCESS.2021.3130407","volume":"9","author":"H Lee","year":"2021","unstructured":"Lee H, Cho S, Jang Y, Lee J, Woo H (2021) A global dag task scheduler using deep reinforcement learning and graph convolution network. IEEE Access 9:158548\u2013158561","journal-title":"IEEE Access"},{"issue":"4","key":"440_CR13","doi-asserted-by":"publisher","first-page":"1595","DOI":"10.1007\/s10586-015-0484-2","volume":"18","author":"Z Peng","year":"2015","unstructured":"Peng Z, Cui D, Zuo J, Li Q, Xu B, Lin W (2015) Random task scheduling scheme based on reinforcement learning in cloud computing. Clust Comput 18(4):1595\u20131607","journal-title":"Clust Comput"},{"key":"440_CR14","unstructured":"Sohn S, Woo H, Choi J, Lee H (2020) Meta reinforcement learning with autonomous inference of subtask dependencies. arXiv preprint arXiv:2001.00248"},{"key":"440_CR15","unstructured":"Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the International conference on machine learning, PMLR, pp 1126\u20131135"},{"key":"440_CR16","unstructured":"Pong VH, Nair AV, Smith LM, Huang C, Levine S (2022) Offline meta-reinforcement learning with online self-supervision. In: International Conference on Machine Learning, PMLR, pp 17811\u201317829"},{"key":"440_CR17","doi-asserted-by":"publisher","first-page":"107605","DOI":"10.1016\/j.asoc.2021.107605","volume":"110","author":"S Wen","year":"2021","unstructured":"Wen S, Wen Z, Zhang D, Zhang H, Wang T (2021) A multi-robot path-planning algorithm for autonomous navigation using meta-reinforcement learning based on transfer learning. Appl Soft Comput 110:107605","journal-title":"Appl Soft Comput"},{"issue":"10","key":"440_CR18","doi-asserted-by":"publisher","first-page":"5374","DOI":"10.1109\/TNNLS.2021.3070584","volume":"33","author":"L Chen","year":"2021","unstructured":"Chen L, Hu B, Guan ZH, Zhao L, Shen X (2021) Multiagent meta-reinforcement learning for adaptive multipath routing optimization. IEEE Trans Neural Netw Learn Syst 33(10):5374\u20135386","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"440_CR19","unstructured":"Kim DK, Liu M, Riemer MD, Sun C, Abdulhai M, Habibi G, Lopez-Cot S, Tesauro G, How J (2021) A policy gradient algorithm for learning to learn in multiagent reinforcement learning. In: International Conference on Machine Learning, PMLR, pp 5541\u20135550"},{"key":"440_CR20","doi-asserted-by":"publisher","first-page":"878","DOI":"10.1016\/j.procs.2016.05.278","volume":"85","author":"P Pradhan","year":"2016","unstructured":"Pradhan P, Behera PK, Ray B (2016) Modified round robin algorithm for resource allocation in cloud computing. Procedia Comput Sci 85:878\u2013890","journal-title":"Procedia Comput Sci"},{"issue":"7","key":"440_CR21","doi-asserted-by":"publisher","first-page":"1866","DOI":"10.1109\/TPDS.2016.2636210","volume":"28","author":"T Chen","year":"2016","unstructured":"Chen T, Marques AG, Giannakis GB (2016) Dglb: Distributed stochastic geographical load balancing over cloud networks. IEEE Trans Parallel Distrib Syst 28(7):1866\u20131880","journal-title":"IEEE Trans Parallel Distrib Syst"},{"issue":"5","key":"440_CR22","doi-asserted-by":"publisher","first-page":"2603","DOI":"10.1007\/s11227-018-2656-3","volume":"75","author":"M Ghobaei-Arani","year":"2019","unstructured":"Ghobaei-Arani M, Souri A (2019) Lp-wsc: a linear programming approach for web service composition in geographically distributed cloud environments. J Supercomput 75(5):2603\u20132628","journal-title":"J Supercomput"},{"key":"440_CR23","doi-asserted-by":"publisher","first-page":"1017","DOI":"10.1007\/s10586-016-0574-9","volume":"19","author":"M Ghobaei-Arani","year":"2016","unstructured":"Ghobaei-Arani M, Jabbehdari S, Pourmina MA (2016) An autonomic approach for resource provisioning of cloud services. Clust Comput 19:1017\u20131036","journal-title":"Clust Comput"},{"issue":"8","key":"440_CR24","doi-asserted-by":"publisher","first-page":"1786","DOI":"10.1109\/TPDS.2019.2893648","volume":"30","author":"D Basu","year":"2019","unstructured":"Basu D, Wang X, Hong Y, Chen H, Bressan S (2019) Learn-as-you-go with megh: Efficient live migration of virtual machines. IEEE Trans Parallel Distrib Syst 30(8):1786\u20131801. https:\/\/doi.org\/10.1109\/TPDS.2019.2893648","journal-title":"IEEE Trans Parallel Distrib Syst"},{"key":"440_CR25","first-page":"147","volume":"19","author":"M Kumar","year":"2018","unstructured":"Kumar M, Sharma SC (2018) Pso-cogent: Cost and energy efficient scheduling in cloud environment with deadline constraint. Sustain Comput Inform Syst 19:147\u2013164","journal-title":"Sustain Comput Inform Syst"},{"key":"440_CR26","unstructured":"Jin HZ, Yang L, Hao O (2015) Scheduling strategy based on genetic algorithm for cloud computer energy optimization. In: Proceedings of the 2015 IEEE International Conference on Communication Problem-Solving (ICCP), IEEE, pp 516\u2013519"},{"key":"440_CR27","doi-asserted-by":"publisher","first-page":"102323","DOI":"10.1016\/j.simpat.2021.102323","volume":"110","author":"R Medara","year":"2021","unstructured":"Medara R, Singh RS et al (2021) Energy-aware workflow task scheduling in clouds with virtual machine consolidation using discrete water wave optimization. Simul Model Pract Theory 110:102323","journal-title":"Simul Model Pract Theory"},{"key":"440_CR28","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1016\/j.future.2020.02.018","volume":"108","author":"D Ding","year":"2020","unstructured":"Ding D, Fan X, Zhao Y, Kang K, Yin Q, Zeng J (2020) Q-learning based dynamic task scheduling for energy-efficient cloud computing. Futur Gener Comput Syst 108:361\u2013371","journal-title":"Futur Gener Comput Syst"},{"issue":"11","key":"440_CR29","doi-asserted-by":"publisher","first-page":"5654","DOI":"10.1002\/cpe.5654","volume":"32","author":"T Dong","year":"2020","unstructured":"Dong T, Xue F, Xiao C, Li J (2020) Task scheduling based on deep reinforcement learning in a cloud manufacturing environment. Concurr Comput Pract Experience 32(11):5654","journal-title":"Concurr Comput Pract Experience"},{"key":"440_CR30","doi-asserted-by":"publisher","first-page":"107688","DOI":"10.1016\/j.compeleceng.2022.107688","volume":"99","author":"J Yan","year":"2022","unstructured":"Yan J, Huang Y, Gupta A, Gupta A, Liu C, Li J, Cheng L (2022) Energy-aware systems for real-time job scheduling in cloud data centers: A deep reinforcement learning approach. Comput Electr Eng 99:107688","journal-title":"Comput Electr Eng"},{"key":"440_CR31","doi-asserted-by":"crossref","unstructured":"Cheng F, Huang Y, Tanpure B, Sawalani P, Cheng L, Liu C (2022a) Cost-aware job scheduling for cloud instances using deep reinforcement learning. Clust Comput 25:1\u201313","DOI":"10.1007\/s10586-021-03436-8"},{"issue":"21","key":"440_CR32","doi-asserted-by":"publisher","first-page":"18579","DOI":"10.1007\/s00521-022-07477-x","volume":"34","author":"L Cheng","year":"2022","unstructured":"Cheng L, Kalapgar A, Jain A, Wang Y, Qin Y, Li Y, Liu C (2022) Cost-aware real-time job scheduling for hybrid cloud using deep reinforcement learning. Neural Comput & Applic 34(21):18579\u201318593","journal-title":"Neural Comput & Applic"},{"key":"440_CR33","doi-asserted-by":"publisher","first-page":"55112","DOI":"10.1109\/ACCESS.2018.2872674","volume":"6","author":"Y Wei","year":"2018","unstructured":"Wei Y, Pan L, Liu S, Wu L, Meng X (2018) Drl-scheduling: An intelligent qos-aware job scheduling framework for applications in clouds. IEEE Access 6:55112\u201355125. https:\/\/doi.org\/10.1109\/ACCESS.2018.2872674","journal-title":"IEEE Access"},{"issue":"3","key":"440_CR34","doi-asserted-by":"publisher","first-page":"4232","DOI":"10.1109\/JSYST.2021.3122126","volume":"16","author":"Y Huang","year":"2021","unstructured":"Huang Y, Cheng L, Xue L, Liu C, Li Y, Li J, Ward T (2021) Deep adversarial imitation reinforcement learning for qos-aware cloud job scheduling. IEEE Syst J 16(3):4232\u20134242","journal-title":"IEEE Syst J"},{"key":"440_CR35","doi-asserted-by":"crossref","unstructured":"Cao Z, Zhou P, Li R, Huang S, Wu D (2020) Multiagent deep reinforcement learning for joint multichannel access and task offloading of mobile-edge computing in industry 4.0. IEEE Internet Things J 7(7):6201\u20136213","DOI":"10.1109\/JIOT.2020.2968951"},{"issue":"5","key":"440_CR36","doi-asserted-by":"publisher","first-page":"3576","DOI":"10.1109\/JIOT.2020.3025015","volume":"8","author":"W Guo","year":"2020","unstructured":"Guo W, Tian W, Ye Y, Xu L, Wu K (2020) Cloud resource scheduling with deep reinforcement learning and imitation learning. IEEE Internet Things J 8(5):3576\u20133586","journal-title":"IEEE Internet Things J"},{"key":"440_CR37","doi-asserted-by":"crossref","unstructured":"Liu N, Li Z, Xu J, Xu Z, Lin S, Qiu Q, Tang J, Wang Y (2017) A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. In: Proceedings of the IEEE 37th international conference on distributed computing systems (ICDCS), IEEE, pp 372\u2013382","DOI":"10.1109\/ICDCS.2017.123"},{"key":"440_CR38","doi-asserted-by":"crossref","unstructured":"Xu Z, Wang Y, Tang J, Wang J, Gursoy MC (2017) A deep reinforcement learning based framework for power-efficient resource allocation in cloud rans. In: 2017 IEEE International Conference on Communications (ICC), IEEE, pp 1\u20136","DOI":"10.1109\/ICC.2017.7997286"},{"key":"440_CR39","unstructured":"Li Y (2017) Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274"},{"key":"440_CR40","doi-asserted-by":"publisher","DOI":"10.1002\/9781118557426","volume-title":"Markov decision processes in artificial intelligence","author":"O Sigaud","year":"2013","unstructured":"Sigaud O, Buffet O (2013) Markov decision processes in artificial intelligence. John Wiley & Sons"},{"key":"440_CR41","unstructured":"Fakoor R, Chaudhari P, Soatto S, Smola AJ (2019) Meta-q-learning. arXiv preprint arXiv:1910.00125"},{"issue":"5","key":"440_CR42","doi-asserted-by":"publisher","first-page":"1568","DOI":"10.1109\/LCOMM.2020.3048075","volume":"25","author":"L Huang","year":"2020","unstructured":"Huang L, Zhang L, Yang S, Qian LP, Wu Y (2020) Meta-learning based dynamic computation task offloading for mobile edge computing networks. IEEE Commun Lett 25(5):1568\u20131572","journal-title":"IEEE Commun Lett"},{"key":"440_CR43","doi-asserted-by":"crossref","unstructured":"Lin J, Peng Z, Cui D (2018) Deep reinforcement learning for multi-resource cloud job scheduling. In: Proceedings of the International conference on neural information processing, Springer, pp 289\u2013302","DOI":"10.1007\/978-3-030-04182-3_26"},{"issue":"3","key":"440_CR44","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3):279\u2013292","journal-title":"Mach Learn"},{"key":"440_CR45","unstructured":"Ye Y, Ren X, Wang J, Xu L, Guo W, Huang W, Tian W (2018) A new approach for resource scheduling with deep reinforcement learning. arXiv preprint arXiv:1806.08122"},{"key":"440_CR46","unstructured":"Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999"},{"key":"440_CR47","doi-asserted-by":"crossref","unstructured":"Romero F, Chaudhry GI, Goiri \u00cd, Gopa P, Batum P, Yadwadkar NJ, et al (2021) Faa$T: A transparent auto-scaling cache for serverless applications. Proceedings of the ACM Symposium on Cloud Computing. Association for Computing Machinery, New York, 122\u2013137","DOI":"10.1145\/3472883.3486974"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00440-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00440-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00440-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,10]],"date-time":"2023-05-10T10:44:16Z","timestamp":1683715456000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00440-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,10]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["440"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00440-8","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,10]]},"assertion":[{"value":"18 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 May 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"75"}}