{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T01:58:58Z","timestamp":1783994338783,"version":"3.55.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,26]],"date-time":"2023-08-26T00:00:00Z","timestamp":1693008000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,26]],"date-time":"2023-08-26T00:00:00Z","timestamp":1693008000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the Beijing Municipal Natural Science Foundation","award":["No. 4202021"],"award-info":[{"award-number":["No. 4202021"]}]},{"name":"the Beijing Municipal Natural Science Foundation","award":["No. 4202021"],"award-info":[{"award-number":["No. 4202021"]}]},{"name":"the Beijing Municipal Natural Science Foundation","award":["No. 4202021"],"award-info":[{"award-number":["No. 4202021"]}]},{"name":"the Beijing Municipal Natural Science Foundation","award":["No. 4202021"],"award-info":[{"award-number":["No. 4202021"]}]},{"name":"the Key-Area Research and Development Program of Guangzhou City","award":["202206030009"],"award-info":[{"award-number":["202206030009"]}]},{"name":"the Key-Area Research and Development Program of Guangzhou City","award":["202206030009"],"award-info":[{"award-number":["202206030009"]}]},{"name":"the Key-Area Research and Development Program of Guangzhou City","award":["202206030009"],"award-info":[{"award-number":["202206030009"]}]},{"name":"the Key-Area Research and Development Program of Guangzhou City","award":["202206030009"],"award-info":[{"award-number":["202206030009"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>With the increase development of Internet of Things devices, the data-intensive workflow has emerged as a new kinds of representation for IoT applications. Because most IoT systems are structured in multi-clouds environment and the data-intensive workflow has the characteristics of scattered data sources and distributed execution requirements at the cloud center and edge clouds, it brings many challenges to the scheduling of such workflow, such as data flow control management, data transmission scheduling, etc. Aiming at the execution constraints of business and technology and data transmission optimization of data-intensive workflow, a data-intensive workflow scheduling method based on deep reinforcement learning in multi-clouds is proposed. First, the execution constraints, edge node load and data transmission volume of IoT data workflow are modeled; then the data-intensive workflow is segmented with the consideration of business constraints and the first optimization goal of data transmission; besides, taking the workflow execution time and average load balancing as the secondary optimization goal, the improved DQN algorithm is used to schedule the workflow. Based on the DQN algorithm, the model reward function and action selection are redesigned and improved. The simulation results based on WorkflowSim show that, compared with MOPSO, NSGA-II, GTBGA and DQN, the algorithm proposed in this paper can effectively reduce the execution time of IoT data workflow under the condition of ensuring the execution constraints and load balancing of multi-clouds.<\/jats:p>","DOI":"10.1186\/s13677-023-00504-9","type":"journal-article","created":{"date-parts":[[2023,8,26]],"date-time":"2023-08-26T09:01:42Z","timestamp":1693040502000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Data-intensive workflow scheduling strategy based on deep reinforcement learning in multi-clouds"],"prefix":"10.1186","volume":"12","author":[{"given":"Shuo","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhuofeng","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chen","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shenghui","family":"Qin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,26]]},"reference":[{"key":"504_CR1","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.future.2022.09.007","volume":"139","author":"J Huang","year":"2023","unstructured":"Huang J, Gao H, Wan S et al (2023) AoI-aware energy control and computation offloading for industrial IoT. Futur Gener Comput Syst 139:29\u201337","journal-title":"Futur Gener Comput Syst"},{"issue":"2","key":"504_CR2","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1049\/cje.2020.02.001","volume":"29","author":"J Huang","year":"2020","unstructured":"Huang J, Zhang C, Zhang J (2020) A multi-queue approach of energy efficient task scheduling for sensor hubs. Chin J Electron 29(2):242\u2013247","journal-title":"Chin J Electron"},{"key":"504_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42979-020-00326-5","volume":"1","author":"C Shyalika","year":"2020","unstructured":"Shyalika C, Silva T, Karunananda A (2020) Reinforcement learning in dynamic task scheduling: a review. SN Comput Sci 1:1\u201317","journal-title":"SN Comput Sci"},{"key":"504_CR4","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.jnca.2016.01.018","volume":"66","author":"M Masdari","year":"2016","unstructured":"Masdari M, ValiKardan S, Shahi Z et al (2016) Towards workflow scheduling in cloud computing: a comprehensive analysis. J Netw Comput Appl 66:64\u201382","journal-title":"J Netw Comput Appl"},{"key":"504_CR5","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1016\/j.procs.2017.12.093","volume":"125","author":"K Dubey","year":"2018","unstructured":"Dubey K, Kumar M, Sharma SC (2018) Modified HEFT algorithm for task scheduling in cloud environment. Procedia Comput Sci 125:725\u2013732","journal-title":"Procedia Comput Sci"},{"key":"504_CR6","doi-asserted-by":"publisher","first-page":"44","DOI":"10.7763\/IJMO.2015.V5.434","volume":"5","author":"NJ Navimipour","year":"2015","unstructured":"Navimipour NJ, Milani FS (2015) Task scheduling in the cloud computing based on the cuckoo search algorithm. Int J Model Optim 5:44\u201347","journal-title":"Int J Model Optim"},{"key":"504_CR7","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.eswa.2016.07.046","volume":"65","author":"KZ Gao","year":"2016","unstructured":"Gao KZ, Suganthan PN, Pan QK, Chua TJ, Chong CS, Cai TX (2016) An improved artificial bee colony algorithm for flexible job-shop scheduling problem with fuzzy processing time. Expert Syst Appl 65:52\u201367","journal-title":"Expert Syst Appl"},{"key":"504_CR8","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.ins.2018.01.027","volume":"436","author":"F Wang","year":"2018","unstructured":"Wang F, Zhang H, Li K et al (2018) A hybrid particle swarm optimization algorithm using adaptive learning strategy. Inf Sci 436:162\u2013177","journal-title":"Inf Sci"},{"key":"504_CR9","doi-asserted-by":"crossref","unstructured":"Pei S, Zhang Q, Cheng X (2020) Workflow scheduling using graph segmentation and reinforcement learning. Int J Perform Eng 16(8)","DOI":"10.23940\/ijpe.20.08.p13.12621270"},{"key":"504_CR10","doi-asserted-by":"crossref","unstructured":"Chen Y, Zhao J, Wu Y, Huang Y, Shen XS (2022) QoE-aware decentralized task Offloading and resource allocation for end-edge-cloud systems: a game-theoretical approach. IEEE Trans Mob Comput. https:\/\/ieeexplore.ieee.org\/document\/9954914","DOI":"10.1109\/TMC.2022.3223119"},{"key":"504_CR11","unstructured":"CHEN Ying, HU Jintao, ZHAO Jie, et al (2023) QoS-Aware Computation offloading in LEO satellite edge computing for IoT: a game-theoretical approach. Chin J Electron. https:\/\/cje.ejournal.org.cn\/article\/doi\/10.23919\/cje.2022.00.412"},{"key":"504_CR12","doi-asserted-by":"crossref","unstructured":"Ying Chen, Jie Zhao, Xiaokang Zhou, Lianyong Qi, Xiaolong Xu, Jiwei Huang (2023) A distributed game theoretical approach for credibility-guaranteed multimedia data offloading in MEC. Inf Sci 644:0020\u20130255","DOI":"10.1016\/j.ins.2023.119306"},{"issue":"4","key":"504_CR13","doi-asserted-by":"publisher","first-page":"3213","DOI":"10.1007\/s13369-020-05141-x","volume":"46","author":"NA Alawad","year":"2021","unstructured":"Alawad NA, Abed-alguni BH (2021) Discrete island-based cuckoo search with highly disruptive polynomial mutation and opposition-based learning strategy for scheduling of workflow applications in cloud environments. Arab J Sci Eng 46(4):3213\u20133233","journal-title":"Arab J Sci Eng"},{"issue":"3","key":"504_CR14","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1002\/spe.2802","volume":"52","author":"A Kaur","year":"2022","unstructured":"Kaur A, Singh P, Singh Batth R et al (2022) Deep-Q learning-based heterogeneous earliest finish time scheduling algorithm for scientific workflows in cloud. Softw Pract Exp 52(3):689\u2013709","journal-title":"Softw Pract Exp"},{"key":"504_CR15","doi-asserted-by":"crossref","unstructured":"Chen Y, WG U, Xu J, Zhang Y, Min G (2023) Dynamic task offloading for digital twin-empowered mobile edge computing via deep reinforcement learning. Chin Commun 1\u201312. https:\/\/ieeexplore.ieee.org\/abstract\/document\/10122834","DOI":"10.23919\/JCC.ea.2022-0372.202302"},{"key":"504_CR16","doi-asserted-by":"crossref","unstructured":"Huang J, Wan J, Lv B, Ye Q, Chen Y (2023) Joint computation offloading and resource allocation for edge-cloud collaboration in internet of vehicles via Deep reinforcement learning. IEEE Syst J 17(2):2500\u20132511","DOI":"10.1109\/JSYST.2023.3249217"},{"key":"504_CR17","doi-asserted-by":"crossref","unstructured":"AL-Tam F, Mazayev A, Correia N, Rodriguez J (2020) Radio resource scheduling with deep pointer networks and reinforcement Learning. 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design ofCommunication Links and Networks (CAMAD). IEEE, Pisa, Italy, p. 1-6","DOI":"10.1109\/CAMAD50429.2020.9209313"},{"key":"504_CR18","doi-asserted-by":"crossref","unstructured":"Ying Chen, Jie Zhao, Jintao Hu, Shaohua Wan, Jiwei Huang (2023) Distributed task offloading and resource purchasing in NOMA-enabled mobile edge computing: hierarchical game theoretical approaches. ACM Trans Embed Comput Syst 1539\u20139087. https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3597023","DOI":"10.1145\/3597023"},{"key":"504_CR19","doi-asserted-by":"crossref","unstructured":"Ling N, Wang K, He Y, et al (2021) Rt-mdl: supporting real-time mixed deep learning tasks on edge platforms. In: Proceedings of the 19th ACM conference on embedded networked sensor systems. pp 1\u201314","DOI":"10.1145\/3485730.3485938"},{"issue":"3","key":"504_CR20","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1109\/TEVC.2004.826067","volume":"8","author":"CAC Coello","year":"2004","unstructured":"Coello CAC, Pulido GT, Lechuga MS (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8(3):256\u2013279","journal-title":"IEEE Trans Evol Comput"},{"key":"504_CR21","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.jnca.2017.04.007","volume":"88","author":"EJ Ghomi","year":"2017","unstructured":"Ghomi EJ, Rahmani AM, Qader NN (2017) Load-balancing algorithms in cloud computing: a survey. J Netw Comput Appl 88:50\u201371","journal-title":"J Netw Comput Appl"},{"issue":"1","key":"504_CR22","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1080\/17517575.2012.688220","volume":"7","author":"G Monakova","year":"2013","unstructured":"Monakova G, Leymann F (2013) Workflow ART: a framework for multidimensional workflow analysis. Enterp Inf Syst 7(1):133\u2013166","journal-title":"Enterp Inf Syst"},{"key":"504_CR23","doi-asserted-by":"publisher","first-page":"108786","DOI":"10.1016\/j.asoc.2022.108786","volume":"122","author":"Z Quan","year":"2022","unstructured":"Quan Z, Wang Y, Ji Z (2022) Multi-objective optimization scheduling for manufacturing process based on virtual workflow models. Appl Soft Comput 122:108786","journal-title":"Appl Soft Comput"},{"key":"504_CR24","doi-asserted-by":"publisher","first-page":"39974","DOI":"10.1109\/ACCESS.2019.2902846","volume":"7","author":"Y Wang","year":"2019","unstructured":"Wang Y, Liu H, Zheng W et al (2019) Multi-objective workflow scheduling with deep-Q-network-based multi-agent reinforcement learning. IEEE Access 7:39974\u201339982","journal-title":"IEEE Access"},{"key":"504_CR25","doi-asserted-by":"crossref","unstructured":"Liu H, Ma Y, Chen P, et al (2020) Scheduling multi-workflows overedge computing resources with time-varying performance, A novel probability-mass function and DQN-based approach. In: Web Services\u2013ICWS 2020: 27th International Conference,Springer, Cham, Honolulu, p. 197\u2013209","DOI":"10.1007\/978-3-030-59618-7_13"},{"key":"504_CR26","doi-asserted-by":"crossref","unstructured":"Wang Y, Jiang J, Xia Y, et al (2018) A multi-stage dynamic game-theoretic approach for multi-workflow scheduling on heterogeneous virtual machines from multiple infrastructure-as-a-service clouds. In: International conference on services computing (SCC). Springer, Zhuhai, pp 137\u2013152","DOI":"10.1007\/978-3-319-94376-3_9"},{"key":"504_CR27","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.1016\/j.ins.2019.10.035","volume":"512","author":"Z Tong","year":"2020","unstructured":"Tong Z, Chen H, Deng X et al (2020) A scheduling scheme in the cloud computing environment using deep Q-learning. Inf Sci 512:1170\u20131191","journal-title":"Inf Sci"},{"key":"504_CR28","doi-asserted-by":"publisher","first-page":"574372","DOI":"10.3389\/fncom.2020.574372","volume":"14","author":"O \u00c7atal","year":"2020","unstructured":"\u00c7atal O, Wauthier S, De Boom C et al (2020) Learning generative state space models for active inference. Front Comput Neurosci 14:574372","journal-title":"Front Comput Neurosci"},{"issue":"11","key":"504_CR29","doi-asserted-by":"publisher","first-page":"2581","DOI":"10.1109\/TMC.2019.2928811","volume":"19","author":"L Huang","year":"2020","unstructured":"Huang L, Bi S, Zhang YJ (2020) Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans Mobile Comput 19(11):2581\u20132593","journal-title":"IEEE Trans Mobile Comput"},{"key":"504_CR30","doi-asserted-by":"crossref","unstructured":"Meng F, Chen P, Wu L (2019) Power allocation in multi-user cellular networks with deep Q learning approach. In: Proc. IEEE Int. Conf. Commun. pp 1\u20136","DOI":"10.1109\/ICC.2019.8761431"},{"key":"504_CR31","doi-asserted-by":"crossref","unstructured":"Jain A, Kumari R (2017) A review on comparison of workflow scheduling algorithms with scientific workflows. In:Proceedings of International Conference on Communication and Networks. vol 508. Springer, Singapore, p. 613\u2013622. https:\/\/link.springer.com\/chapter\/10.1007\/978-981-10-2750-5_63","DOI":"10.1007\/978-981-10-2750-5_63"},{"issue":"8","key":"504_CR32","doi-asserted-by":"publisher","first-page":"e4949","DOI":"10.1002\/cpe.4949","volume":"31","author":"A Rehman","year":"2019","unstructured":"Rehman A, Hussain SS, urRehman Z et al (2019) Multi-objective approach of energy efficient workflow scheduling in cloud environments. Concurr Comput Pract Exp 31(8):e4949","journal-title":"Concurr Comput Pract Exp"},{"issue":"2","key":"504_CR33","doi-asserted-by":"publisher","first-page":"982","DOI":"10.1109\/TASE.2021.3054501","volume":"19","author":"H Li","year":"2021","unstructured":"Li H, Wang B, Yuan Y et al (2021) Scoring and dynamic hierarchy-based NSGA-II for multiobjective workflow scheduling in the cloud. IEEE Trans Autom Sci Eng 19(2):982\u2013993","journal-title":"IEEE Trans Autom Sci Eng"},{"key":"504_CR34","doi-asserted-by":"crossref","unstructured":"Dong T, Xue F, Xiao C, Zhang J (2021) Deep reinforcement learning for dynamic workflow scheduling in cloud environment. 2021 IEEE International Conference on Services Computing (SCC), Chicago, IL, USA. p. 107\u2013115","DOI":"10.1109\/SCC53864.2021.00023"},{"key":"504_CR35","doi-asserted-by":"crossref","unstructured":"Huo D, Wu H, Wang B, et al (2022) A DQN-based workflow task assignment approach in cloud-fog cooperative considering terminal mobility. In: The 6th International Conference on Control Engineering and Artificial Intelligence. pp 78\u201382","DOI":"10.1145\/3522749.3523083"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00504-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00504-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00504-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T17:10:17Z","timestamp":1700241017000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00504-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,26]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["504"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00504-9","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-2431749\/v1","asserted-by":"object"}]},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,26]]},"assertion":[{"value":"31 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Consent has been granted by all authors and there is no confict.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"125"}}