{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T02:35:17Z","timestamp":1781836517949,"version":"3.54.5"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Edge-cloud computing is an emerging approach in which tasks are offloaded from mobile devices to edge or cloud servers. However, Task offloading may result in increased energy consumption and delays, and the decision to offload the task is dependent on various factors such as time-varying radio channels, available computation resources, and the location of devices. As edge-cloud computing is a dynamic and resource-constrained environment, making optimal offloading decisions is a challenging task. This paper aims to optimize offloading and resource allocation to minimize delay and meet computation and communication needs in edge-cloud computing. The problem of optimizing task offloading in the edge-cloud computing environment is a multi-objective problem, for which we employ deep reinforcement learning to find the optimal solution. To accomplish this, we formulate the problem as a Markov decision process and use a Double Deep Q-Network (DDQN) algorithm. Our DDQN-edge-cloud (DDQNEC) scheme dynamically makes offloading decisions by analyzing resource utilization, task constraints, and the current status of the edge-cloud network. Simulation results demonstrate that DDQNEC outperforms heuristic approaches in terms of resource utilization, task offloading, and task rejection.<\/jats:p>","DOI":"10.1186\/s13677-023-00461-3","type":"journal-article","created":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T11:48:06Z","timestamp":1690372086000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":77,"title":["Optimizing task offloading and resource allocation in edge-cloud networks: a DRL approach"],"prefix":"10.1186","volume":"12","author":[{"given":"Ihsan","family":"Ullah","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyun-Kyo","family":"Lim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yeong-Jun","family":"Seok","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youn-Hee","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"issue":"8","key":"461_CR1","doi-asserted-by":"publisher","first-page":"9801","DOI":"10.1007\/s13369-021-06348-2","volume":"47","author":"A Singh","year":"2022","unstructured":"Singh A, Satapathy SC, Roy A, Gutub A (2022) AI-based mobile edge computing for IoT: applications, challenges, and future scope. Arabian J Sci Engin (AJSE) 47(8):9801\u20139831. https:\/\/doi.org\/10.1007\/s13369-021-06348-2","journal-title":"Arabian J Sci Engin (AJSE)"},{"issue":"19","key":"461_CR2","doi-asserted-by":"publisher","first-page":"19450","DOI":"10.1109\/JIOT.2022.3168036","volume":"9","author":"B Dai","year":"2022","unstructured":"Dai B, Niu J, Ren T, Atiquzzaman M (2022) Towards mobility-aware computation offloading and resource allocation in end-edge-cloud orchestrated computing. IEEE Internet Things J 9(19):19450\u201362","journal-title":"IEEE Internet Things J"},{"issue":"3","key":"461_CR3","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1109\/MWC.2019.1800411","volume":"26","author":"Y Dai","year":"2019","unstructured":"Dai Y, Xu D, Maharjan S, Qiao G, Zhang Y (2019) Artificial intelligence empowered edge computing and caching for internet of vehicles. IEEE Wirel Commun 26(3):12\u201318","journal-title":"IEEE Wirel Commun"},{"issue":"1","key":"461_CR4","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1109\/COMST.2019.2943405","volume":"22","author":"TK Rodrigues","year":"2019","unstructured":"Rodrigues TK, Suto K, Nishiyama H, Liu J, Kato N (2019) Machine learning meets computation and communication control in evolving edge and cloud: challenges and future perspective. IEEE Commun Surv Tutor 22(1):38\u201367","journal-title":"IEEE Commun Surv Tutor"},{"issue":"9","key":"461_CR5","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1109\/TC.2018.2818144","volume":"67","author":"TG Rodrigues","year":"2018","unstructured":"Rodrigues TG, Suto K, Nishiyama H, Kato N, Temma K (2018) Cloudlets activation scheme for scalable mobile edge computing with transmission power control and virtual machine migration. IEEE Trans Comput 67(9):1287\u20131300","journal-title":"IEEE Trans Comput"},{"issue":"8","key":"461_CR6","doi-asserted-by":"publisher","first-page":"7944","DOI":"10.1109\/TVT.2019.2917890","volume":"68","author":"J Zhao","year":"2019","unstructured":"Zhao J, Li Q, Gong Y, Zhang K (2019) Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Trans Veh Technol 68(8):7944\u20137956","journal-title":"IEEE Trans Veh Technol"},{"issue":"6","key":"461_CR7","first-page":"2011","volume":"14","author":"TT Nguyen","year":"2019","unstructured":"Nguyen TT, Le LB, Le-Trung Q (2019) Computation offloading in MIMO based mobile edge computing systems under perfect and imperfect CSI estimation. IEEE Trans Serv Comput 14(6):2011\u20132025","journal-title":"IEEE Trans Serv Comput"},{"issue":"12","key":"461_CR8","doi-asserted-by":"publisher","first-page":"12313","DOI":"10.1109\/TVT.2018.2876804","volume":"67","author":"Y Dai","year":"2018","unstructured":"Dai Y, Xu D, Maharjan S, Zhang Y (2018) Joint computation offloading and user association in multi-task mobile edge computing. IEEE Trans Veh Technol 67(12):12313\u201312325","journal-title":"IEEE Trans Veh Technol"},{"key":"461_CR9","doi-asserted-by":"crossref","unstructured":"Ullah I, Lim H.-K.,  Seok Y.-J., and Han Y.-H (2022) \u201cOptimal task offloading with deep Q-network for edge-cloud computing environment,\u201d presented at the 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), IEEE, pp. 406\u2013411","DOI":"10.1109\/ICTC55196.2022.9952511"},{"issue":"1","key":"461_CR10","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.dcan.2018.10.003","volume":"5","author":"L Huang","year":"2019","unstructured":"Huang L, Feng X, Zhang C, Qian L, Wu Y (2019) Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing. Digit Commun Netw 5(1):10\u201317","journal-title":"Digit Commun Netw"},{"key":"461_CR11","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.jnca.2018.06.009","volume":"119","author":"F Gu","year":"2018","unstructured":"Gu F, Niu J, Qi Z, Atiquzzaman M (2018) Partitioning and offloading in smart mobile devices for mobile cloud computing: State of the art and future directions. J Netw Comput Appl 119:83\u201396","journal-title":"J Netw Comput Appl"},{"issue":"11","key":"461_CR12","doi-asserted-by":"publisher","first-page":"2581","DOI":"10.1109\/TMC.2019.2928811","volume":"19","author":"L Huang","year":"2019","unstructured":"Huang L, Bi S, Zhang Y-JA (2019) Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans Mob Comput 19(11):2581\u20132593","journal-title":"IEEE Trans Mob Comput"},{"key":"461_CR13","doi-asserted-by":"crossref","unstructured":"Li J,\u00a0 Gao H,\u00a0 Lv T,\u00a0 Lu Y (2018)\u00a0 \u201cDeep reinforcement learning based computation offloading and resource allocation for MEC\u201d, presented at the, 2018 IEEE wireless communications and networking conference (WCNC) IEEE, pp. 1\u20136","DOI":"10.1109\/WCNC.2018.8377343"},{"issue":"6","key":"461_CR14","doi-asserted-by":"publisher","first-page":"1133","DOI":"10.1109\/JSAC.2020.2986615","volume":"38","author":"X Xiong","year":"2020","unstructured":"Xiong X, Zheng K, Lei L, Hou L (2020) Resource allocation based on deep reinforcement learning in IoT edge computing. IEEE J Sel Areas Commun 38(6):1133\u20131146","journal-title":"IEEE J Sel Areas Commun"},{"issue":"3","key":"461_CR15","doi-asserted-by":"publisher","first-page":"4005","DOI":"10.1109\/JIOT.2018.2876279","volume":"6","author":"X Chen","year":"2018","unstructured":"Chen X, Zhang H, Wu C, Mao S, Ji Y, Bennis M (2018) Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J 6(3):4005\u20134018","journal-title":"IEEE Internet Things J"},{"key":"461_CR16","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1016\/j.future.2019.07.019","volume":"102","author":"H Lu","year":"2020","unstructured":"Lu H, Gu C, Luo F, Ding W, Liu X (2020) Optimization of lightweight task offloading strategy for mobile edge computing based on deep reinforcement learning. Future Gener Comput Syst 102:847\u2013861","journal-title":"Future Gener Comput Syst"},{"issue":"5","key":"461_CR17","doi-asserted-by":"publisher","first-page":"7635","DOI":"10.1109\/JIOT.2019.2903191","volume":"6","author":"K Zhang","year":"2019","unstructured":"Zhang K, Zhu Y, Leng S, He Y, Maharjan S, Zhang Y (2019) Deep learning empowered task offloading for mobile edge computing in urban informatics. IEEE Internet Things J 6(5):7635\u20137647","journal-title":"IEEE Internet Things J"},{"issue":"11","key":"461_CR18","doi-asserted-by":"publisher","first-page":"11158","DOI":"10.1109\/TVT.2019.2935450","volume":"68","author":"Y Liu","year":"2019","unstructured":"Liu Y, Yu H, Xie S, Zhang Y (2019) Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks. IEEE Trans Veh Technol 68(11):11158\u201311168","journal-title":"IEEE Trans Veh Technol"},{"issue":"3","key":"461_CR19","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1109\/TETC.2019.2902661","volume":"9","author":"J Wang","year":"2019","unstructured":"Wang J, Zhao L, Liu J, Kato N (2019) Smart resource allocation for mobile edge computing: a deep reinforcement learning approach. IEEE Trans Emerg Top Comput 9(3):1529\u20131541","journal-title":"IEEE Trans Emerg Top Comput"},{"issue":"7","key":"461_CR20","doi-asserted-by":"publisher","first-page":"1603","DOI":"10.1109\/TPDS.2020.3046737","volume":"32","author":"C Liu","year":"2020","unstructured":"Liu C, Tang F, Hu Y, Li K, Tang Z, Li K (2020) Distributed task migration optimization in MEC by extending multi-agent deep reinforcement learning approach. IEEE Trans Parallel Distrib Syst 32(7):1603\u20131614","journal-title":"IEEE Trans Parallel Distrib Syst"},{"key":"461_CR21","doi-asserted-by":"crossref","unstructured":"Chen X,\u00a0 Zhang H,\u00a0 Wu C,\u00a0 Mao S,\u00a0 Ji Y,\u00a0 Bennis M (2018)\u00a0 \u201cPerformance optimization in mobile-edge computing via deep reinforcement learning\u201d, presented at the, 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall) IEEE , pp. 1\u20136","DOI":"10.1109\/VTCFall.2018.8690980"},{"key":"461_CR22","doi-asserted-by":"crossref","unstructured":"Xie Y, Xu Z, Xu J, Gong S, and Wang Y (2019) \u201cBackscatter-aided hybrid data offloading for mobile edge computing via deep reinforcement learning,\u201d presented at the International Conference on Machine Learning and Intelligent Communications, Springer, pp. 525\u2013537","DOI":"10.1007\/978-3-030-32388-2_45"},{"issue":"6","key":"461_CR23","doi-asserted-by":"publisher","first-page":"879","DOI":"10.3390\/electronics11060879","volume":"11","author":"K Tian","year":"2022","unstructured":"Tian K, Chai H, Liu Y, Liu B (2022) Edge Intelligence empowered dynamic offloading and resource management of MEC for Smart City internet of things. Electronics 11(6):879","journal-title":"Electronics"},{"key":"461_CR24","doi-asserted-by":"publisher","first-page":"54074","DOI":"10.1109\/ACCESS.2020.2981434","volume":"8","author":"T Alfakih","year":"2020","unstructured":"Alfakih T, Hassan MM, Gumaei A, Savaglio C, Fortino G (2020) Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. IEEE Access 8:54074\u201354084","journal-title":"IEEE Access"},{"key":"461_CR25","doi-asserted-by":"publisher","first-page":"107660","DOI":"10.1016\/j.knosys.2021.107660","volume":"235","author":"M Chen","year":"2022","unstructured":"Chen M, Liu W, Wang T, Zhang S, Liu A (2022) A game-based deep reinforcement learning approach for energy-efficient computation in MEC systems. Knowl.-Based Syst 235:107660","journal-title":"Knowl.-Based Syst"},{"issue":"10","key":"461_CR26","doi-asserted-by":"publisher","first-page":"9255","DOI":"10.1109\/JIOT.2020.2981557","volume":"7","author":"H Lu","year":"2020","unstructured":"Lu H, He X, Du M, Ruan X, Sun Y, Wang K (2020) Edge QoE: Computation offloading with deep reinforcement learning for Internet of Things. IEEE Internet Things J 7(10):9255\u20139265","journal-title":"IEEE Internet Things J"},{"issue":"9","key":"461_CR27","doi-asserted-by":"publisher","first-page":"2968","DOI":"10.1109\/LCOMM.2021.3094842","volume":"25","author":"J Chen","year":"2021","unstructured":"Chen J, Wu Z (2021) Dynamic computation offloading with energy harvesting devices: a graph-based deep reinforcement learning approach. IEEE Commun Lett 25(9):2968\u20132972","journal-title":"IEEE Commun Lett"},{"issue":"1","key":"461_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13638-020-01801-6","volume":"2020","author":"Z Chen","year":"2020","unstructured":"Chen Z, Wang X (2020) Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach. EURASIP J Wirel Commun Netw 2020(1):1\u201321","journal-title":"EURASIP J Wirel Commun Netw"},{"key":"461_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.119743","volume":"221","author":"M Chen","year":"2023","unstructured":"Chen M, Yi M, Huang M, Huang G, Ren Y, Liu A (2023) A novel deep policy gradient action quantization for trusted collaborative computation in intelligent vehicle networks. Expert Syst Appl 221:119743","journal-title":"Expert Syst Appl"},{"key":"461_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118394","volume":"210","author":"M Chen","year":"2022","unstructured":"Chen M et al (2022) GPDS: a multi-agent deep reinforcement learning game for anti-jamming secure computing in MEC network. Expert Syst Appl 210:118394","journal-title":"Expert Syst Appl"},{"key":"461_CR31","doi-asserted-by":"crossref","unstructured":"Liu K.-H,\u00a0 and  Liao W (2020) \u201cIntelligent offloading for multi-access edge computing: A new actor-critic approach,\u201d presented at the ICC 2020\u20132020 IEEE International Conference on Communications (ICC), IEEE, pp. 1\u20136","DOI":"10.1109\/ICC40277.2020.9149387"},{"issue":"5","key":"461_CR32","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.1109\/TPDS.2020.3042599","volume":"32","author":"X Qiu","year":"2020","unstructured":"Qiu X, Zhang W, Chen W, Zheng Z (2020) Distributed and collective deep reinforcement learning for computation offloading: a practical perspective. IEEE Trans Parallel Distrib Syst 32(5):1085\u20131101","journal-title":"IEEE Trans Parallel Distrib Syst"},{"key":"461_CR33","doi-asserted-by":"publisher","first-page":"85204","DOI":"10.1109\/ACCESS.2020.2991773","volume":"8","author":"Y Li","year":"2020","unstructured":"Li Y, Qi F, Wang Z, Yu X, Shao S (2020) Distributed edge computing offloading algorithm based on deep reinforcement learning. IEEE Access 8:85204\u201385215","journal-title":"IEEE Access"},{"key":"461_CR34","doi-asserted-by":"crossref","unstructured":"Chai F, Zhang Q, Yao H, Xin X, Gao R, Guizani, M (2023) Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT. IEEE Trans Veh Technol 1\u201315","DOI":"10.1109\/TVT.2023.3238771"},{"issue":"7","key":"461_CR35","doi-asserted-by":"publisher","first-page":"3293","DOI":"10.1007\/s11276-022-03054-1","volume":"28","author":"S Chen","year":"2022","unstructured":"Chen S, Ge X, Wang Q, Miao Y, Ruan X (2022) DDPG-based intelligent rechargeable fog computation offloading for IoT. Wirel Netw 28(7):3293\u20133304","journal-title":"Wirel Netw"},{"key":"461_CR36","doi-asserted-by":"crossref","unstructured":"Cheng M,\u00a0 Li J,\u00a0 Nazarian S (2018 ) \u201cDRL-cloud: Deep reinforcement learning-based resource provisioning and task scheduling for cloud service providers\u201d, presented at the, 2018 23rd Asia and South pacific design automation conference (ASP-DAC) IEEE ,129:134","DOI":"10.1109\/ASPDAC.2018.8297294"},{"key":"461_CR37","doi-asserted-by":"crossref","unstructured":"Nath S,\u00a0 and Wu J (2020) \u201cDynamic Computation Offloading and Resource Allocation for Multi-user Mobile Edge Computing,\u201d presented at the GLOBECOM 2020\u20132020 IEEE Global Communications Conference, IEEE, pp. 1\u20136","DOI":"10.1109\/GLOBECOM42002.2020.9348161"},{"issue":"24","key":"461_CR38","doi-asserted-by":"publisher","first-page":"17508","DOI":"10.1109\/JIOT.2021.3081694","volume":"8","author":"J Chen","year":"2021","unstructured":"Chen J, Xing H, Xiao Z, Xu L, Tao T (2021) A DRL agent for jointly optimizing computation offloading and resource allocation in MEC. IEEE Internet Things J 8(24):17508\u201317524","journal-title":"IEEE Internet Things J"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00461-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00461-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00461-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T11:49:51Z","timestamp":1690372191000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00461-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["461"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00461-3","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-2522525\/v1","asserted-by":"object"}]},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]},"assertion":[{"value":"28 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"112"}}