{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T01:33:15Z","timestamp":1772933595913,"version":"3.50.1"},"reference-count":68,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T00:00:00Z","timestamp":1700265600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T00:00:00Z","timestamp":1700265600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Major Science and Technology Innovation Project of Shandong Province","award":["No.2019TSLH0214"],"award-info":[{"award-number":["No.2019TSLH0214"]}]},{"name":"Tai Shan Industry Leading Talent Project","award":["No.tscy20180416"],"award-info":[{"award-number":["No.tscy20180416"]}]},{"name":"National major special projects","award":["2021YFA1000102"],"award-info":[{"award-number":["2021YFA1000102"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Recently, the development of Low Earth Orbit (LEO) satellites and the advancement of the Mobile Edge Computing (MEC) paradigm have driven the emergence of the Satellite Mobile Edge Computing (Sat-MEC). Sat-MEC has been developed to support communication and task computation for Internet of Things (IoT) Mobile Devices (IMDs) in the absence of terrestrial networks. However, due to the heterogeneity of tasks and Sat-MEC servers, it is still a great challenge to efficiently schedule tasks in Sat-MEC servers. Here, we propose a scheduling algorithm based on the Deep Reinforcement Learning (DRL) method in the Sat-MEC architecture to minimize the average task processing time. We consider multiple factors, including the cooperation between LEO satellites, the concurrency and heterogeneity of tasks, the dynamics of LEO satellites, the heterogeneity of the computational capacity of Sat-MEC servers, and the heterogeneity of the initial queue for task computation. Further, we use the self-attention mechanism to act as a Q-network to extract high-dimensional dynamic information of tasks and Sat-MEC servers. In this work, we model the Sat-MEC environment simulation at the application level and propose a DRL-based task scheduling algorithm. The simulation results confirm the effectiveness of our proposed scheduling algorithm, which reduces the average task processing time by 22.1\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\%$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mo>%<\/mml:mo>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    , 30.6\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\%$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mo>%<\/mml:mo>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    , and 41.3\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\%$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mo>%<\/mml:mo>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    , compared to the genetic algorithm(GA), the greedy algorithm, and the random algorithm, respectively.\n                  <\/jats:p>","DOI":"10.1186\/s13677-023-00538-z","type":"journal-article","created":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T03:02:07Z","timestamp":1700276527000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Minimize average tasks processing time in satellite mobile edge computing systems via a deep reinforcement learning method"],"prefix":"10.1186","volume":"12","author":[{"given":"Shanchen","family":"Pang","sequence":"first","affiliation":[]},{"given":"Jianyang","family":"Zheng","sequence":"additional","affiliation":[]},{"given":"Min","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Sibo","family":"Qiao","sequence":"additional","affiliation":[]},{"given":"Xiao","family":"He","sequence":"additional","affiliation":[]},{"given":"Changnan","family":"Gao","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,11,18]]},"reference":[{"key":"538_CR1","doi-asserted-by":"crossref","unstructured":"Qian L, Luo Z, Du Y, Guo L (2009) Cloud computing: An overview. In: Cloud Computing: First International Conference, CloudCom 2009, Beijing, China, December 1-4, 2009. Proceedings 1, Springer, pp 626\u2013631","DOI":"10.1007\/978-3-642-10665-1_63"},{"key":"538_CR2","doi-asserted-by":"publisher","unstructured":"Yi S, Li C, Li Q (2015) A Survey of Fog Computing: Concepts, Applications and Issues. In Proceedings of the 2015 Workshop on Mobile Big Data (Mobidata '15). Association for Computing Machinery, New York, 37\u201342. https:\/\/doi.org\/10.1145\/2757384.2757397","DOI":"10.1145\/2757384.2757397"},{"issue":"5","key":"538_CR3","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/MC.2016.145","volume":"49","author":"W Shi","year":"2016","unstructured":"Shi W, Dustdar S (2016) The promise of edge computing. Computer 49(5):78\u201381","journal-title":"Computer"},{"key":"538_CR4","doi-asserted-by":"publisher","first-page":"86769","DOI":"10.1109\/ACCESS.2019.2923610","volume":"7","author":"Q Qi","year":"2019","unstructured":"Qi Q, Tao F (2019) A smart manufacturing service system based on edge computing, fog computing, and cloud computing. IEEE Access 7:86769\u201386777","journal-title":"IEEE Access"},{"issue":"2","key":"538_CR5","doi-asserted-by":"publisher","first-page":"1193","DOI":"10.1109\/COMST.2021.3059644","volume":"23","author":"NN Dao","year":"2021","unstructured":"Dao NN, Pham QV, Tu NH, Thanh TT, Bao VNQ, Lakew DS, Cho S (2021) Survey on aerial radio access networks: Toward a comprehensive 6g access infrastructure. IEEE Commun Surv Tutor 23(2):1193\u20131225","journal-title":"IEEE Commun Surv Tutor"},{"issue":"107","key":"538_CR6","first-page":"496","volume":"182","author":"A Shakarami","year":"2020","unstructured":"Shakarami A, Ghobaei-Arani M, Shahidinejad A (2020) A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective. Comput Netw 182(107):496","journal-title":"Comput Netw"},{"issue":"9","key":"538_CR7","doi-asserted-by":"publisher","first-page":"1719","DOI":"10.1002\/spe.2839","volume":"50","author":"A Shakarami","year":"2020","unstructured":"Shakarami A, Shahidinejad A, Ghobaei-Arani M (2020) A review on the computation offloading approaches in mobile edge computing: A g ame-theoretic perspective. Softw Pract Experience 50(9):1719\u20131759","journal-title":"Softw Pract Experience"},{"key":"538_CR8","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1007\/s10723-020-09530-2","volume":"18","author":"A Shakarami","year":"2020","unstructured":"Shakarami A, Ghobaei-Arani M, Masdari M, Hosseinzadeh M (2020) A survey on the computation offloading approaches in mobile edge\/cloud computing environment: a stochastic-based perspective. J Grid Comput 18:639\u2013671","journal-title":"J Grid Comput"},{"key":"538_CR9","doi-asserted-by":"publisher","first-page":"3522","DOI":"10.1007\/s11227-018-2292-y","volume":"75","author":"D Usha Nandini","year":"2019","unstructured":"Usha Nandini D, Leni ES (2019) Efficient shadow detection by using PSO segmentation and region-based boundary detection technique. J Supercomput 75:3522\u20133533","journal-title":"J Supercomput"},{"issue":"4","key":"538_CR10","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1287\/mnsc.45.4.560","volume":"45","author":"TK Das","year":"1999","unstructured":"Das TK, Gosavi A, Mahadevan S, Marchalleck N (1999) Solving semi-Markov decision problems using average reward reinforcement learning. Manag Sci 45(4):560\u2013574","journal-title":"Manag Sci"},{"key":"538_CR11","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1613\/jair.301","volume":"4","author":"LP Kaelbling","year":"1996","unstructured":"Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: A survey. J Artif Intell Res 4:237\u2013285","journal-title":"J Artif Intell Res"},{"key":"538_CR12","doi-asserted-by":"publisher","first-page":"102974","DOI":"10.1016\/j.jnca.2021.102974","volume":"178","author":"A Shakarami","year":"2021","unstructured":"Shakarami A, Shahidinejad A, Ghobaei-Arani M (2021) An autonomous computation offloading strategy in mobile edge computing: a deep learning-based hybrid approach. J Netw Comput Appl 178:102974","journal-title":"J Netw Comput Appl"},{"key":"538_CR13","doi-asserted-by":"publisher","unstructured":"van Hasselt H, Guez A, Silver D (2016) Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence 30(1). https:\/\/doi.org\/10.1609\/aaai.v30i1.10295","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"538_CR14","doi-asserted-by":"publisher","unstructured":"Imambi S, Prakash KB, Kanagachidambaresan GR (2021) PyTorch[J]. Programming with TensorFlow: Solution for Edge Computing Applications 87\u2013104. https:\/\/doi.org\/10.1007\/978-3-030-57077-4_10","DOI":"10.1007\/978-3-030-57077-4_10"},{"key":"538_CR15","doi-asserted-by":"publisher","first-page":"12510","DOI":"10.1109\/ACCESS.2019.2963068","volume":"8","author":"Y Wang","year":"2019","unstructured":"Wang Y, Yang J, Guo X, Qu Z (2019) A game-theoretic approach to computation offloading in satellite edge computing. IEEE Access 8:12510\u201312520","journal-title":"IEEE Access"},{"issue":"8","key":"538_CR16","doi-asserted-by":"publisher","first-page":"199","DOI":"10.23919\/JCC.2020.08.016","volume":"17","author":"C Li","year":"2020","unstructured":"Li C, Zhang Y, Hao X, Huang T (2020) Jointly optimized request dispatching and service placement for MEC in LEO network. China Commun 17(8):199\u2013208","journal-title":"China Commun"},{"key":"538_CR17","doi-asserted-by":"crossref","unstructured":"Wang H, Han J, Cao S, Zhang X (2021) Computation offloading strategy of multi-satellite cooperative tasks based on genetic algorithm in satellite edge computing. In: 2021 International Conference on Space-Air-Ground Computing (SAGC), IEEE, pp 22\u201328","DOI":"10.1109\/SAGC52752.2021.00011"},{"issue":"11","key":"538_CR18","doi-asserted-by":"publisher","first-page":"9164","DOI":"10.1109\/JIOT.2021.3056569","volume":"8","author":"Q Tang","year":"2021","unstructured":"Tang Q, Fei Z, Li B, Han Z (2021) Computation offloading in LEO satellite networks with hybrid cloud and edge computing. IEEE Internet Things J 8(11):9164\u20139176","journal-title":"IEEE Internet Things J"},{"key":"538_CR19","doi-asserted-by":"crossref","unstructured":"Zhu D, Liu H, Li T, Sun J, Liang J, Zhang H, Geng L, Liu Y (2021) Deep reinforcement learning-based task offloading in satellite-terrestrial edge computing networks. In: 2021 IEEE Wireless Communications and Networking Conference (WCNC), IEEE, pp 1\u20137","DOI":"10.1109\/WCNC49053.2021.9417127"},{"issue":"8","key":"538_CR20","doi-asserted-by":"publisher","first-page":"5742","DOI":"10.1109\/JIOT.2021.3052542","volume":"9","author":"S Yu","year":"2021","unstructured":"Yu S, Gong X, Shi Q, Wang X, Chen X (2021) EC-SAGINs: Edge-computing-enhanced space-air-ground-integrated networks for internet of vehicles. IEEE Internet Things J 9(8):5742\u20135754","journal-title":"IEEE Internet Things J"},{"issue":"3","key":"538_CR21","doi-asserted-by":"publisher","first-page":"3992","DOI":"10.1109\/JSYST.2020.3041706","volume":"15","author":"S Mao","year":"2020","unstructured":"Mao S, He S, Wu J (2020) Joint UAV position optimization and resource scheduling in space-air-ground integrated networks with mixed cloud-edge computing. IEEE Syst J 15(3):3992\u20134002","journal-title":"IEEE Syst J"},{"issue":"4","key":"538_CR22","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/MCOM.001.2100818","volume":"60","author":"P Cassar\u00e1","year":"2022","unstructured":"Cassar\u00e1 P, Gotta A, Marchese M, Patrone F (2022) Orbital edge offloading on mega-LEO satellite constellations for equal access to computing. IEEE Commun Mag 60(4):32\u201336","journal-title":"IEEE Commun Mag"},{"key":"538_CR23","doi-asserted-by":"crossref","unstructured":"He Y, Ren J, Yu G, Cai Y (2019) Joint computation offloading and resource allocation in d2d enabled mec networks. In: ICC 2019-2019 IEEE International Conference on Communications (ICC), IEEE, pp 1\u20136","DOI":"10.1109\/ICC.2019.8761169"},{"key":"538_CR24","doi-asserted-by":"publisher","unstructured":"Seng S, Li X, Luo C, Ji H, Zhang H (2019) A d2d-assisted MEC computation offloading in the blockchain-based framework for UDNs. In: ICC 2019 - 2019 IEEE International Conference on Communications (ICC), pp 1\u20136. https:\/\/doi.org\/10.1109\/ICC.2019.8762023","DOI":"10.1109\/ICC.2019.8762023"},{"issue":"1","key":"538_CR25","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1109\/TMC.2021.3075947","volume":"22","author":"S Zang","year":"2023","unstructured":"Zang S, Bao W, Yeoh PL, Vucetic B, Li Y (2023) Soar: Smart online aggregated reservation for mobile edge computing brokerage services. IEEE Trans Mob Comput 22(1):527\u2013540. https:\/\/doi.org\/10.1109\/TMC.2021.3075947","journal-title":"IEEE Trans Mob Comput"},{"issue":"1","key":"538_CR26","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1109\/TNSE.2022.3207214","volume":"10","author":"Y Zhang","year":"2022","unstructured":"Zhang Y, Chen C, Liu L, Lan D, Jiang H, Wan S (2022) Aerial edge computing on orbit: A task offloading and allocation scheme. IEEE Trans Netw Sci Eng 10(1):275\u2013285","journal-title":"IEEE Trans Netw Sci Eng"},{"key":"538_CR27","doi-asserted-by":"publisher","unstructured":"Chai F, Zhang Q, Yao H, Xin X, Gao R, Guizani M (2023) Joint Multi-Task Offloading and Resource Allocation for Mobile Edge Computing Systems in Satellite IoT.\u00a0IEEE Trans Veh Technol\u00a072(6):7783\u20137795. https:\/\/doi.org\/10.1109\/TVT.2023.3238771","DOI":"10.1109\/TVT.2023.3238771"},{"issue":"2","key":"538_CR28","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1109\/TNSE.2021.3130251","volume":"9","author":"J Liu","year":"2021","unstructured":"Liu J, Zhao X, Qin P, Geng S, Meng S (2021) Joint dynamic task offloading and resource scheduling for WPT enabled space-air-ground power internet of things. IEEE Trans Netw Sci Eng 9(2):660\u2013677","journal-title":"IEEE Trans Netw Sci Eng"},{"key":"538_CR29","doi-asserted-by":"publisher","first-page":"37191","DOI":"10.1109\/ACCESS.2020.2975741","volume":"8","author":"MK Hussein","year":"2020","unstructured":"Hussein MK, Mousa MH (2020) Efficient task offloading for IoT-based applications in fog computing using ant colony optimization. IEEE Access 8:37191\u201337201","journal-title":"IEEE Access"},{"issue":"12","key":"538_CR30","doi-asserted-by":"publisher","first-page":"2519","DOI":"10.1002\/spe.2867","volume":"51","author":"S Javanmardi","year":"2021","unstructured":"Javanmardi S, Shojafar M, Persico V, Pescap\u00e8 A (2021) FPFTS: A joint fuzzy particle swarm optimization mobility-aware approach to fog task scheduling algorithm for internet of things devices. Softw Pract Experience 51(12):2519\u20132539","journal-title":"Softw Pract Experience"},{"key":"538_CR31","doi-asserted-by":"publisher","unstructured":"Zhang X et al. Energy-Efficient Computation Peer Offloading in Satellite Edge Computing Networks. IEEE Trans Mob Comput. https:\/\/doi.org\/10.1109\/TMC.2023.3269801","DOI":"10.1109\/TMC.2023.3269801"},{"issue":"2","key":"538_CR32","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1007\/s11277-023-10310-w","volume":"130","author":"KM Matrouk","year":"2023","unstructured":"Matrouk KM, Matrouk AD (2023) Mobility aware-task scheduling and virtual fog for offloading in IoT-fog-cloud environment. Wirel Pers Commun 130(2):801\u2013836","journal-title":"Wirel Pers Commun"},{"key":"538_CR33","doi-asserted-by":"crossref","unstructured":"Chen J, Xing H, Xiao Z, Xu L, Tao T (2021) A DRL agent for jointly optimizing computation offloading and resource allocation in MEC. IEEE Internet Things J 8(24):17508\u201317524","DOI":"10.1109\/JIOT.2021.3081694"},{"key":"538_CR34","doi-asserted-by":"crossref","unstructured":"Seid AM, Boateng GO, Anokye S, Kwantwi T, Sun G, Liu G (2021) Collaborative computation offloading and resource allocation in multi-UAV-assisted IoT networks: A deep reinforcement learning approach. IEEE Internet Things J 8(15):12203\u201312218","DOI":"10.1109\/JIOT.2021.3063188"},{"key":"538_CR35","first-page":"1","volume":"2020","author":"F Zheng","year":"2020","unstructured":"Zheng F, Pi Z, Zhou Z, Wang K (2020) Leo satellite channel allocation scheme based on reinforcement learning. Mob Inf Syst 2020:1\u201310","journal-title":"Mob Inf Syst"},{"key":"538_CR36","unstructured":"Liu L, Chang Z, Guo X, Ristaniemi T (2017) Multi-objective optimization for computation offloading in mobile-edge computing. In: 2017 IEEE symposium on computers and communications (ISCC), IEEE, pp 832\u2013837"},{"issue":"11","key":"538_CR37","doi-asserted-by":"publisher","first-page":"12486","DOI":"10.1007\/s11227-021-03781-w","volume":"77","author":"W Li","year":"2021","unstructured":"Li W, Jin S (2021) Performance evaluation and optimization of a task offloading strategy on the mobile edge computing with edge heterogeneity. J Supercomput 77(11):12486\u201312507","journal-title":"J Supercomput"},{"issue":"3","key":"538_CR38","doi-asserted-by":"publisher","first-page":"779","DOI":"10.3390\/s21030779","volume":"21","author":"S Chen","year":"2021","unstructured":"Chen S, Li Q, Zhou M, Abusorrah A (2021) Recent advances in collaborative scheduling of computing tasks in an edge computing paradigm. Sensors 21(3):779","journal-title":"Sensors"},{"issue":"2","key":"538_CR39","first-page":"544","volume":"35","author":"Z Sharif","year":"2023","unstructured":"Sharif Z, Jung LT, Ayaz M, Yahya M, Pitafi S (2023) Priority-based task scheduling and resource allocation in edge computing for health monitoring system. J King Saud Univ-Comput Inf Sci 35(2):544\u2013559","journal-title":"J King Saud Univ-Comput Inf Sci"},{"key":"538_CR40","doi-asserted-by":"publisher","unstructured":"Zhou W et al (2023) Priority-Aware Resource Scheduling for UAV-Mounted Mobile Edge Computing Networks.\u00a0IEEE Trans Veh Technol\u00a072(7):9682\u20139687. https:\/\/doi.org\/10.1109\/TVT.2023.3247431","DOI":"10.1109\/TVT.2023.3247431"},{"issue":"3","key":"538_CR41","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1109\/JSTSP.2022.3140660","volume":"16","author":"Y Guo","year":"2022","unstructured":"Guo Y, Zhao R, Lai S, Fan L, Lei X, Karagiannidis GK (2022) Distributed machine learning for multiuser mobile edge computing systems. IEEE J Sel Top Signal Process 16(3):460\u2013473","journal-title":"IEEE J Sel Top Signal Process"},{"key":"538_CR42","doi-asserted-by":"publisher","unstructured":"Wang H, An J, Zhou H (2023) Task assignment strategy in LEO-muti-access edge computing based on matching game. Computing 105:1571\u20131596. https:\/\/doi.org\/10.1007\/s00607-023-01151-3","DOI":"10.1007\/s00607-023-01151-3"},{"issue":"1","key":"538_CR43","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s10922-022-09696-y","volume":"31","author":"V Jain","year":"2023","unstructured":"Jain V, Kumar B (2023) Qos-aware task offloading in fog environment using multi-agent deep reinforcement learning. J Netw Syst Manag 31(1):7","journal-title":"J Netw Syst Manag"},{"issue":"12","key":"538_CR44","doi-asserted-by":"publisher","first-page":"2357","DOI":"10.1109\/LCOMM.2019.2943461","volume":"23","author":"X Diao","year":"2019","unstructured":"Diao X, Zheng J, Cai Y, Wu Y, Anpalagan A (2019) Fair data allocation and trajectory optimization for UAV-assisted mobile edge computing. IEEE Commun Lett 23(12):2357\u20132361","journal-title":"IEEE Commun Lett"},{"issue":"102","key":"538_CR45","first-page":"725","volume":"124","author":"S Pang","year":"2023","unstructured":"Pang S, He X, Yu S, Wang M, Qiao S, Gui H, Qi Y (2023) A Stackelberg game scheme for pricing and task offloading based on idle node-assisted edge computational model. Simul Model Pract Theory 124(102):725","journal-title":"Simul Model Pract Theory"},{"key":"538_CR46","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1007\/s12083-020-00985-4","volume":"14","author":"F Zeng","year":"2021","unstructured":"Zeng F, Chen Y, Yao L, Wu J (2021) A novel reputation incentive mechanism and game theory analysis for service caching in software-defined vehicle edge computing. Peer Peer Netw Appl 14:467\u2013481","journal-title":"Peer Peer Netw Appl"},{"issue":"11","key":"538_CR47","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1109\/CC.2018.8543056","volume":"15","author":"F Wei","year":"2018","unstructured":"Wei F, Chen S, Zou W (2018) A greedy algorithm for task offloading in mobile edge computing system. China Commun 15(11):149\u2013157","journal-title":"China Commun"},{"issue":"2","key":"538_CR48","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1109\/TCSS.2021.3049152","volume":"8","author":"Y Fan","year":"2021","unstructured":"Fan Y, Wang L, Wu W, Du D (2021) Cloud\/edge computing resource allocation and pricing for mobile blockchain: an iterative greedy and search approach. IEEE Trans Comput Soc Syst 8(2):451\u2013463","journal-title":"IEEE Trans Comput Soc Syst"},{"key":"538_CR49","doi-asserted-by":"publisher","first-page":"107446","DOI":"10.1016\/j.comnet.2020.107446","volume":"182","author":"N Zhang","year":"2020","unstructured":"Zhang N, Guo S, Dong Y, Liu D (2020) Joint task offloading and data caching in mobile edge computing networks. Comput Netw 182:107446","journal-title":"Comput Netw"},{"issue":"1","key":"538_CR50","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1109\/SURV.2012.022412.00172","volume":"15","author":"C Phillips","year":"2012","unstructured":"Phillips C, Sicker D, Grunwald D (2012) A survey of wireless path loss prediction and coverage mapping methods. IEEE Commun Surv Tutor 15(1):255\u2013270","journal-title":"IEEE Commun Surv Tutor"},{"key":"538_CR51","doi-asserted-by":"crossref","unstructured":"Tang Z, Zhou H, Ma T, Yu K, Shen XS (2021) Leveraging LEO assisted cloud-edge collaboration for energy efficient computation offloading. In: 2021 IEEE Global Communications Conference (GLOBECOM), IEEE, pp 1\u20136","DOI":"10.1109\/GLOBECOM46510.2021.9685309"},{"issue":"1","key":"538_CR52","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1109\/TWC.2018.2875980","volume":"18","author":"B Di","year":"2018","unstructured":"Di B, Zhang H, Song L, Li Y, Li GY (2018) Ultra-dense LEO: Integrating terrestrial-satellite networks into 5g and beyond for data offloading. IEEE Trans Wirel Commun 18(1):47\u201362","journal-title":"IEEE Trans Wirel Commun"},{"issue":"9","key":"538_CR53","doi-asserted-by":"publisher","first-page":"2561","DOI":"10.1080\/00207543.2019.1620362","volume":"58","author":"Y Zhou","year":"2020","unstructured":"Zhou Y, Jj Yang, Huang Z (2020) Automatic design of scheduling policies for dynamic flexible job shop scheduling via surrogate-assisted cooperative co-evolution genetic programming. Int J Prod Res 58(9):2561\u20132580","journal-title":"Int J Prod Res"},{"key":"538_CR54","doi-asserted-by":"publisher","unstructured":"Zhang S, Liu A, Han C, Liang X, Xu X, Wang G. Multi-agent Reinforcement Learning-Based Orbital Edge Offloading in SAGIN Supporting Internet of Remote Things.\u00a0IEEE Internet Things J. https:\/\/doi.org\/10.1109\/JIOT.2023.3287737","DOI":"10.1109\/JIOT.2023.3287737"},{"issue":"2","key":"538_CR55","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1109\/TWC.2020.3029143","volume":"20","author":"C Zhou","year":"2020","unstructured":"Zhou C, Wu W, He H, Yang P, Lyu F, Cheng N, Shen X (2020) Deep reinforcement learning for delay-oriented IoT task scheduling in SAGIN. IEEE Trans Wirel Commun 20(2):911\u2013925","journal-title":"IEEE Trans Wirel Commun"},{"key":"538_CR56","doi-asserted-by":"publisher","unstructured":"Liu Y, Jiang L, Qi Q, Xie K, Xie S. Online Computation Offloading for Collaborative Space\/Aerial-Aided Edge Computing Toward 6G System.\u00a0IEEE Trans Veh Technol. https:\/\/doi.org\/10.1109\/TVT.2023.3312676","DOI":"10.1109\/TVT.2023.3312676"},{"issue":"3","key":"538_CR57","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1109\/JSTSP.2021.3135751","volume":"16","author":"H Liao","year":"2021","unstructured":"Liao H, Wang Z, Zhou Z, Wang Y, Zhang H, Mumtaz S, Guizani M (2021) Blockchain and semi-distributed learning-based secure and low-latency computation offloading in space-air-ground-integrated power IoT. IEEE J Sel Top Signal Process 16(3):381\u2013394","journal-title":"IEEE J Sel Top Signal Process"},{"issue":"5","key":"538_CR58","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1109\/MCOM.2018.1700888","volume":"56","author":"W Li","year":"2018","unstructured":"Li W, Yang T, Delicato FC, Pires PF, Tari Z, Khan SU, Zomaya AY (2018) On enabling sustainable edge computing with renewable energy resources. IEEE Commun Mag 56(5):94\u2013101","journal-title":"IEEE Commun Mag"},{"key":"538_CR59","doi-asserted-by":"crossref","unstructured":"Li S, Huang J (2017) Energy efficient resource management and task scheduling for IoT services in edge computing paradigm. In: 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA\/IUCC), IEEE, pp 846\u2013851","DOI":"10.1109\/ISPA\/IUCC.2017.00129"},{"key":"538_CR60","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-8176-4755-1","volume-title":"Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations,","author":"M Bardi","year":"1997","unstructured":"Bardi M, Dolcetta IC et al (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, vol 12. Springer"},{"key":"538_CR61","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 22447. MIT press, Cambridge"},{"key":"538_CR62","doi-asserted-by":"crossref","unstructured":"Zhang X, Wang G, Meng X, Wang S, Zhang Y, Rodriguez-Paton A, Wang J, Wang X (2022) Molormer: a lightweight self-attention-based method focused on spatial structure of molecular graph for drug\u2013drug interactions prediction. Brief Bioinform 23(5):bbac296","DOI":"10.1093\/bib\/bbac296"},{"issue":"7540","key":"538_CR63","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529\u2013533","journal-title":"Nature"},{"key":"538_CR64","unstructured":"Lin LJ (1992) Reinforcement learning for robots using neural networks. Carnegie Mellon University"},{"key":"538_CR65","unstructured":"Version SUM (2000) 4.2. 1 for pcs. Analytical Graphics, INC (AGI)"},{"key":"538_CR66","unstructured":"Rajan JA (2002) Highlights of GPS II-R Autonomous Navigation. Proceedings of the 58th Annual Meeting of The Institute of Navigation and CIGTF 21st Guidance Test Symposium (2002), Albuquerque, NM, pp. 354\u2013363"},{"key":"538_CR67","volume-title":"Mitigating the effect of weather on ka-band high-capacity satellites","author":"J Petranovich","year":"2012","unstructured":"Petranovich J (2012) Mitigating the effect of weather on ka-band high-capacity satellites. ViaSat Inc, Carlsbad"},{"issue":"3","key":"538_CR68","doi-asserted-by":"publisher","first-page":"1839","DOI":"10.1109\/COMST.2020.2990499","volume":"22","author":"N Saeed","year":"2020","unstructured":"Saeed N, Elzanaty A, Almorad H, Dahrouj H, Al-Naffouri TY, Alouini MS (2020) Cubesat communications: Recent advances and future challenges. IEEE Commun Surv Tutor 22(3):1839\u20131862","journal-title":"IEEE Commun Surv Tutor"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00538-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00538-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00538-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,18]],"date-time":"2023-11-18T03:16:45Z","timestamp":1700277405000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00538-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,18]]},"references-count":68,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["538"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00538-z","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3151728\/v1","asserted-by":"object"}]},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,18]]},"assertion":[{"value":"8 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 October 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We confirm that our research does not involves a survey asking real human participants to give opinions, or animals data to make.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"159"}}