{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T16:28:55Z","timestamp":1784996935424,"version":"3.55.0"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:00:00Z","timestamp":1712534400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:00:00Z","timestamp":1712534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003392","name":"Natural Science Foundation of Fujian Province","doi-asserted-by":"publisher","award":["2023J01967"],"award-info":[{"award-number":["2023J01967"]}],"id":[{"id":"10.13039\/501100003392","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In this study, we investigate the challenges associated with dynamic time slot server selection in mobile edge computing (MEC) systems. This study considers the fluctuating nature of user access at edge servers and the various factors that influence server workload, including offloading policies, offloading ratios, users\u2019 transmission power, and the servers\u2019 reserved capacity. To streamline the process of selecting edge servers with an eye on long-term optimization, we cast the problem as a Markov Decision Process (MDP) and propose a Deep Reinforcement Learning (DRL)-based algorithm as a solution. Our approach involves learning the selection strategy by analyzing the performance of server selections in previous iterations. Simulation outcomes show that our DRL-based algorithm surpasses benchmarks, delivering minimal average latency.<\/jats:p>","DOI":"10.1186\/s13634-024-01142-2","type":"journal-article","created":{"date-parts":[[2024,4,7]],"date-time":"2024-04-07T23:10:11Z","timestamp":1712531411000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["A deep reinforcement approach for computation offloading in MEC dynamic networks"],"prefix":"10.1186","volume":"2024","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9803-2067","authenticated-orcid":false,"given":"Yibiao","family":"Fan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaowei","family":"Cai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,8]]},"reference":[{"issue":"6","key":"1142_CR1","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1016\/j.pmcj.2009.07.007","volume":"5","author":"J Liu","year":"2009","unstructured":"J. Liu, L. Zhong, J. Wickramasuriya, V. Vasudevan, uWave: Accelerometer-based personalized gesture recognition and its applications. Pervasive Mob. Comput. 5(6), 657\u2013675 (2009)","journal-title":"Pervasive Mob. Comput."},{"issue":"3","key":"1142_CR2","doi-asserted-by":"publisher","first-page":"398","DOI":"10.1109\/LWC.2017.2696539","volume":"6","author":"A Al-Shuwaili","year":"2017","unstructured":"A. Al-Shuwaili, O. Simeone, Energy-efficient resource allocation for mobile edge computing-based augmented reality applications. IEEE Wirel. Commun. Lett. 6(3), 398\u2013401 (2017)","journal-title":"IEEE Wirel. Commun. Lett."},{"issue":"5","key":"1142_CR3","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1109\/JIOT.2016.2579198","volume":"3","author":"W Shi","year":"2016","unstructured":"W. Shi, J. Cao, Q. Zhang, Y. Li, L. Xu, Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637\u2013646 (2016)","journal-title":"IEEE Internet Things J."},{"key":"1142_CR4","doi-asserted-by":"crossref","unstructured":"V. Farhadi et al., Service placement and request scheduling for data-intensive applications in edge clouds. In: Presented at the IEEE INFOCOM 2019\u2014IEEE Conference on Computer Communications (2019)","DOI":"10.1109\/INFOCOM.2019.8737368"},{"issue":"2","key":"1142_CR5","doi-asserted-by":"publisher","first-page":"1334","DOI":"10.1109\/JIOT.2018.2811808","volume":"5","author":"L Zhao","year":"2018","unstructured":"L. Zhao, W. Sun, Y. Shi, J. Liu, Optimal placement of cloudlets for access delay minimization in SDN-based internet of things networks. IEEE Internet Things J. 5(2), 1334\u20131344 (2018)","journal-title":"IEEE Internet Things J."},{"key":"1142_CR6","doi-asserted-by":"crossref","unstructured":"B. Shen, X. Xu, F. Dar, L. Qi, X. Zhang, W. Dou, Dynamic task offloading with minority game for internet of vehicles in cloud-edge computing. In: Presented at the 2020 IEEE International Conference on Web Services (ICWS) (2020)","DOI":"10.1109\/ICWS49710.2020.00055"},{"issue":"4","key":"1142_CR7","doi-asserted-by":"publisher","first-page":"651","DOI":"10.1007\/s10723-019-09492-0","volume":"17","author":"H Baraki","year":"2019","unstructured":"H. Baraki, A. Jahl, S. Jakob, C. Schwarzbach, M. Fax, K. Geihs, Optimizing applications for mobile cloud computing through MOCCAA. J. Grid Comput. 17(4), 651\u2013676 (2019)","journal-title":"J. Grid Comput."},{"issue":"11","key":"1142_CR8","first-page":"1","volume":"11","author":"YC Hu","year":"2015","unstructured":"Y.C. Hu, M. Patel, D. Sabella, N. Sprecher, V. Young, Mobile edge computing\u2014A key technology towards 5G. ETSI White Paper 11(11), 1\u201316 (2015)","journal-title":"ETSI White Paper"},{"issue":"4","key":"1142_CR9","doi-asserted-by":"publisher","first-page":"2322","DOI":"10.1109\/COMST.2017.2745201","volume":"19","author":"Y Mao","year":"2017","unstructured":"Y. Mao, C. You, J. Zhang, K. Huang, K.B. Letaief, A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tutor. 19(4), 2322\u20132358 (2017)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"3","key":"1142_CR10","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1109\/MNET.011.2000587","volume":"35","author":"Y Zhang","year":"2021","unstructured":"Y. Zhang et al., Edge intelligence for plug-in electrical vehicle charging service. IEEE Netw. 35(3), 81\u201387 (2021)","journal-title":"IEEE Netw."},{"key":"1142_CR11","doi-asserted-by":"crossref","unstructured":"Varadharajan V, Mantri S, Shah B, et al. Emerging edge computing applications. In: IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) (2022). p. 1\u20134","DOI":"10.1109\/CONECCT55679.2022.9865769"},{"key":"1142_CR12","doi-asserted-by":"crossref","unstructured":"Z. Yang, S. Bi, Y.-J.A. Zhang, Dynamic trajectory and offloading control of UAV-enabled MEC under user mobility. In: Presented at the 2021 IEEE International Conference on Communications Workshops (ICC Workshops) (2021).","DOI":"10.1109\/ICCWorkshops50388.2021.9473504"},{"key":"1142_CR13","doi-asserted-by":"publisher","first-page":"121456","DOI":"10.1109\/ACCESS.2021.3109132","volume":"9","author":"S Li","year":"2021","unstructured":"S. Li, X. Hu, Y. Du, Deep reinforcement learning and game theory for computation offloading in dynamic edge computing markets. IEEE Access 9, 121456\u2013121466 (2021)","journal-title":"IEEE Access"},{"key":"1142_CR14","doi-asserted-by":"publisher","first-page":"53708","DOI":"10.1109\/ACCESS.2021.3070908","volume":"9","author":"L Zhang","year":"2021","unstructured":"L. Zhang et al., Task offloading and trajectory control for UAV-assisted mobile edge computing using deep reinforcement learning. IEEE Access 9, 53708\u201353719 (2021)","journal-title":"IEEE Access"},{"issue":"10","key":"1142_CR15","doi-asserted-by":"publisher","first-page":"4228","DOI":"10.1109\/TCYB.2019.2916728","volume":"50","author":"PQ Huang","year":"2020","unstructured":"P.Q. Huang, Y. Wang, K. Wang, Z.Z. Liu, A bilevel optimization approach for joint offloading decision and resource allocation in cooperative mobile edge computing. IEEE Trans Cybern 50(10), 4228\u20134241 (2020)","journal-title":"IEEE Trans Cybern"},{"issue":"18","key":"1142_CR16","doi-asserted-by":"publisher","first-page":"17280","DOI":"10.1109\/JIOT.2022.3152139","volume":"9","author":"T Nisha","year":"2022","unstructured":"T. Nisha, D.T. Nguyen, V.K. Bhargava, A bilevel programming framework for joint edge resource management and pricing. IEEE Internet Things J. 9(18), 17280\u201317291 (2022)","journal-title":"IEEE Internet Things J."},{"issue":"4","key":"1142_CR17","doi-asserted-by":"publisher","first-page":"4225","DOI":"10.1109\/TVT.2022.3141799","volume":"71","author":"Y Liu","year":"2022","unstructured":"Y. Liu, J. Yan, X. Zhao, Deep reinforcement learning based latency minimization for mobile edge computing with virtualization in maritime UAV communication network. IEEE Trans. Veh. Technol. 71(4), 4225\u20134236 (2022)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"11","key":"1142_CR18","doi-asserted-by":"publisher","first-page":"4593","DOI":"10.1109\/TFUZZ.2022.3158000","volume":"30","author":"X Xu","year":"2022","unstructured":"X. Xu et al., Game theory for distributed IoV task offloading with fuzzy neural network in edge computing. IEEE Trans. Fuzzy Syst. 30(11), 4593\u20134604 (2022)","journal-title":"IEEE Trans. Fuzzy Syst."},{"issue":"3","key":"1142_CR19","doi-asserted-by":"publisher","first-page":"1248","DOI":"10.1109\/TGCN.2021.3073939","volume":"5","author":"R Zhao","year":"2021","unstructured":"R. Zhao, J. Xia, Z. Zhao, S. Lai, L. Fan, D. Li, Green MEC networks design under UAV attack: a deep reinforcement learning approach. IEEE Trans. Green Commun. Netw. 5(3), 1248\u20131258 (2021)","journal-title":"IEEE Trans. Green Commun. Netw."},{"issue":"6","key":"1142_CR20","doi-asserted-by":"publisher","first-page":"4177","DOI":"10.1109\/TWC.2018.2821664","volume":"17","author":"S Bi","year":"2018","unstructured":"S. Bi, Y.J. Zhang, Computation rate maximization for wireless powered mobile-edge computing with binary computation offloading. IEEE Trans. Wirel. Commun. 17(6), 4177\u20134190 (2018)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"3","key":"1142_CR21","doi-asserted-by":"publisher","first-page":"1784","DOI":"10.1109\/TWC.2017.2785305","volume":"17","author":"F Wang","year":"2018","unstructured":"F. Wang, J. Xu, X. Wang, S. Cui, Joint offloading and computing optimization in wireless powered mobile-edge computing systems. IEEE Trans. Wirel. Commun. 17(3), 1784\u20131797 (2018)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"5","key":"1142_CR22","doi-asserted-by":"publisher","first-page":"1757","DOI":"10.1109\/JSAC.2016.2545382","volume":"34","author":"C You","year":"2016","unstructured":"C. You, K. Huang, H. Chae, Energy efficient mobile cloud computing powered by wireless energy transfer. IEEE J. Sel. Areas Commun. 34(5), 1757\u20131771 (2016)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"9","key":"1142_CR23","doi-asserted-by":"publisher","first-page":"4569","DOI":"10.1109\/TWC.2013.072513.121842","volume":"12","author":"W Zhang","year":"2013","unstructured":"W. Zhang, Y. Wen, K. Guan, D. Kilper, H. Luo, D.O. Wu, Energy-optimal mobile cloud computing under stochastic wireless channel. IEEE Trans. Wirel. Commun. 12(9), 4569\u20134581 (2013)","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"1142_CR24","doi-asserted-by":"crossref","unstructured":"M.-H. Chen, B. Liang, M. Dong, Joint offloading decision and resource allocation for multi-user multi-task mobile cloud. In: 2016 IEEE International Conference on Communications (ICC) (2016), p. 1\u20136","DOI":"10.1109\/ICC.2016.7510999"},{"issue":"8","key":"1142_CR25","first-page":"3571","volume":"65","author":"TQ Thinh","year":"2017","unstructured":"T.Q. Thinh, J. Tang, Q.D. La, T.Q.S. Quek, Offloading in mobile edge computing: task allocation and computational frequency scaling. IEEE Trans. Commun. 65(8), 3571\u20133584 (2017)","journal-title":"IEEE Trans. Commun."},{"issue":"3","key":"1142_CR26","doi-asserted-by":"publisher","first-page":"1397","DOI":"10.1109\/TWC.2016.2633522","volume":"16","author":"C You","year":"2017","unstructured":"C. You, K. Huang, H. Chae, B.-H. Kim, Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Trans. Wirel. Commun. 16(3), 1397\u20131411 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"10","key":"1142_CR27","first-page":"4268","volume":"64","author":"Y Wang","year":"2016","unstructured":"Y. Wang, M. Sheng, X. Wang, L. Wang, J. Li, Mobile-edge computing: partial computation offloading using dynamic voltage scaling. IEEE Trans. Commun. 64(10), 4268\u20134282 (2016)","journal-title":"IEEE Trans. Commun."},{"issue":"7","key":"1142_CR28","doi-asserted-by":"publisher","first-page":"3870","DOI":"10.1109\/TMC.2022.3148254","volume":"22","author":"T Liu","year":"2023","unstructured":"T. Liu, S. Ni, X. Li, Y. Zhu, L. Kong, Y. Yang, Deep reinforcement learning based approach for online service placement and computation resource allocation in edge computing. IEEE Trans. Mob. Comput. 22(7), 3870\u20133881 (2023)","journal-title":"IEEE Trans. Mob. Comput."},{"issue":"8","key":"1142_CR29","doi-asserted-by":"publisher","first-page":"8373","DOI":"10.1109\/TVT.2021.3094584","volume":"70","author":"S Joo","year":"2021","unstructured":"S. Joo, H. Kang, J. Kang, CoSMoS: Cooperative sky-ground mobile edge computing system. IEEE Trans. Veh. Technol. 70(8), 8373\u20138377 (2021)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"6","key":"1142_CR30","doi-asserted-by":"publisher","first-page":"4193","DOI":"10.1109\/TCOMM.2019.2903088","volume":"67","author":"H Xing","year":"2019","unstructured":"H. Xing, L. Liu, J. Xu, A. Nallanathan, Joint task assignment and resource allocation for D2D-enabled mobile-edge computing. IEEE Trans. Commun. 67(6), 4193\u20134207 (2019)","journal-title":"IEEE Trans. Commun."},{"issue":"3","key":"1142_CR31","doi-asserted-by":"publisher","first-page":"2422","DOI":"10.1109\/TITS.2021.3114295","volume":"23","author":"X Zhu","year":"2022","unstructured":"X. Zhu, Y. Luo, A. Liu, N.N. Xiong, M. Dong, S. Zhang, A deep reinforcement learning-based resource management game in vehicular edge computing. IEEE Trans. Intell. Transp. Syst. 23(3), 2422\u20132433 (2022)","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"1142_CR32","doi-asserted-by":"crossref","unstructured":"L. He, J. Zhao, X. Sun, D. Zhang, Dynamic task offloading for mobile edge computing in urban rail transit. In Presented at the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) (2021)","DOI":"10.1109\/WCSP52459.2021.9613269"},{"issue":"1","key":"1142_CR33","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1109\/TGCN.2021.3122911","volume":"6","author":"N Irtija","year":"2022","unstructured":"N. Irtija, I. Anagnostopoulos, G. Zervakis, E.E. Tsiropoulou, H. Amrouch, J. Henkel, Energy efficient edge computing enabled by satisfaction games and approximate computing. IEEE Trans. Green Commun. Netw. 6(1), 281\u2013294 (2022)","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"1142_CR34","doi-asserted-by":"crossref","unstructured":"Y. Zou, F. Shen, F. Yan, L. Tang, Task-oriented resource allocation for mobile edge computing with multi-agent reinforcement learning. In Presented at the 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall) (2021)","DOI":"10.1109\/VTC2021-Fall52928.2021.9625217"},{"issue":"6","key":"1142_CR35","doi-asserted-by":"publisher","first-page":"3691","DOI":"10.1109\/TMC.2021.3139362","volume":"22","author":"Q Li","year":"2023","unstructured":"Q. Li, X. Ma, A. Zhou, X. Luo, F. Yang, S. Wang, User-oriented edge node grouping in mobile edge computing. IEEE Trans. Mob. Comput. 22(6), 3691\u20133705 (2023)","journal-title":"IEEE Trans. Mob. Comput."},{"key":"1142_CR36","doi-asserted-by":"crossref","unstructured":"Y. Zhang, X. Dong, Y. Zhao, Decentralized computation offloading over wireless-powered mobile-edge computing networks. In: IEEE International Conference on Artificial Intelligence and Information Systems (ICAIIS) (2020). pp. 137\u2013140","DOI":"10.1109\/ICAIIS49377.2020.9194840"},{"key":"1142_CR37","doi-asserted-by":"crossref","unstructured":"P. Zhou, B. Yang, C. Chen: Joint computation offloading and resource allocation for NOMA-enabled industrial internet of things. In: 39th Chinese Control Conference (CCC) (2020). pp. 5241\u20135246","DOI":"10.23919\/CCC50068.2020.9188893"},{"issue":"3","key":"1142_CR38","doi-asserted-by":"publisher","first-page":"1548","DOI":"10.1109\/TCOMM.2020.3044085","volume":"69","author":"Z Song","year":"2021","unstructured":"Z. Song, Y. Liu, X. Sun, Joint task offloading and resource allocation for NOMA-enabled multi-access mobile edge computing. IEEE Trans. Commun. 69(3), 1548\u20131564 (2021)","journal-title":"IEEE Trans. Commun."},{"key":"1142_CR39","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2021.108256","volume":"196","author":"Z Wan","year":"2021","unstructured":"Z. Wan, D. Xu, D. Xu et al., Joint computation offloading and resource allocation for NOMA-based multi-access mobile edge computing systems. Comput. Netw. 196, 108256 (2021)","journal-title":"Comput. Netw."},{"issue":"4","key":"1142_CR40","doi-asserted-by":"publisher","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","volume":"21","author":"NC Luong","year":"2019","unstructured":"N.C. Luong, D.T. Hoang, S. Gong, D. Niyato, I.K. Dong, Applications of deep reinforcement learning in communications and networking: A survey. IEEE Commun. Surv. Tuts 21(4), 3133\u20133174 (2019)","journal-title":"IEEE Commun. Surv. Tuts"},{"issue":"3","key":"1142_CR41","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1109\/JSAC.2018.2815360","volume":"36","author":"M Chen","year":"2018","unstructured":"M. Chen, Y. Hao, Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J. Sel. Areas Commun. 36(3), 587\u2013597 (2018)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"2","key":"1142_CR42","doi-asserted-by":"publisher","first-page":"1517","DOI":"10.1109\/JIOT.2021.3091142","volume":"9","author":"H Zhou","year":"2022","unstructured":"H. Zhou, K. Jiang, X. Liu, X. Li, V.C.M. Leung, Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing. IEEE Internet Things J. 9(2), 1517\u20131530 (2022)","journal-title":"IEEE Internet Things J."}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-024-01142-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-024-01142-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-024-01142-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,7]],"date-time":"2024-04-07T23:12:31Z","timestamp":1712531551000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-024-01142-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,8]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1142"],"URL":"https:\/\/doi.org\/10.1186\/s13634-024-01142-2","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3651954\/v1","asserted-by":"object"}]},"ISSN":["1687-6180"],"issn-type":[{"value":"1687-6180","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,8]]},"assertion":[{"value":"7 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 March 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"48"}}