{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T23:56:58Z","timestamp":1770335818931,"version":"3.49.0"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T00:00:00Z","timestamp":1615939200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T00:00:00Z","timestamp":1615939200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"VC Research","award":["VCR 0000057"],"award-info":[{"award-number":["VCR 0000057"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61802095, 61572162"],"award-info":[{"award-number":["No. 61802095, 61572162"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Zhejiang Provincial Key Science and Technology Project Foundation","award":["2018C01012"],"award-info":[{"award-number":["2018C01012"]}]},{"name":"Zhejiang Provincial National Science Foundation of China","award":["LQ19F020011"],"award-info":[{"award-number":["LQ19F020011"]}]},{"name":"Open Foundation of State Key Laboratory of Networking and Switching Technology","award":["SKLNST-2019-2-15"],"award-info":[{"award-number":["SKLNST-2019-2-15"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Wireless Com Network"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the development of the wireless network, increasing mobile applications are emerging and receiving great popularity. These applications cover a wide area, such as traffic monitoring, smart homes, real-time vision processing, objective tracking, and so on, and typically require computation-intensive resources to achieve a high quality of experience. Although the performance of mobile devices (MDs) has been continuously enhanced, running all the applications on a single MD still causes high energy consumption and latency. Fortunately, mobile edge computing (MEC) allows MDs to offload their computation-intensive tasks to proximal eNodeBs (eNBs) to augment computational capabilities. However, the current task offloading schemes mainly concentrate on average-based performance metrics, failing to meet the deadline constraint of the tasks. Based on the deep reinforcement learning (DRL) approach, this paper proposes an Energy-aware Task Offloading with Deadline constraint (DRL-E2D) algorithm for a multi-eNB MEC environment, which is to maximize the reward under the deadline constraint of the tasks. In terms of the actor-critic framework, we integrate the action representation into DRL-E2D to handle the large discrete action space problem, i.e., using the low-complexity k-nearest neighbor as an approximate approach to extract optimal discrete actions from the continuous action space. The extensive experimental results show that DRL-E2D achieves better performance than the comparison algorithms on all parameter settings, indicating that DRL-E2D is robust to the state changes in the MEC environment.<\/jats:p>","DOI":"10.1186\/s13638-021-01941-3","type":"journal-article","created":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T12:03:06Z","timestamp":1615982586000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Energy-aware task offloading with deadline constraint in mobile edge computing"],"prefix":"10.1186","volume":"2021","author":[{"given":"Zhongjin","family":"Li","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8012-5852","authenticated-orcid":false,"given":"Victor","family":"Chang","sequence":"additional","affiliation":[]},{"given":"Jidong","family":"Ge","sequence":"additional","affiliation":[]},{"given":"Linxuan","family":"Pan","sequence":"additional","affiliation":[]},{"given":"Haiyang","family":"Hu","sequence":"additional","affiliation":[]},{"given":"Binbin","family":"Huang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,17]]},"reference":[{"issue":"3","key":"1941_CR1","doi-asserted-by":"publisher","first-page":"2133","DOI":"10.1109\/COMST.2018.2828120","volume":"20","author":"F Jameel","year":"2018","unstructured":"F. Jameel, Z. Hamid, F. Jabeen, S. Zeadally, M.A. Javed, A survey of device-to-device communications: research issues and challenges. IEEE Commun. Surv. Tutor. 20(3), 2133\u20132168 (2018)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"1","key":"1941_CR2","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1109\/SURV.2013.062613.00160","volume":"16","author":"AR Khan","year":"2014","unstructured":"A.R. Khan, M. Othman, S.A. Madani, S.U. Khan, A survey of mobile cloud computing application models. IEEE Commun. Surv. Tutor. 16(1), 393\u2013413 (2014)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"3","key":"1941_CR3","doi-asserted-by":"publisher","first-page":"1628","DOI":"10.1109\/COMST.2017.2682318","volume":"19","author":"P Mach","year":"2017","unstructured":"P. Mach, Z. Becvar, Mobile edge computing: a survey on architecture and computation offloading. IEEE Commun. Surv. Tutor. 19(3), 1628\u20131656 (2017)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"6","key":"1941_CR4","doi-asserted-by":"publisher","first-page":"4187","DOI":"10.1109\/TII.2019.2936869","volume":"16","author":"X Xu","year":"2020","unstructured":"X. Xu, X. Zhang, H. Gao, Y. Xue, L. Qi, W. Dou, BeCome: blockchain-enabled computation offloading for IoT in mobile edge computing. IEEE Trans. Ind. Inform. 16(6), 4187\u20134195 (2020)","journal-title":"IEEE Trans. Ind. Inform."},{"key":"1941_CR5","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1186\/s13638-019-1526-x","volume":"19","author":"K Peng","year":"2019","unstructured":"K. Peng, M. Zhu, Y. Zhang, L. Liu, J. Zhang, V.C.M. Leung, L. Zheng, An energy- and cost-aware computation offloading method for workflow applications in mobile edge computing. EURASIP J. Wirel. Commun. Netw. 19, 207 (2019)","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"1941_CR6","unstructured":"Cisco Systems, Cisco visual networking index: global mobile data traffic forecast update. Technique Report (2019)"},{"issue":"3","key":"1941_CR7","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1109\/JSAC.2018.2815360","volume":"36","author":"M Chen","year":"2018","unstructured":"M. Chen, Y. Hao, Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J. Sel. Areas Commun. 36(3), 587\u2013597 (2018)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"8","key":"1941_CR8","first-page":"3571","volume":"65","author":"TQ Dinh","year":"2017","unstructured":"T.Q. Dinh, J. Tang, Q.D. La, T.Q.S. Quek, Offloading in mobile edge computing: task allocation and computational frequency scaling. IEEE Trans. Commun. 65(8), 3571\u20133584 (2017)","journal-title":"IEEE Trans. Commun."},{"issue":"1","key":"1941_CR9","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1186\/s13638-020-1652-5","volume":"2020","author":"X Huang","year":"2020","unstructured":"X. Huang, K. Xu, C. Lai, Q. Chen, J. Zhang, Energy-efficient offloading decision-making for mobile edge computing in vehicular networks. EURASIP J. Wirel. Commun. Netw. 2020(1), 35 (2020)","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"1941_CR10","doi-asserted-by":"crossref","unstructured":"T. Li, C. M. S. Magurawalage, K. Wang, K. Xu, K. Yang, H. Wang, in 37th IEEE International Conference on Distributed Computing Systems (ICDCS). On efficient offloading control in cloud radio access network with mobile edge computing (IEEE, 2017), pp. 2258\u20132263","DOI":"10.1109\/ICDCS.2017.24"},{"issue":"5","key":"1941_CR11","doi-asserted-by":"publisher","first-page":"2795","DOI":"10.1109\/TNET.2015.2487344","volume":"24","author":"X Chen","year":"2016","unstructured":"X. Chen, L. Jiao, W. Li, X. Fu, Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE\/ACM Trans. Netw. 24(5), 2795\u20132808 (2016)","journal-title":"IEEE\/ACM Trans. Netw."},{"issue":"7540","key":"1941_CR12","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"V. Mnih et al., Human-level control through deep reinforcement learning. Nature 518(7540), 529\u2013533 (2015)","journal-title":"Nature"},{"issue":"7587","key":"1941_CR13","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"D. Silver et al., Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484\u2013489 (2016)","journal-title":"Nature"},{"key":"1941_CR14","doi-asserted-by":"crossref","unstructured":"Z. Xu, J. Tang, J. Meng, W. Zhang, Y. Wang, C. H. Liu, D. Yang, in IEEE Conference on Computer Communications (INFOCOM). Experience-driven networking: A deep reinforcement learning based approach (IEEE, 2018), pp. 1871\u20131879","DOI":"10.1109\/INFOCOM.2018.8485853"},{"key":"1941_CR15","doi-asserted-by":"crossref","unstructured":"J. Li, Q. Liu, P. Wu, F. Shu, S. Jin, in IEEE\/CIC International Conference on Communications in China (ICCC). Task offloading for UAV-based mobile edge computing via deep reinforcement learning, (IEEE, 2018), pp. 798\u2013802","DOI":"10.1109\/ICCChina.2018.8641189"},{"key":"1941_CR16","unstructured":"D. V. Le, C. K. Tham, in IEEE Conference on Computer Communications Workshops (INFOCOM Workshops). A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds (IEEE, 2018), pp. 760\u2013765"},{"key":"1941_CR17","doi-asserted-by":"crossref","unstructured":"J. Li, H. Gao, T. Lv, Y. Lu, in IEEE Wireless Communications and Networking Conference (WCNC). Deep reinforcement learning based computation offloading and resource allocation for MEC (IEEE, 2018), pp. 1\u20136","DOI":"10.1109\/WCNC.2018.8377343"},{"key":"1941_CR18","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/j.future.2018.07.050","volume":"90","author":"MGR Alam","year":"2019","unstructured":"M.G.R. Alam, M.M. Hassan, M.Z. Uddin, A. Almogren, G. Fortino, Autonomic computation offloading in mobile edge for IoT applications. Future Gener. Comput. Syst. 90, 149\u2013157 (2019)","journal-title":"Future Gener. Comput. Syst."},{"key":"1941_CR19","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1016\/j.future.2019.07.019","volume":"102","author":"H Lu","year":"2020","unstructured":"H. Lu, C. Gu, F. Luo, W. Ding, X. Liu, Optimization of lightweight task offloading strategy for mobile edge computing based on deep reinforcement learning. Future Gener. Comput. Syst. 102, 847\u2013861 (2020)","journal-title":"Future Gener. Comput. Syst."},{"key":"1941_CR20","unstructured":"T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, in International Conference on Learning Representations (ICLR). Continuous control with deep reinforcement learning (2016)"},{"key":"1941_CR21","unstructured":"R. Beraldi, A. Mtibaa, H. M. Alnuweiri, in International Conference on Fog and Mobile Edge Computing (FMEC). Cooperative load balancing scheme for edge computing resources (2017), pp. 94\u2013100"},{"key":"1941_CR22","doi-asserted-by":"crossref","unstructured":"C. Wang, F. R. Yu, Q. Chen, L. Tang, in IEEE International Conference on Communications (ICC). Joint computation and radio resource management for cellular networks with mobile edge computing (IEEE, 2017), pp. 1\u20136","DOI":"10.1109\/ICC.2017.7996356"},{"key":"1941_CR23","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1016\/j.future.2019.09.035","volume":"102","author":"Y Miao","year":"2020","unstructured":"Y. Miao, G. Wu, M. Li, A. Ghoneim, M. Al-Rakhami, M.S. Hossain, Intelligent task prediction and computation offloading based on mobile-edge cloud computing. Future Gener. Comput. Syst. 102, 925\u2013931 (2020)","journal-title":"Future Gener. Comput. Syst."},{"issue":"1","key":"1941_CR24","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1109\/TWC.2019.2943563","volume":"19","author":"J Yan","year":"2020","unstructured":"J. Yan, S. Bi, Y.J. Zhang, M. Tao, Optimal task offloading and resource allocation in mobile-edge computing with inter-user task dependency. IEEE Trans. Wirel. Commun. 19(1), 235\u2013250 (2020)","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"1941_CR25","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1016\/j.comcom.2019.10.021","volume":"149","author":"R Wang","year":"2020","unstructured":"R. Wang, Y. Cao, A. Noor, T.A. Alamoudi, R. Nour, Agent-enabled task offloading in UAV-aided mobile edge computing. Comput. Commun. 149, 324\u2013331 (2020)","journal-title":"Comput. Commun."},{"key":"1941_CR26","unstructured":"C. F. Liu, M. Bennis, H. V. Poor, in Global Communications Conference Workshops (GLOBECOM Workshops). Latency and reliability-aware task offloading and resource allocation for mobile edge computing (2017), pp. 1\u20137"},{"issue":"6","key":"1941_CR27","doi-asserted-by":"publisher","first-page":"4132","DOI":"10.1109\/TCOMM.2019.2898573","volume":"67","author":"CF Liu","year":"2019","unstructured":"C.F. Liu, M. Bennis, M. Debbah, H.V. Poor, Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing. IEEE Trans. Commun. 67(6), 4132\u20134150 (2019)","journal-title":"IEEE Trans. Commun."},{"issue":"11","key":"1941_CR28","doi-asserted-by":"publisher","first-page":"2606","DOI":"10.1109\/JSAC.2017.2760186","volume":"35","author":"X Lyu","year":"2017","unstructured":"X. Lyu, W. Ni, H. Tian, R.P. Liu, X. Wang, G.B. Giannakis, A. Paulraj, Optimal schedule of mobile edge computing for internet of things using partial information. IEEE J. Sel. Areas Commun. 35(11), 2606\u20132615 (2017)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"11","key":"1941_CR29","doi-asserted-by":"publisher","first-page":"2637","DOI":"10.1109\/JSAC.2017.2760160","volume":"35","author":"Y Sun","year":"2017","unstructured":"Y. Sun, S. Zhou, J. Xu, EMM: energy-aware mobility management for mobile edge computing in ultra dense networks. IEEE J. Sel. Areas Commun. 35(11), 2637\u20132646 (2017)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"1941_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jnca.2019.01.020","volume":"131","author":"C Li","year":"2019","unstructured":"C. Li, J. Tang, Y. Luo, Dynamic multi-user computation offloading for wireless powered mobile edge computing. J. Netw. Comput. Appl. 131, 1\u201315 (2019)","journal-title":"J. Netw. Comput. Appl."},{"key":"1941_CR31","doi-asserted-by":"publisher","first-page":"2581","DOI":"10.1109\/TMC.2019.2928811","volume":"19","author":"L Huang","year":"2019","unstructured":"L. Huang, S. Bi, Y.J. Zhang, Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans. Mob. Comput. 19, 2581\u20132593 (2019)","journal-title":"IEEE Trans. Mob. Comput."},{"key":"1941_CR32","doi-asserted-by":"crossref","unstructured":"L. Lei, H. Xu, X. Xiong, K. Zheng, W. Xiang, X. Wang, Multi-user resource control with deep reinforcement learning in IoT edge computing. arXiv:1906.07860 (2019)","DOI":"10.1109\/JIOT.2019.2935543"},{"key":"1941_CR33","doi-asserted-by":"crossref","unstructured":"B. Huang, Y. Li, Z. Li, L. Pan, S. Wang, Y. Xu, H. Hu, Security and cost-aware computation offloading via deep reinforcement learning in mobile edge computing. Wirel. Commun. Mob. Comput. 2019, 3816237:1\u20133816237:20 (2019)","DOI":"10.1155\/2019\/3816237"},{"key":"1941_CR34","unstructured":"G. Dulac-Arnold, R. Evans, H. van Hasselt, P. Sunehag, T. Lillicrap, J. Hunt, T. Mann, T. Weber, T. Degris, B. Coppin. Deep reinforcement learning in large discrete action spaces. arXiv:1512.07679 (2015)"},{"issue":"3","key":"1941_CR35","doi-asserted-by":"publisher","first-page":"1784","DOI":"10.1109\/TWC.2017.2785305","volume":"17","author":"F Wang","year":"2018","unstructured":"F. Wang, J. Xu, X. Wang, S. Cui, Joint offloading and computing optimization in wireless powered mobile-edge computing systems. IEEE Trans. Wirel. Commun. 17(3), 1784\u20131797 (2018)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"1","key":"1941_CR36","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1109\/TPDS.2012.341","volume":"25","author":"Y Yao","year":"2014","unstructured":"Y. Yao, L. Huang, A.B. Sharma, L. Golubchik, M.J. Neely, Power cost reduction in distributed data centers: a two-time-scale approach for delay tolerant workloads. IEEE Trans. Parallel Distrib. Syst. 25(1), 200\u2013211 (2014)","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"1941_CR37","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","unstructured":"R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT press, Cambridge, 2018)."},{"issue":"4","key":"1941_CR38","doi-asserted-by":"publisher","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","volume":"21","author":"NC Luong","year":"2019","unstructured":"N.C. Luong, D.T. Hoang, S. Gong, D. Niyato, P. Wang, Y.C. Liang, D.I. Kim, Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21(4), 3133\u20133174 (2019)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"3","key":"1941_CR39","doi-asserted-by":"publisher","first-page":"1610","DOI":"10.1109\/TWC.2019.2894403","volume":"18","author":"W Jiang","year":"2019","unstructured":"W. Jiang, G. Feng, S. Qin, T.S. Peter Yum, G. Cao, Multi-agent reinforcement learning for efficient content caching in mobile d2d networks. IEEE Trans. Wirel. Commun. 18(3), 1610\u20131622 (2019)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"9","key":"1941_CR40","doi-asserted-by":"publisher","first-page":"5994","DOI":"10.1109\/TWC.2017.2717986","volume":"16","author":"Y Mao","year":"2017","unstructured":"Y. Mao, J. Zhang, S.H. Song, K.B. Letaief, Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems. IEEE Trans. Wirel. Commun. 16(9), 5994\u20136009 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"11","key":"1941_CR41","doi-asserted-by":"publisher","first-page":"2227","DOI":"10.1109\/TPAMI.2014.2321376","volume":"36","author":"M Muja","year":"2014","unstructured":"M. Muja, D.G. Lowe, Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2227\u20132240 (2014)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["EURASIP Journal on Wireless Communications and Networking"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-021-01941-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s13638-021-01941-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-021-01941-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T12:08:48Z","timestamp":1615982928000},"score":1,"resource":{"primary":{"URL":"https:\/\/jwcn-eurasipjournals.springeropen.com\/articles\/10.1186\/s13638-021-01941-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,17]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1941"],"URL":"https:\/\/doi.org\/10.1186\/s13638-021-01941-3","relation":{},"ISSN":["1687-1499"],"issn-type":[{"value":"1687-1499","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,17]]},"assertion":[{"value":"27 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"56"}}