{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T20:13:07Z","timestamp":1783973587113,"version":"3.55.0"},"reference-count":34,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2021,9,29]],"date-time":"2021-09-29T00:00:00Z","timestamp":1632873600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11461038"],"award-info":[{"award-number":["11461038"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Innovation Foundation of Colleges and Universities in Gansu Province","award":["2020A-033"],"award-info":[{"award-number":["2020A-033"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Computation offloading technology extends cloud computing to the edge of the access network close to users, bringing many benefits to terminal devices with limited battery and computational resources. Nevertheless, the existing computation offloading approaches are challenging to apply to specific scenarios, such as the dense distribution of end-users and the sparse distribution of network infrastructure. The technological revolution in the unmanned aerial vehicle (UAV) and chip industry has granted UAVs more computing resources and promoted the emergence of UAV-assisted mobile edge computing (MEC) technology, which could be applied to those scenarios. However, in the MEC system with multiple users and multiple servers, making reasonable offloading decisions and allocating system resources is still a severe challenge. This paper studies the offloading decision and resource allocation problem in the UAV-assisted MEC environment with multiple users and servers. To ensure the quality of service for end-users, we set the weighted total cost of delay, energy consumption, and the size of discarded tasks as our optimization objective. We further formulate the joint optimization problem as a Markov decision process and apply the soft actor\u2013critic (SAC) deep reinforcement learning algorithm to optimize the offloading policy. Numerical simulation results show that the offloading policy optimized by our proposed SAC-based dynamic computing offloading (SACDCO) algorithm effectively reduces the delay, energy consumption, and size of discarded tasks for the UAV-assisted MEC system. Compared with the fixed local-UAV scheme in the specific simulation setting, our proposed approach reduces system delay and energy consumption by approximately 50% and 200%, respectively.<\/jats:p>","DOI":"10.3390\/s21196499","type":"journal-article","created":{"date-parts":[[2021,10,8]],"date-time":"2021-10-08T21:26:20Z","timestamp":1633728380000},"page":"6499","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Deep Reinforcement Learning for Computation Offloading and Resource Allocation in Unmanned-Aerial-Vehicle Assisted Edge Computing"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5365-4137","authenticated-orcid":false,"given":"Shuyang","family":"Li","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6960-2988","authenticated-orcid":false,"given":"Xiaohui","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1808-8904","authenticated-orcid":false,"given":"Yongwen","family":"Du","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102781","DOI":"10.1016\/j.jnca.2020.102781","article-title":"A survey on computation offloading modeling for edge computing","volume":"169","author":"Lin","year":"2020","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Alghamdi, I., Anagnostopoulos, C., and Pezaros, D.P. (2019). Delay-tolerant sequential decision making for task offloading in mobile edge computing environments. Information, 10.","DOI":"10.3390\/info10100312"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, K., Mao, Y., Leng, S., Maharjan, S., and Zhang, Y. (2017, January 21\u201325). Optimal delay constrained offloading for vehicular edge computing networks. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997360"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1109\/JSTSP.2019.2893057","article-title":"Delay-minimization nonorthogonal multiple access enabled multi-user mobile edge computation offloading","volume":"13","author":"Wu","year":"2019","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7590","DOI":"10.1109\/TWC.2018.2868710","article-title":"Asynchronous mobile-edge computation offloading: Energy-efficient resource management","volume":"17","author":"You","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1109\/LCOMM.2018.2882846","article-title":"Energy-efficient NOMA-based mobile edge computing offloading","volume":"23","author":"Pan","year":"2018","journal-title":"IEEE Commun. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.jnca.2019.02.008","article-title":"An energy-aware computation offloading method for smart edge computing in wireless metropolitan area networks","volume":"133","author":"Xu","year":"2019","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4642","DOI":"10.1109\/TII.2018.2843365","article-title":"Energy-delay tradeoff for dynamic offloading in mobile-edge computing system with energy harvesting devices","volume":"14","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4216","DOI":"10.1109\/TII.2019.2897001","article-title":"MASM: A multiple-algorithm service model for energy-delay optimization in edge artificial intelligence","volume":"15","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Vu, T.T., Van Huynh, N., Hoang, D.T., Nguyen, D.N., and Dutkiewicz, E. (2018, January 9\u201313). Offloading energy efficiency with delay constraint for cooperative mobile edge computing networks. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/GLOCOM.2018.8647856"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1109\/TAC.2009.2013652","article-title":"Intelligent packet dropping for optimal energy-delay tradeoffs in wireless downlinks","volume":"54","author":"Neely","year":"2009","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1605","DOI":"10.1109\/TNET.2018.2844284","article-title":"A new backpressure algorithm for joint rate control and routing with vanishing utility optimality gaps and finite queue lengths","volume":"26","author":"Yu","year":"2018","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1109\/TNET.2007.905154","article-title":"Delay and capacity trade-offs in mobile ad hoc networks: A global perspective","volume":"15","author":"Sharma","year":"2007","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_14","first-page":"815","article-title":"Near optimal power and rate control of multi-hop sensor networks with energy replenishment: Basic limitations with finite energy and data storage","volume":"57","author":"Mao","year":"2011","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4983","DOI":"10.1109\/TCOMM.2016.2611512","article-title":"Throughput maximization for UAV-enabled mobile relaying systems","volume":"64","author":"Zeng","year":"2016","journal-title":"IEEE Trans. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/TWC.2017.2789293","article-title":"Joint trajectory and communication design for multi-UAV enabled wireless networks","volume":"17","author":"Wu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1879","DOI":"10.1109\/JIOT.2018.2878876","article-title":"Joint offloading and trajectory design for UAV-enabled mobile edge computing systems","volume":"6","author":"Hu","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1109\/TVT.2017.2706308","article-title":"Mobile edge computing via a UAV-mounted cloudlet: Optimization of bit allocation and path planning","volume":"67","author":"Jeong","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1927","DOI":"10.1109\/JSAC.2018.2864426","article-title":"Computation rate maximization in UAV-enabled wireless-powered mobile-edge computing systems","volume":"36","author":"Zhou","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1109\/LCOMM.2019.2891662","article-title":"Task offloading in UAV-aided edge computing: Bit allocation and trajectory optimization","volume":"23","author":"Xiong","year":"2019","journal-title":"IEEE Commun. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"107496","DOI":"10.1016\/j.comnet.2020.107496","article-title":"A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective","volume":"182","author":"Shakarami","year":"2020","journal-title":"Comput. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.dcan.2018.10.003","article-title":"Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing","volume":"5","author":"Huang","year":"2019","journal-title":"Digit. Commun. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6898","DOI":"10.1109\/JIOT.2020.2971645","article-title":"Multi-UAV-enabled load-balance mobile-edge computing for IoT networks","volume":"7","author":"Yang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"12229","DOI":"10.1109\/TVT.2020.3016840","article-title":"Cooperative offloading and resource management for UAV-enabled mobile edge computing in power IoT system","volume":"69","author":"Liu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"928","DOI":"10.1109\/TVT.2020.3048938","article-title":"Learning-based computation offloading approaches in UAVs-assisted edge computing","volume":"70","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"8753","DOI":"10.1109\/JIOT.2019.2923702","article-title":"Hierarchical game-theoretic and reinforcement learning framework for computational offloading in UAV-enabled mobile edge computing networks with multiple service providers","volume":"6","author":"Asheralieva","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MNET.2013.6616110","article-title":"Follow me cloud: Interworking federated clouds and distributed mobile networks","volume":"27","author":"Taleb","year":"2013","journal-title":"IEEE Netw."},{"key":"ref_28","first-page":"4268","article-title":"Mobile-edge computing: Partial computation offloading using dynamic voltage scaling","volume":"64","author":"Wang","year":"2016","journal-title":"IEEE Trans. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Introduction to Reinforcement Learning, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mohammed, A., Nahom, H., Tewodros, A., Habtamu, Y., and Hayelom, G. (2020, January 18\u201320). Deep reinforcement learning for computation offloading and resource allocation in blockchain-based multi-UAV-enabled mobile edge computing. Proceedings of the 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China.","DOI":"10.1109\/ICCWAMTIP51612.2020.9317445"},{"key":"ref_31","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft actor-critic algorithms and applications. arXiv."},{"key":"ref_32","unstructured":"Shenzhen DJI Innovation Technology Co., Ltd. (2021, August 27). Technical parameters of DJI Air 2S. Available online: https:\/\/www.dji.com\/air-2s\/specs."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/LWC.2018.2872547","article-title":"Deployment of unmanned aerial vehicle base stations for optimal quality of coverage","volume":"8","author":"Savkin","year":"2018","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"10403","DOI":"10.1109\/ACCESS.2020.2965162","article-title":"Joint position and resource optimization for multi-UAV-aided relaying systems","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6499\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:07:13Z","timestamp":1760166433000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6499"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,29]]},"references-count":34,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["s21196499"],"URL":"https:\/\/doi.org\/10.3390\/s21196499","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,29]]}}}