{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T05:02:55Z","timestamp":1762059775391,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T00:00:00Z","timestamp":1658793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Development Project of Jilin Province of China","award":["20200401076GX"],"award-info":[{"award-number":["20200401076GX"]}]},{"name":"Jilin University of China","award":["20200401076GX"],"award-info":[{"award-number":["20200401076GX"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Unmanned aerial vehicles (UAVs) have the characteristics of portability, safety, and strong adaptability. In the case of a maritime disaster, they can be used for personnel search and rescue, real-time monitoring, and disaster assessment. However, the power, computing power, and other resources of UAVs are often limited. Therefore, this paper combines a UAV and mobile edge computing (MEC), and designs a deep reinforcement learning-based online task offloading (DOTO) algorithm. The algorithm can obtain an online offloading strategy that maximizes the residual energy of the UAV by jointly optimizing the UAV\u2019s time and communication resources. The DOTO algorithm adopts time division multiple access (TDMA) to offload and schedule the UAV computing task, integrates wireless power transfer (WPT) to supply power to the UAV, calculates the residual energy corresponding to the offloading action through the convex optimization method, and uses an adaptive K method to reduce the computational complexity of the algorithm. The simulation results show that the DOTO algorithm proposed in this paper for the energy-saving goal of maximizing the residual energy of UAVs in MEC can provide the UAV with an online task offloading strategy that is superior to other traditional benchmark schemes. In particular, when an individual UAV exits the system due to insufficient power or failure, or a new UAV is connected to the system, it can perform timely and automatic adjustment without manual participation, and has good stability and adaptability.<\/jats:p>","DOI":"10.3390\/fi14080226","type":"journal-article","created":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T22:03:42Z","timestamp":1658873022000},"page":"226","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Energy Saving Strategy of UAV in MEC Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"14","author":[{"given":"Zhiqiang","family":"Dai","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gaochao","family":"Xu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziqi","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiaqi","family":"Ge","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9114113","DOI":"10.1155\/2021\/9114113","article-title":"IoT Workflow Scheduling Using Intelligent Arithmetic Optimization Algorithm in Fog Computing","volume":"2021","author":"Abualigah","year":"2021","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"17817","DOI":"10.1109\/JIOT.2021.3081556","article-title":"IoT-Enabled Social Relationships Meet Artificial Social Intelligence","volume":"8","author":"Dhelim","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"114937","DOI":"10.1016\/j.eswa.2021.114937","article-title":"Search and Rescue Operation Using UAVs: A Case Study","volume":"178","author":"Golcarenarenji","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rozlosnik, A., Infrarroja, S.T., and de Bustamante, S. (2018, January 25\u201329). Potential Contribution of the Infrared Industry in the Future of IoT\/IIoT. Proceedings of the 14th Quantitative InfraRed Thermography Conference, Berlin, Germany.","DOI":"10.21611\/qirt.2018.p50"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.comcom.2019.10.001","article-title":"Optimized Unmanned Aerial Vehicles Deployment for Static and Mobile Targets\u2019 Monitoring","volume":"149","author":"Zahmatkesh","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Avgeris, M., Spatharakis, D., Dechouniotis, D., Kalatzis, N., Roussaki, I., and Papavassiliou, S. (2019). Where There Is Fire There Is Smoke: A Scalable Edge Computing Framework for Early Fire Detection. Sensors, 19.","DOI":"10.3390\/s19030639"},{"key":"ref_7","first-page":"5614","article-title":"Joint Optimization for Residual Energy Maximization in Wireless Powered Mobile-Edge Computing Systems","volume":"12","author":"Liu","year":"2018","journal-title":"KSII Trans. Internet Inf. Syst. (TIIS)"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, L., Xu, G., Liu, P., Li, Y., and Ge, J. (2020). Jointly Optimize the Residual Energy of Multiple Mobile Devices in the MEC\u2013WPT System. Future Internet, 12.","DOI":"10.3390\/fi12120233"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2375","DOI":"10.1109\/TWC.2018.2794345","article-title":"Wireless Powered Cooperation-Assisted Mobile Edge Computing","volume":"17","author":"Hu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ahmed, A., and Ahmed, E. (2016, January 7\u20138). A Survey on Mobile Edge Computing. Proceedings of the International Conference on Intelligent Systems & Control, Coimbatore, India.","DOI":"10.1109\/ISCO.2016.7727082"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2322","DOI":"10.1109\/COMST.2017.2745201","article-title":"A Survey on Mobile Edge Computing: The Communication Perspective","volume":"19","author":"Mao","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yu, Z., Xu, G., Li, Y., Liu, P., and Li, L. (2021). Joint Offloading and Energy Harvesting Design in Multiple Time Blocks for FDMA Based Wireless Powered MEC. Future Internet, 13.","DOI":"10.3390\/fi13030070"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2581","DOI":"10.1109\/TMC.2019.2928811","article-title":"Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks","volume":"19","author":"Huang","year":"2019","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/COMST.2014.2368999","article-title":"Wireless Networks with RF Energy Harvesting: A Contemporary Survey","volume":"17","author":"Lu","year":"2014","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3647","DOI":"10.1109\/ACCESS.2016.2579598","article-title":"Cooperative Wireless Energy Harvesting and Spectrum Sharing in 5G Networks","volume":"4","author":"Gao","year":"2016","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3590","DOI":"10.1109\/JSAC.2016.2611964","article-title":"Dynamic Computation Offloading for Mobile-Edge Computing with Energy Harvesting Devices","volume":"34","author":"Mao","year":"2016","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7140","DOI":"10.1109\/TCOMM.2020.3011990","article-title":"Real-Time Resource Allocation for Wireless Powered Multiuser Mobile Edge Computing with Energy and Task Causality","volume":"68","author":"Wang","year":"2020","journal-title":"IEEE Trans. Commun."},{"key":"ref_18","first-page":"4268","article-title":"Mobile-Edge Computing: Partial Computation Offloading Using Dynamic Voltage Scaling","volume":"64","author":"Wang","year":"2016","journal-title":"IEEE Trans. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4924","DOI":"10.1109\/TWC.2017.2703901","article-title":"Computation Offloading and Resource Allocation in Wireless Cellular Networks with Mobile Edge Computing","volume":"16","author":"Wang","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4177","DOI":"10.1109\/TWC.2018.2821664","article-title":"Computation Rate Maximization for Wireless Powered Mobile-Edge Computing with Binary Computation Offloading","volume":"17","author":"Bi","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"856","DOI":"10.1109\/TVT.2018.2881191","article-title":"Joint Task Offloading and Resource Allocation for Multi-Server Mobile-Edge Computing Networks","volume":"68","author":"Tran","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Guo, S., Xiao, B., Yang, Y., and Yang, Y. (2016, January 10\u201314). Energy-Efficient Dynamic Offloading and Resource Scheduling in Mobile Cloud Computing. Proceedings of the IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, USA.","DOI":"10.1109\/INFOCOM.2016.7524497"},{"key":"ref_23","first-page":"3571","article-title":"Offloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling","volume":"65","author":"Dinh","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_24","unstructured":"Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., and Coppin, B. (2015). Deep Reinforcement Learning in Large Discrete Action Spaces. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level Control through Deep Reinforcement Learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bi, S., Huang, L., Wang, H., and Zhang, Y. (2021, January 14\u201323). Stable Online Computation Offloading via Lyapunov-Guided Deep Reinforcement Learning. Proceedings of the ICC 2021-IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500520"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"13065","DOI":"10.1109\/JIOT.2021.3064225","article-title":"A Novel Framework for Mobile-Edge Computing by Optimizing Task Offloading","volume":"8","author":"Naouri","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1109\/JSAC.2016.2545382","article-title":"Energy Efficient Mobile Cloud Computing Powered by Wireless Energy Transfer","volume":"34","author":"You","year":"2016","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1109\/TIT.2007.915717","article-title":"Power-Efficient Resource Allocation for Time-Division Multiple Access over Fading Channels","volume":"54","author":"Wang","year":"2008","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1784","DOI":"10.1109\/TWC.2017.2785305","article-title":"Joint Offloading and Computing Optimization in Wireless Powered Mobile-Edge Computing Systems","volume":"17","author":"Wang","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1927","DOI":"10.1109\/JSAC.2018.2864426","article-title":"Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems","volume":"36","author":"Zhou","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Liu, T., Zhu, Y., and Yang, Y. (2020, January 15\u201317). A Deep Reinforcement Learning Approach for Online Computation Offloading in Mobile Edge Computing. Proceedings of the 2020 IEEE\/ACM 28th International Symposium on Quality of Service (IWQoS), Hangzhou, China.","DOI":"10.1109\/IWQoS49365.2020.9212868"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"7519","DOI":"10.1109\/TWC.2021.3085319","article-title":"Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks","volume":"20","author":"Bi","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1109\/TVT.2018.2890685","article-title":"Learning-Based Computation Offloading for IoT Devices with Energy Harvesting","volume":"68","author":"Min","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhao, N., Ye, Z., Pei, Y., Liang, Y.-C., and Niyato, D. (2022). Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-Assisted Mobile Edge Computing. IEEE Trans. Wirel. Commun., 1.","DOI":"10.1109\/TWC.2022.3153316"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"53708","DOI":"10.1109\/ACCESS.2021.3070908","article-title":"Task Offloading and Trajectory Control for UAV-Assisted Mobile Edge Computing Using Deep Reinforcement Learning","volume":"9","author":"Zhang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2390","DOI":"10.1109\/TCOMM.2014.2328635","article-title":"Characterizing the Spectral Properties and Time Variation of the In-Vehicle Wireless Communication Channel","volume":"62","author":"Herbert","year":"2014","journal-title":"IEEE Trans. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mao, S., Leng, S., Yang, K., Huang, X., and Zhao, Q. (2017, January 4\u20138). Fair Energy-Efficient Scheduling in Wireless Powered Full-Duplex Mobile-Edge Computing Systems. Proceedings of the GLOBECOM 2017-2017 IEEE Global Communications Conference, Singapore.","DOI":"10.1109\/GLOCOM.2017.8254208"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Boyd, S., Boyd, S.P., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Calmet, J., Benhamou, B., Caprotti, O., Henocque, L., and Sorge, V. (2002). Artificial Intelligence, Automated Reasoning, and Symbolic Computation. Prceeedings AISC 2002 and Calculemus 2002, Marseille, France, 1\u20135 July 2002, Springer. Available online: https:\/\/link.springer.com\/book\/10.1007\/3-540-45470-5.","DOI":"10.1007\/3-540-45470-5"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Tato, A., and Nkambou, R. (2022, July 07). Infusing Expert Knowledge into a Deep Neural Network Using Attention Mechanism for Personalized Learning Environments. Available online: https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2022.921476\/full.","DOI":"10.3389\/frai.2022.921476"},{"key":"ref_42","unstructured":"Dozat, T. (2022, July 07). Incorporating Nesterov Momentum into Adam; ICLR 2016 Workshop; 11 March 2016. Available online: https:\/\/openreview.net\/pdf\/OM0jvwB8jIp57ZJjtNEZ.pdf."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/8\/226\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:56:52Z","timestamp":1760140612000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/8\/226"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,26]]},"references-count":42,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["fi14080226"],"URL":"https:\/\/doi.org\/10.3390\/fi14080226","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2022,7,26]]}}}