{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T08:33:59Z","timestamp":1792139639424,"version":"4.4.0"},"reference-count":45,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFB3305500"],"award-info":[{"award-number":["2022YFB3305500"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["62273089"],"award-info":[{"award-number":["62273089"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["62102080"],"award-info":[{"award-number":["62102080"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["BK20210204"],"award-info":[{"award-number":["BK20210204"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2022YFB3305500"],"award-info":[{"award-number":["2022YFB3305500"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62273089"],"award-info":[{"award-number":["62273089"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62102080"],"award-info":[{"award-number":["62102080"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["BK20210204"],"award-info":[{"award-number":["BK20210204"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["2022YFB3305500"],"award-info":[{"award-number":["2022YFB3305500"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["62273089"],"award-info":[{"award-number":["62273089"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["62102080"],"award-info":[{"award-number":["62102080"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20210204"],"award-info":[{"award-number":["BK20210204"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Efficient task offloading decision is a crucial technology in vehicular edge computing, which aims to fulfill the computational performance demands of complex vehicular tasks with respect to delay and energy consumption while minimizing network resource competition and consumption. Conventional distributed task offloading decisions rely solely on the local state of the vehicle, failing to optimize the utilization of the server\u2019s resources to its fullest potential. In addition, the mobility aspect of vehicles is often neglected in these decisions. In this paper, a cloud-edge-vehicle three-tier vehicular edge computing (VEC) system is proposed, where vehicles partially offload their computing tasks to edge or cloud servers while keeping the remaining tasks local to the vehicle terminals. Under the restrictions of vehicle mobility and discrete variables, task scheduling and task offloading proportion are jointly optimized with the objective of minimizing the total system cost. Considering the non-convexity, high-dimensional complex state and continuous action space requirements of the optimization problem, we propose a task offloading decision-making algorithm based on deep deterministic policy gradient (TODM_DDPG). TODM_DDPG algorithm adopts the actor\u2013critic framework in which the actor network outputs floating point numbers to represent deterministic policy, while the critic network evaluates the action output by the actor network, and adjusts the network evaluation policy according to the rewards with the environment to maximize the long-term reward. To explore the algorithm performance, this conduct parameter setting experiments to correct the algorithm core hyper-parameters and select the optimal combination of parameters. In addition, in order to verify algorithm performance, we also carry out a series of comparative experiments with baseline algorithms. The results demonstrate that in terms of reducing system costs, the proposed algorithm outperforms the compared baseline algorithm, such as the deep Q network (DQN) and the actor\u2013critic (AC), and the performance is improved by about 13% on average.<\/jats:p>","DOI":"10.3390\/s23177595","type":"journal-article","created":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T09:12:22Z","timestamp":1693559542000},"page":"7595","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Task Offloading Decision-Making Algorithm for Vehicular Edge Computing: A Deep-Reinforcement-Learning-Based Approach"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-5787-6003","authenticated-orcid":false,"given":"Wei","family":"Shi","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Southeast University, Nanjing 211189, China"},{"name":"The Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 211189, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Long","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Southeast University, Nanjing 211189, China"},{"name":"The Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 211189, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xia","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Southeast University, Nanjing 211189, China"},{"name":"The Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 211189, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1109\/JAS.2020.1003021","article-title":"Artificial intelligence applications in the development of autonomous vehicles: A survey","volume":"7","author":"Ma","year":"2020","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_2","first-page":"1","article-title":"An Improved Deep Network-Based Scene Classification Method for Self-Driving Cars","volume":"71","author":"Ni","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MWC.2019.1800522","article-title":"Internet of Autonomous Vehicles: Architecture, Features, and Socio-Technological Challenges","volume":"26","author":"Jameel","year":"2019","journal-title":"IEEE Wirel. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"104089","DOI":"10.1016\/j.scs.2022.104089","article-title":"Applications of ML\/DL in the management of smart cities and societies based on new trends in information technologies: A systematic literature review","volume":"85","author":"Heidari","year":"2022","journal-title":"Sustain. Cities Soc."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3832","DOI":"10.1109\/TITS.2020.3048844","article-title":"Resource Allocation in 5G IoV Architecture Based on SDN and Fog-Cloud Computing","volume":"22","author":"Cao","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/JIOT.2018.2872437","article-title":"5-GHz Obstructed Vehicle-to-Vehicle Channel Characterization for Internet of Intelligent Vehicles","volume":"6","author":"Guan","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1002\/ett.2790","article-title":"A novel link allocation method for vehicle-to-vehicle-based relaying networks","volume":"27","author":"Zheng","year":"2016","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sewalkar, P., and Seitz, J. (2019). Vehicle-to-Pedestrian Communication for Vulnerable Road Users: Survey, Design Considerations, and Challenges. Sensors, 19.","DOI":"10.3390\/s19020358"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4374","DOI":"10.1109\/TWC.2018.2824333","article-title":"Performance Analysis and Power Control for Multi-Antenna V2V Underlay Massive MIMO","volume":"17","author":"Liu","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2378","DOI":"10.1109\/COMST.2020.3005361","article-title":"Caching in Vehicular Named Data Networking: Architecture, Schemes and Future Directions","volume":"22","author":"Chen","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7097","DOI":"10.1109\/JIOT.2020.2982292","article-title":"Reliable Computation Offloading for Edge-Computing-Enabled Software-Defined IoV","volume":"7","author":"Hou","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1109\/MCOM.2017.1600863","article-title":"Collaborative Mobile Edge Computing in 5G Networks: New Paradigms, Scenarios, and Challenges","volume":"55","author":"Tran","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2581","DOI":"10.1109\/TMC.2019.2928811","article-title":"Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks","volume":"19","author":"Huang","year":"2020","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4584","DOI":"10.1109\/TVT.2021.3133586","article-title":"Efficient Multi-Vehicle Task Offloading for Mobile Edge Computing in 6G Networks","volume":"71","author":"Chen","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1109\/JPROC.2019.2947490","article-title":"Mobile Edge Intelligence and Computing for the Internet of Vehicles","volume":"108","author":"Zhang","year":"2020","journal-title":"Proc. IEEE"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.comcom.2019.11.019","article-title":"Efficient task scheduling for servers with dynamic states in vehicular edge computing","volume":"150","author":"Wu","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"8175","DOI":"10.1109\/TVT.2022.3175238","article-title":"Energy-Delay Minimization of Task Migration Based on Game Theory in MEC-Assisted Vehicular Networks","volume":"71","author":"Wang","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1687","DOI":"10.1109\/JIOT.2020.3015534","article-title":"Modeling Data Redundancy and Cost-Aware Task Allocation in MEC-Enabled Internet-of-Vehicles Applications","volume":"8","author":"Nguyen","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2252","DOI":"10.1109\/TITS.2020.3016002","article-title":"QoE-Based Task Offloading With Deep Reinforcement Learning in Edge-Enabled Internet of Vehicles","volume":"22","author":"He","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10227","DOI":"10.1109\/TVT.2019.2937825","article-title":"Edge-Assisted Vehicle Mobility Prediction to Support V2X Communications","volume":"68","author":"Liu","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep Reinforcement Learning for Autonomous Driving: A Survey","volume":"23","author":"Kiran","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Song, I., Tam, P., Kang, S., Ros, S., and Kim, S. (2023). DRL-Based Backbone SDN Control Methods in UAV-Assisted Networks for Computational Resource Efficiency. Electronics, 12.","DOI":"10.3390\/electronics12132984"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1109\/TITS.2018.2815678","article-title":"Big Data Analytics in Intelligent Transportation Systems: A Survey","volume":"20","author":"Zhu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2897","DOI":"10.1109\/TSC.2021.3064579","article-title":"Minimizing the Delay and Cost of Computation Offloading for Vehicular Edge Computing","volume":"15","author":"Luo","year":"2022","journal-title":"IEEE Trans. Serv. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"10466","DOI":"10.1109\/ACCESS.2020.2965620","article-title":"Joint Optimization of Computation Offloading and Task Scheduling in Vehicular Edge Computing Networks","volume":"8","author":"Sun","year":"2020","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, M., Zhai, Q., Chen, Y., Feng, S., and Shu, F. (2021). Multi-Objective Whale Optimization Algorithm for Computation Offloading Optimization in Mobile Edge Computing. Sensors, 21.","DOI":"10.3390\/s21082628"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2795","DOI":"10.1109\/TNET.2015.2487344","article-title":"Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing","volume":"24","author":"Chen","year":"2016","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5792","DOI":"10.1109\/JIOT.2019.2952767","article-title":"A Cloud\u2013MEC Collaborative Task Offloading Scheme With Service Orchestration","volume":"7","author":"Huang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4791","DOI":"10.1109\/JIOT.2018.2869226","article-title":"Joint Optimization of Energy Consumption and Latency in Mobile Edge Computing for Internet of Things","volume":"6","author":"Cui","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.1109\/JIOT.2020.3033285","article-title":"Offloading Time Optimization via Markov Decision Process in Mobile-Edge Computing","volume":"8","author":"Yang","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of Deep Reinforcement Learning in Communications and Networking: A Survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2226","DOI":"10.1109\/TITS.2020.3015210","article-title":"Intelligent Task Offloading for Heterogeneous V2X Communications","volume":"22","author":"Xiong","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.dcan.2018.10.003","article-title":"Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing","volume":"5","author":"Huang","year":"2019","journal-title":"Digit. Commun. Netw."},{"key":"ref_34","first-page":"172","article-title":"Task distribution offloading algorithm of vehicle edge network based on DQN","volume":"41","author":"Zhao","year":"2020","journal-title":"J. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"173779","DOI":"10.1109\/ACCESS.2020.3023939","article-title":"Task Offloading Strategy Based on Reinforcement Learning Computing in Edge Computing Architecture of Internet of Vehicles","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"137052","DOI":"10.1109\/ACCESS.2020.3011705","article-title":"Advanced Deep Learning-Based Computational Offloading for Multilevel Vehicular Edge-Cloud Computing Networks","volume":"8","author":"Khayyat","year":"2020","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Fu, J., Qin, X., Huang, Y., Tang, L., and Liu, Y. (2022). Deep Reinforcement Learning-Based Resource Allocation for Cellular Vehicular Network Mode 3 with Underlay Approach. Sensors, 22.","DOI":"10.3390\/s22051874"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"7279","DOI":"10.1109\/JIOT.2020.2982699","article-title":"DRL-Based Energy-Efficient Resource Allocation Frameworks for Uplink NOMA Systems","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.comcom.2021.04.028","article-title":"Deep reinforcement learning for computation offloading in mobile edge computing environment","volume":"175","author":"Chen","year":"2021","journal-title":"Comput. Commun."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"9255","DOI":"10.1109\/JIOT.2020.2981557","article-title":"Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things","volume":"7","author":"Lu","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1145\/3317572","article-title":"Deep Reinforcement Learning for Vehicular Edge Computing: An Intelligent Offloading System","volume":"10","author":"Ning","year":"2019","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"108690","DOI":"10.1016\/j.comnet.2021.108690","article-title":"Optimization for computational offloading in multi-access edge computing: A deep reinforcement learning scheme","volume":"204","author":"Wang","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"53040","DOI":"10.1109\/ACCESS.2019.2912200","article-title":"Review of Deep Learning Algorithms and Architectures","volume":"7","author":"Shrestha","year":"2019","journal-title":"IEEE Access"},{"key":"ref_44","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"8577","DOI":"10.1109\/JIOT.2019.2921159","article-title":"Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications","volume":"6","author":"Qiu","year":"2019","journal-title":"IEEE Internet Things J."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/17\/7595\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:44:52Z","timestamp":1760129092000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/17\/7595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,1]]},"references-count":45,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["s23177595"],"URL":"https:\/\/doi.org\/10.3390\/s23177595","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,1]]}}}