{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T11:14:24Z","timestamp":1777806864460,"version":"3.51.4"},"reference-count":32,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,1,7]],"date-time":"2021-01-07T00:00:00Z","timestamp":1609977600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61571038"],"award-info":[{"award-number":["61571038"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61931001"],"award-info":[{"award-number":["61931001"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100015013","name":"National Mobile Communications Research Laboratory, Southeast University","doi-asserted-by":"publisher","award":["2020D06"],"award-info":[{"award-number":["2020D06"]}],"id":[{"id":"10.13039\/501100015013","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the rapid development of vehicular networks, vehicle-to-everything (V2X) communications have huge number of tasks to be calculated, which brings challenges to the scarce network resources. Cloud servers can alleviate the terrible situation regarding the lack of computing abilities of vehicular user equipment (VUE), but the limited resources, the dynamic environment of vehicles, and the long distances between the cloud servers and VUE induce some potential issues, such as extra communication delay and energy consumption. Fortunately, mobile edge computing (MEC), a promising computing paradigm, can ameliorate the above problems by enhancing the computing abilities of VUE through allocating the computational resources to VUE. In this paper, we propose a joint optimization algorithm based on a deep reinforcement learning algorithm named the double deep Q network (double DQN) to minimize the cost constituted of energy consumption, the latency of computation, and communication with the proper policy. The proposed algorithm is more suitable for dynamic scenarios and requires low-latency vehicular scenarios in the real world. Compared with other reinforcement learning algorithms, the algorithm we proposed algorithm improve the performance in terms of convergence, defined cost, and speed by around 30%, 15%, and 17%.<\/jats:p>","DOI":"10.3390\/s21020372","type":"journal-article","created":{"date-parts":[[2021,1,10]],"date-time":"2021-01-10T23:03:42Z","timestamp":1610319822000},"page":"372","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Deep Reinforcement Learning-Empowered Resource Allocation for Mobile Edge Computing in Cellular V2X Networks"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8026-5038","authenticated-orcid":false,"given":"Dongji","family":"Li","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5737-2874","authenticated-orcid":false,"given":"Shaoyi","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China"},{"name":"National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pengyu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/JIOT.2017.2750180","article-title":"Mobile Edge Computing: A Survey","volume":"5","author":"Abbas","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1109\/TII.2017.2777846","article-title":"Collaborative Energy and Information Transfer in Green Wireless Sensor Networks for Smart Cities","volume":"14","author":"Lu","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/MCOM.2009.4939288","article-title":"Wave: A tutorial","volume":"47","author":"Uzcategui","year":"2009","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MWC.2017.8246820","article-title":"Emerging technology for 5G-enabled vehicular networks","volume":"24","author":"Vinel","year":"2017","journal-title":"IEEE Wirel. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Alahmadi, A.A., Lawey, A.Q., El-Gorashi, T.E., and Elmirghani, J.M. (2017, January 22\u201324). Distributed processing in vehicular cloud networks. Proceedings of the 2017 8th International Conference on the Network of the Future (NOF), London, UK.","DOI":"10.1109\/NOF.2017.8251215"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"7944","DOI":"10.1109\/TVT.2019.2917890","article-title":"Computation offloading and resources allocation for cloud assisted mobile edge. Computing in vehicular networks","volume":"68","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Behbehani, F.S., Musa, M., Elgorashi, T., and Elmirghani, J.M. (2019). Energy-Efficient Distributed Processing in Vehicular Cloud Architecture. arXiv.","DOI":"10.1109\/ICTON.2019.8840335"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"732","DOI":"10.1109\/COMST.2015.2481183","article-title":"Data Center Energy Consumption Modeling: A Survey","volume":"18","author":"Dayarathna","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/MCOMSTD.2018.1800013","article-title":"Multi-access Edge Computing: The driver behind the wheel of 5G-connected cars","volume":"2","author":"Giust","year":"2018","journal-title":"IEEE Commun. Stand. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1109\/LWC.2017.2740927","article-title":"Performance guaranteed computation offloading for mobile-edge cloud computing","volume":"6","author":"Tao","year":"2017","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MCE.2016.2590118","article-title":"Mobile-edge computing architecture: The role of MEC in the Internet of Things","volume":"5","author":"Sabella","year":"2016","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.adhoc.2012.04.009","article-title":"Improving path duration in high mobility vehicular ad hoc networks","volume":"11","author":"Oliveira","year":"2013","journal-title":"Ad Hoc Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Technical Note: Q-Learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn."},{"key":"ref_15","unstructured":"Thrun, S., and Anton, S. (1993). Issues in using function approximation for reinforcement learning. Proceedings of the 1993 Connectionist Models Summer School Hillsdale, Lawrence Erlbaum."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2795","DOI":"10.1109\/TNET.2015.2487344","article-title":"Efficient multi-user computation offloading for mobile-edge cloud computing","volume":"24","author":"Chen","year":"2015","journal-title":"IEEE ACM Trans. Netw."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ma, X., Lin, C., Xiang, X., and Chen, C. (2015, January 2\u20136). Game-theoretic analysis of computation offloading for cloudlet-based mobile cloud computing. Proceedings of the 18th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Cancun, Mexico.","DOI":"10.1145\/2811587.2811598"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liwang, M., Gao, Z., Hosseinalipour, S., and Dai, H. (2020, January 7\u201311). Multi-Task Offloading over Vehicular Clouds under Graph-based Representation. Proceedings of the ICC 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9149265"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ma, X., Zhang, S., Li, W., Zhang, P., Lin, C., and Shen, X. (2017, January 14\u201316). Cost-efficient workload scheduling in cloud assisted mobile edge computing. Proceedings of the 2017 IEEE\/ACM 25th International Symposium on Quality of Service (IWQoS), Vilanova i la Geltru, Spain.","DOI":"10.1109\/IWQoS.2017.7969148"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7938","DOI":"10.1109\/TIE.2015.2481792","article-title":"Cooperative resources management in cloud-enabled vehicular networks","volume":"62","author":"Yu","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, J., Xia, W., Cheng, Z., Zou, Q., Huang, B., Shen, F., Yan, F., and Shen, L. (2017, January 11\u201312). An evolutionary game for joint wireless and cloud resources allocation in mobile edge computing. Proceedings of the 2017 9th International Conference on Wireless Communications and Signal Processing (WCSP), Nanjing, China.","DOI":"10.1109\/WCSP.2017.8170956"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4192","DOI":"10.1109\/TVT.2019.2894437","article-title":"Knowledge-Driven Service Offloading Decision for Vehicular Edge Computing: A Deep Reinforcement Learning Approach","volume":"68","author":"Qi","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e3818","DOI":"10.1002\/ett.3818","article-title":"Joint optimization of computation cost and delay for task offloading in vehicular fog networks","volume":"31","author":"Li","year":"2020","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"54288","DOI":"10.1109\/ACCESS.2020.2981463","article-title":"Cooperative Autonomous Driving Oriented MEC-Aided 5G-V2X: Prototype System Design, Field Tests and AI-Based Optimization Tools","volume":"8","author":"Ma","year":"2020","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"266","DOI":"10.23919\/JCC.2020.05.020","article-title":"V2X offloading and resource allocation in SDN-assisted MEC-based vehicular networks","volume":"17","author":"Zhang","year":"2020","journal-title":"China Commun."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1109\/TPDS.2014.2316834","article-title":"Decentralized Computation Offloading Game for Mobile Cloud Computing","volume":"26","author":"Chen","year":"2015","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5212","DOI":"10.1109\/TWC.2018.2839183","article-title":"Cellular V2X in Unlicensed Spectrum: Harmonious Coexistence with VANET in 5G systems","volume":"17","author":"Wang","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"15576","DOI":"10.1109\/ACCESS.2018.2810115","article-title":"Joint optimization of energy consumption and packet scheduling for mobile edge computing in cyber-physical networks","volume":"6","author":"Yang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","unstructured":"Hasselt, H.V. (2010). Double Q-learning. Advances in Neural Information Processing Systems, Mit Press."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016). Deep Reinforcement Learning with Double Q-Learning, Association for the Advance of Artificial Intelligence.","DOI":"10.1609\/aaai.v30i1.10295"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/372\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:08:10Z","timestamp":1760159290000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/372"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,7]]},"references-count":32,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21020372"],"URL":"https:\/\/doi.org\/10.3390\/s21020372","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,7]]}}}