{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:09:54Z","timestamp":1760058594080,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T00:00:00Z","timestamp":1744588800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Fundamental Research Funds for the Central Universities","award":["2022JBZY021","L234084","62071033"],"award-info":[{"award-number":["2022JBZY021","L234084","62071033"]}]},{"name":"Changping Innovation Joint Fund of Beijing Natural Science Foundation","award":["2022JBZY021","L234084","62071033"],"award-info":[{"award-number":["2022JBZY021","L234084","62071033"]}]},{"name":"National Natural Science Foundation of China","award":["2022JBZY021","L234084","62071033"],"award-info":[{"award-number":["2022JBZY021","L234084","62071033"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Network"],"abstract":"<jats:p>With the widespread deployment of various emerging intelligent applications, information timeliness is crucial for intelligent decision-making in vehicular networks, where vehicular edge computing (VEC) has become an important paradigm to enhance computing capabilities by offloading tasks to edge nodes. To promote the information timeliness in VEC, an optimization problem is formulated to minimize the age of information (AoI) by jointly optimizing task offloading and subcarrier allocation. Due to the time-varying channel and the coupling of the continuous and discrete optimization variables, the problem exhibits non-convexity, which is difficult to solve using traditional mathematical optimization methods. To efficiently tackle this challenge, we employ a hybrid proximal policy optimization (HPPO)-based deep reinforcement learning (DRL) method by designing the mixed action space involving both continuous and discrete variables. Moreover, an action masking mechanism is designed to filter out invalid actions in the action space caused by limitations in the effective communication distance between vehicles. As a result, a mask-assisted HPPO (MHPPO) method is proposed by integrating the action masking mechanism into the HPPO. Simulation results show that the proposed MHPPO method achieves an approximately 28.9% reduction in AoI compared with the HPPO method and about a 23% reduction compared with the mask-assisted deep deterministic policy gradient (MDDPG).<\/jats:p>","DOI":"10.3390\/network5020012","type":"journal-article","created":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T07:42:01Z","timestamp":1744616521000},"page":"12","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Age of Information Minimization in Vehicular Edge Computing Networks: A Mask-Assisted Hybrid PPO-Based Method"],"prefix":"10.3390","volume":"5","author":[{"given":"Xiaoli","family":"Qin","sequence":"first","affiliation":[{"name":"Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8623-5262","authenticated-orcid":false,"given":"Zhifei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3812-4406","authenticated-orcid":false,"given":"Chanyuan","family":"Meng","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China"}]},{"given":"Rui","family":"Dong","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9364-0207","authenticated-orcid":false,"given":"Ke","family":"Xiong","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0658-6079","authenticated-orcid":false,"given":"Pingyi","family":"Fan","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1109\/COMST.2021.3059896","article-title":"A Survey on Resource Allocation for 5G Heterogeneous Networks: Current Research, Future Trends, and Challenges","volume":"23","author":"Xu","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Pham, X.Q., Nguyen, T.D., Nguyen, V., and Huh, E.N. (2019). Joint node selection and resource allocation for task offloading in scalable vehicle-assisted multi-access edge computing. Symmetry, 11.","DOI":"10.3390\/sym11010058"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cui, T., Hu, Y., Shen, B., and Chen, Q. (2019). Task offloading based on lyapunov optimization for mec-assisted vehicular platooning networks. Sensors, 19.","DOI":"10.3390\/s19224974"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"108797","DOI":"10.1016\/j.comnet.2022.108797","article-title":"Mobile edge computing for V2X architectures and applications: A survey","volume":"206","author":"Kacimi","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3882","DOI":"10.1109\/TMC.2022.3153346","article-title":"DRL-Based V2V Computation Offloading for Blockchain-Enabled Vehicular Networks","volume":"22","author":"Shi","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zeng, F., Zhang, Z., and Wu, J. Task offloading delay minimization in vehicular edge computing based on vehicle trajectory prediction. Digit. Commun. Netw., 2024. in press.","DOI":"10.1016\/j.dcan.2024.08.003"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Nie, X., Yan, Y., Zhou, T., Chen, X., and Zhang, D. (2023). A delay-optimal task scheduling strategy for vehicle edge computing based on the multi-agent deep reinforcement learning approach. Electronics, 12.","DOI":"10.3390\/electronics12071655"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5214","DOI":"10.1109\/JIOT.2022.3221966","article-title":"Energy consumption and qos-aware co-offloading for vehicular edge computing","volume":"10","author":"Lv","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"10709","DOI":"10.1109\/TWC.2022.3186590","article-title":"Energy-efficient cooperative offloading for edge computing-enabled vehicular networks","volume":"21","author":"Cho","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, Z., Jia, Z., and Pang, X. (2023). DRL-based hybrid task offloading and resource allocation in vehicular networks. Electronics, 12.","DOI":"10.3390\/electronics12214392"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1830","DOI":"10.1109\/TIV.2023.3321679","article-title":"EPtask: Deep Reinforcement Learning Based Energy-Efficient and Priority-Aware Task Scheduling for Dynamic Vehicular Edge Computing","volume":"9","author":"Li","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4377","DOI":"10.1109\/JIOT.2018.2876298","article-title":"Joint Load Balancing and Offloading in Vehicular Edge Computing and Networks","volume":"6","author":"Dai","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10466","DOI":"10.1109\/ACCESS.2020.2965620","article-title":"Joint optimization of computation offloading and task scheduling in vehicular edge computing networks","volume":"8","author":"Sun","year":"2020","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5343","DOI":"10.1109\/TVT.2022.3151806","article-title":"Latency Minimization of Reverse Offloading in Vehicular Edge Computing","volume":"71","author":"Feng","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"16369","DOI":"10.1109\/TVT.2023.3289236","article-title":"Latency-Energy Joint Optimization for Task Offloading and Resource Allocation in MEC-Assisted Vehicular Networks","volume":"72","author":"Cong","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"11934","DOI":"10.1109\/TVT.2024.3375840","article-title":"Mobility-Aware Computation Offloading and Resource Allocation for NOMA MEC in Vehicular Networks","volume":"73","author":"Li","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"6547","DOI":"10.1109\/TWC.2023.3244391","article-title":"Joint Task Offloading and Resource Allocation for Vehicular Edge Computing with Result Feedback Delay","volume":"22","author":"Nan","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"16266","DOI":"10.1109\/TVT.2023.3298599","article-title":"Location-Aware and Delay-Minimizing Task Offloading in Vehicular Edge Computing Networks","volume":"72","author":"Xia","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1807","DOI":"10.1109\/TNET.2023.3330255","article-title":"Multi-User Layer-Aware Online Container Migration in Edge-Assisted Vehicular Networks","volume":"32","author":"Tang","year":"2024","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1109\/TVT.2023.3306939","article-title":"Partial Offloading and Resource Allocation for MEC-Assisted Vehicular Networks","volume":"73","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4277","DOI":"10.1109\/TITS.2022.3230430","article-title":"Joint Task Offloading and Resource Allocation for Vehicular Edge Computing Based on V2I and V2V Modes","volume":"24","author":"Fan","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"11918","DOI":"10.1109\/TITS.2024.3371096","article-title":"Joint Spectrum Sharing and V2V\/V2I Task Offloading for Vehicular Edge Computing Networks Based on Coalition Formation Game","volume":"25","author":"Huang","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"102778","DOI":"10.1109\/ACCESS.2022.3206359","article-title":"Delay Constrained Hybrid Task Offloading of Internet of Vehicle: A Deep Reinforcement Learning Method","volume":"10","author":"Wu","year":"2022","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5546","DOI":"10.1109\/TVT.2023.3312301","article-title":"Road Side Unit-Assisted Learning-Based Partial Task Offloading for Vehicular Edge Computing System","volume":"73","author":"Li","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"103497","DOI":"10.1016\/j.adhoc.2024.103497","article-title":"Vehicle task offloading strategy based on DRL in communication and sensing scenarios","volume":"159","author":"Xue","year":"2024","journal-title":"Ad Hoc Netw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"20137","DOI":"10.1109\/TITS.2024.3472033","article-title":"Minimizing AoI in High-Speed Railway Mobile Networks: DQN-Based Methods","volume":"25","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"8729","DOI":"10.1109\/TMC.2024.3356229","article-title":"AoI-Minimal Power Adjustment in RF-EH-Powered Industrial IoT Networks: A Soft Actor-Critic-Based Method","volume":"23","author":"Ge","year":"2024","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"103164","DOI":"10.1016\/j.adhoc.2023.103164","article-title":"Average AoI minimization for data collection in UAV-enabled IoT backscatter communication systems with the finite blocklength regime","volume":"145","author":"Shen","year":"2023","journal-title":"Ad Hoc Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"153","DOI":"10.23919\/JCC.ea.2021-0544.202401","article-title":"Age of information based user scheduling and data assignment in multi-user mobile edge computing networks: An online algorithm","volume":"21","author":"Yiyang","year":"2024","journal-title":"China Commun."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"13115","DOI":"10.1109\/JIOT.2021.3082281","article-title":"Freshness-Aware Information Update and Computation Offloading in Mobile-Edge Computing","volume":"8","author":"Ma","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Narayanasamy, I., and Rajamanickam, V. (2024). A Cascaded Multi-Agent Reinforcement Learning-Based Resource Allocation for Cellular-V2X Vehicular Platooning Networks. Sensors, 24.","DOI":"10.3390\/s24175658"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"25287","DOI":"10.1109\/TITS.2022.3180928","article-title":"Age Efficient Optimization in UAV-Aided VEC Network: A Game Theory Viewpoint","volume":"23","author":"Han","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"6764","DOI":"10.1109\/JIOT.2024.3492535","article-title":"AoI-Energy-Efficient Edge Caching in UAV-Assisted Vehicular Networks","volume":"12","author":"Xiao","year":"2025","journal-title":"IEEE Internet Things J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"19782","DOI":"10.1109\/JIOT.2023.3283287","article-title":"Age-of-Information-Based Computation Offloading and Transmission Scheduling in Mobile-Edge-Computing-Enabled IoT Networks","volume":"10","author":"Jiang","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"38248","DOI":"10.1109\/ACCESS.2020.2976048","article-title":"Joint Task Offloading and Resource Allocation for Obtaining Fresh Status Updates in Multi-Device MEC Systems","volume":"8","author":"Liu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Xiao, L., Lin, Y., Zhang, Y., Li, J., and Shu, F. (2024, January 24\u201327). AoI-Aware Energy-Efficient Vehicular Edge Computing Using Multi-Agent Reinforcement Learning with Actor-Attention-Critic. Proceedings of the 2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring), Singapore.","DOI":"10.1109\/VTC2024-Spring62846.2024.10683441"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"9521","DOI":"10.1109\/TVT.2017.2714863","article-title":"Mobile Service Amount Based Link Scheduling for High-Mobility Cooperative Vehicular Networks","volume":"66","author":"Xiong","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1972","DOI":"10.1109\/COMST.2021.3057017","article-title":"Casta\u00f1eda and Molina-Galan, Alejandro and Boban, Mate and Gozalvez, Javier and Coll-Perales, Baldomero and \u015eahin, Taylan and Kousaridas, Apostolos A Tutorial on 5G NR V2X Communications","volume":"23","author":"Garcia","year":"2021","journal-title":"IEEE Commun. Surv. Tutor"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1833","DOI":"10.1109\/TGCN.2022.3157735","article-title":"Joint Optimization of Trajectory, Task Offloading, and CPU Control in UAV-Assisted Wireless Powered Fog Computing Networks","volume":"6","author":"Xiong","year":"2022","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2084","DOI":"10.1109\/TMC.2021.3115348","article-title":"Distributed Design of Wireless Powered Fog Computing Networks with Binary Computation Offloading","volume":"22","author":"Li","year":"2023","journal-title":"IEEE Trans. Mobile Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5667","DOI":"10.1109\/JIOT.2023.3309859","article-title":"Sum-Rate Maximization in STAR-RIS-Assisted RSMA Networks: A PPO-Based Algorithm","volume":"11","author":"Meng","year":"2024","journal-title":"IEEE Internet Things J."}],"container-title":["Network"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-8732\/5\/2\/12\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:14:13Z","timestamp":1760030053000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-8732\/5\/2\/12"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,14]]},"references-count":41,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["network5020012"],"URL":"https:\/\/doi.org\/10.3390\/network5020012","relation":{},"ISSN":["2673-8732"],"issn-type":[{"type":"electronic","value":"2673-8732"}],"subject":[],"published":{"date-parts":[[2025,4,14]]}}}