{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T16:13:20Z","timestamp":1776269600932,"version":"3.50.1"},"reference-count":37,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,2,27]],"date-time":"2022-02-27T00:00:00Z","timestamp":1645920000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Vehicle-to-vehicle (V2V) communication has attracted increasing attention since it can improve road safety and traffic efficiency. In the underlay approach of mode 3, the V2V links need to reuse the spectrum resources preoccupied with vehicle-to-infrastructure (V2I) links, which will interfere with the V2I links. Therefore, how to allocate wireless resources flexibly and improve the throughput of the V2I links while meeting the low latency requirements of the V2V links needs to be determined. This paper proposes a V2V resource allocation framework based on deep reinforcement learning. The base station (BS) uses a double deep Q network to allocate resources intelligently. In particular, to reduce the signaling overhead for the BS to acquire channel state information (CSI) in mode 3, the BS optimizes the resource allocation strategy based on partial CSI in the framework of this article. The simulation results indicate that the proposed scheme can meet the low latency requirements of V2V links while increasing the capacity of the V2I links compared with the other methods. In addition, the proposed partial CSI design has comparable performance to complete CSI.<\/jats:p>","DOI":"10.3390\/s22051874","type":"journal-article","created":{"date-parts":[[2022,2,27]],"date-time":"2022-02-27T20:48:33Z","timestamp":1645994913000},"page":"1874","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Deep Reinforcement Learning-Based Resource Allocation for Cellular Vehicular Network Mode 3 with Underlay Approach"],"prefix":"10.3390","volume":"22","author":[{"given":"Jinjuan","family":"Fu","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xizhong","family":"Qin","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Huang","sequence":"additional","affiliation":[{"name":"Network Department, China Mobile Communications Group Xinjiang Co., Ltd., Urumqi 830000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Tang","sequence":"additional","affiliation":[{"name":"Network Department, China Mobile Communications Group Xinjiang Co., Ltd., Urumqi 830000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Balkus, S.V., Wang, H., Cornet, B.D., Mahabal, C., Ngo, H., and Fang, H. (2022). A Survey of Collaborative Machine Learning Using 5G Vehicular Communications. IEEE Commun. Surv. Tutor.","DOI":"10.1109\/COMST.2022.3149714"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3396","DOI":"10.1109\/TVT.2021.3063694","article-title":"Performance Analysis of Cellular-Relay Vehicle-to-Vehicle Communications","volume":"70","author":"Kimura","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1109\/TITS.2020.3019322","article-title":"A Survey on Resource Allocation in Vehicular Networks","volume":"23","author":"Rahim","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"181117","DOI":"10.1109\/ACCESS.2019.2954466","article-title":"Emerging Technologies for 5G-Enabled Vehicular Networks","volume":"7","author":"Yang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"123117","DOI":"10.1109\/ACCESS.2021.3109894","article-title":"Comprehensive Survey of Radio Resource Allocation Schemes for 5G V2X Communications","volume":"9","author":"Le","year":"2021","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1109\/COMST.2020.3029723","article-title":"Challenges and Solutions for Cellular Based V2X Communications","volume":"23","author":"Gyawali","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1726","DOI":"10.1109\/TVT.2021.3134272","article-title":"Multi-Agent Deep Reinforcement Learning-Empowered Channel Allocation in Vehicular Networks","volume":"71","author":"Kumar","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3872","DOI":"10.1109\/JIOT.2020.2974823","article-title":"A Vision of C-V2X: Technologies, Field Testing, and Challenges with Chinese De-velopment","volume":"7","author":"Chen","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"7970","DOI":"10.1109\/TVT.2019.2921352","article-title":"Joint Power Control and Resource Allocation Mode Selection for Safety-Related V2X Communication","volume":"68","author":"Li","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MVT.2017.2752798","article-title":"LTE-V for Sidelink 5G V2X Vehicular Communications: A New 5G Technology for Short-Range Vehicle-to-Everything Communications","volume":"12","author":"Gozalvez","year":"2017","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2185","DOI":"10.1109\/TITS.2018.2865173","article-title":"A Novel Low-Latency V2V Resource Allocation Scheme Based on Cellular V2X Communications","volume":"20","author":"Abbas","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3545","DOI":"10.1109\/JIOT.2020.2973267","article-title":"Resource Allocation for D2D-Based V2X Communication with Imperfect CSI","volume":"7","author":"Li","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8601","DOI":"10.1109\/TVT.2020.2997853","article-title":"Resource Allocation for Cellular V2X Networks Mode-3 with Underlay Approach in LTE-V Standard","volume":"69","author":"Aslani","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3547","DOI":"10.1109\/TITS.2020.3001682","article-title":"Efficient Power-Splitting and Resource Allocation for Cellular V2X Communications","volume":"22","author":"Jameel","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1109\/JPROC.2019.2954595","article-title":"Future Intelligent and Secure Vehicular Network toward 6G: Machine-Learning Approaches","volume":"108","author":"Tang","year":"2019","journal-title":"Proc. IEEE"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1251","DOI":"10.1109\/COMST.2020.2964534","article-title":"Machine Learning for Resource Management in Cellular and IoT Networks: Potentials, Current Solutions, and Open Challenges","volume":"22","author":"Hussain","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2027","DOI":"10.1109\/COMST.2021.3089688","article-title":"Comprehensive Survey on Machine Learning in Vehicular Network: Technology, Applications and Challenges","volume":"23","author":"Tang","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1109\/JPROC.2019.2957798","article-title":"Deep-Learning-Based Wireless Resource Allocation with Application to Vehicular Networks","volume":"108","author":"Liang","year":"2020","journal-title":"Proc. IEEE"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1828","DOI":"10.1109\/TVT.2019.2961405","article-title":"Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications","volume":"69","author":"Li","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2282","DOI":"10.1109\/JSAC.2019.2933962","article-title":"Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning","volume":"37","author":"Liang","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, Z., Wang, L., Ye, H., Li, G.Y., and Juang, B.-H.F. (2020, January 7\u201311). Resource Allocation based on Graph Neural Networks in Vehicular Communications. Proceedings of the GLOBECOM 2020\u20142020 IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322537"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8964","DOI":"10.1109\/TVT.2021.3098854","article-title":"Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications","volume":"70","author":"Yuan","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7279","DOI":"10.1109\/JIOT.2020.2982699","article-title":"DRL-Based Energy-Efficient Resource Allocation Frameworks for Uplink NOMA Systems","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gyawali, S., Qian, Y., and Hu, R.Q. (2019, January 9\u201313). Resource Allocation in Vehicular Communications Using Graph and Deep Reinforcement Learning. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA.","DOI":"10.1109\/GLOBECOM38437.2019.9013594"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2200","DOI":"10.1109\/JSAC.2019.2933762","article-title":"Joint Power Allocation and Channel Assignment for NOMA with Deep Reinforcement Learning","volume":"37","author":"He","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3163","DOI":"10.1109\/TVT.2019.2897134","article-title":"Deep Reinforcement Learning Based Resource Allocation for V2V Communications","volume":"68","author":"Ye","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6380","DOI":"10.1109\/JIOT.2019.2962715","article-title":"Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications","volume":"7","author":"Zhang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4157","DOI":"10.1109\/TVT.2018.2890686","article-title":"Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks","volume":"68","author":"Yang","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1109\/TCCN.2020.2983170","article-title":"A Reinforcement Learning Method for Joint Mode Selection and Power Adaptation in the V2V Communication Network in 5G","volume":"6","author":"Zhao","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_30","unstructured":"(2022, February 23). Technical Specification Group Radio Access Network; Study LTE-Based V2X Services; (Release 14), Document 3GPP TR 36.885 V14.0.0, 3rd Generation Partnership Project, June 2016, Available online: http:\/\/www.doc88.com\/p-67387023571695.html."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1972","DOI":"10.1109\/COMST.2021.3057017","article-title":"A Tutorial on 5G NR V2X Communications","volume":"23","author":"Garcia","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5199","DOI":"10.1109\/TWC.2021.3065996","article-title":"Learning-Based Robust Resource Allocation for Ultra-Reliable V2X Communications","volume":"20","author":"Wu","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_34","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double Q-learning. Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_37","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv, Available online: https:\/\/arxiv.org\/abs\/1609.04747."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1874\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:28:51Z","timestamp":1760135331000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1874"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,27]]},"references-count":37,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22051874"],"URL":"https:\/\/doi.org\/10.3390\/s22051874","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,27]]}}}