{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:33:33Z","timestamp":1783096413447,"version":"3.54.6"},"reference-count":29,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,4,28]],"date-time":"2024-04-28T00:00:00Z","timestamp":1714262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Efficient spectrum sharing is essential for maximizing data communication performance in Vehicular Networks (VNs). In this article, we propose a novel hybrid framework that leverages Multi-Agent Reinforcement Learning (MARL), thereby combining both centralized and decentralized learning approaches. This framework addresses scenarios where multiple vehicle-to-vehicle (V2V) links reuse the frequency spectrum preoccupied by vehicle-to-infrastructure (V2I) links. We introduce the QMIX technique with the Deep Q Networks (DQNs) algorithm to facilitate collaborative learning and efficient spectrum management. The DQN technique uses a neural network to approximate the Q value function in high-dimensional state spaces, thus mapping input states to (action, Q value) tables that facilitate self-learning across diverse scenarios. Similarly, the QMIX is a value-based technique for multi-agent environments. In the proposed model, each V2V agent having its own DQN observes the environment, receives observation, and obtains a common reward. The QMIX network receives Q values from all agents considering individual benefits and collective objectives. This mechanism leads to collective learning while V2V agents dynamically adapt to real-time conditions, thus improving VNs performance. Our research finding highlights the potential of hybrid MARL models for dynamic spectrum sharing in VNs and paves the way for advanced cooperative learning strategies in vehicular communication environments. Furthermore, we conducted an in-depth exploration of the simulation environment and performance evaluation criteria, thus concluding in a comprehensive comparative analysis with cutting-edge solutions in the field. Simulation results show that the proposed framework efficiently performs against the benchmark architecture in terms of V2V transmission probability and V2I peak data transfer.<\/jats:p>","DOI":"10.3390\/fi16050152","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T04:26:16Z","timestamp":1714364776000},"page":"152","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["A Hybrid Multi-Agent Reinforcement Learning Approach for Spectrum Sharing in Vehicular Networks"],"prefix":"10.3390","volume":"16","author":[{"given":"Mansoor","family":"Jamal","sequence":"first","affiliation":[{"name":"Department of Electronics, Quaid-i-Azam University, Islamabad 44000, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2200-4868","authenticated-orcid":false,"given":"Zaib","family":"Ullah","sequence":"additional","affiliation":[{"name":"Artificial Intelligence and Robotics Lab, Universit\u00e0 Telematica Giustino Fortunato, 82100 Benevento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0815-4883","authenticated-orcid":false,"given":"Muddasar","family":"Naeem","sequence":"additional","affiliation":[{"name":"Artificial Intelligence and Robotics Lab, Universit\u00e0 Telematica Giustino Fortunato, 82100 Benevento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Musarat","family":"Abbas","sequence":"additional","affiliation":[{"name":"Department of Electronics, Quaid-i-Azam University, Islamabad 44000, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8177-032X","authenticated-orcid":false,"given":"Antonio","family":"Coronato","sequence":"additional","affiliation":[{"name":"Artificial Intelligence and Robotics Lab, Universit\u00e0 Telematica Giustino Fortunato, 82100 Benevento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"86","DOI":"10.2174\/1874447802014010086","article-title":"Vehicle to vehicle \u201cV2V\u201d communication: Scope, importance, challenges, research directions and future","volume":"14","author":"Yasser","year":"2020","journal-title":"Open Transp. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"10647","DOI":"10.1109\/TVT.2017.2750903","article-title":"Vehicular Communications: A Physical Layer Perspective","volume":"66","author":"Liang","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1007\/s11277-019-06222-3","article-title":"A near optimal scheduling algorithm for efficient radio resource management in multi-user MIMO systems","volume":"106","author":"Naeem","year":"2019","journal-title":"Wirel. Pers. Commun."},{"key":"ref_4","first-page":"100638","article-title":"Revolutionizing Intelligent Transportation Systems with Cellular Vehicle-to-Everything (C-V2X) technology: Current trends, use cases, emerging technologies, standardization bodies, industry analytics and future directions","volume":"43","author":"Rammohan","year":"2023","journal-title":"Veh. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Naeem, M., Coronato, A., Ullah, Z., Bashir, S., and Paragliola, G. (2022). Optimal user scheduling in multi antenna system using multi agent reinforcement learning. Sensors, 22.","DOI":"10.3390\/s22218278"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11298","DOI":"10.1109\/JIOT.2023.3329893","article-title":"Deep Reinforcement Learning based Joint Spectrum Allocation and Configuration Design for STAR-RIS-Assisted V2X Communications","volume":"11","author":"Aung","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Naeem, M., Bashir, S., Khan, M.U., and Syed, A.A. (2016, January 12\u201316). Performance comparison of scheduling algorithms for MU-MIMO systems. Proceedings of the 2016 13th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan.","DOI":"10.1109\/IBCAST.2016.7429939"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/MCOM.001.2000526","article-title":"3GPP New Radio Release 16: Evolution of 5G for Industrial Internet of Things","volume":"59","author":"Baek","year":"2021","journal-title":"IEEE Commun. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MWC.2016.7498068","article-title":"Support for vehicle-to-everything services based on LTE","volume":"23","author":"Sun","year":"2016","journal-title":"IEEE Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"10397","DOI":"10.1109\/JIOT.2021.3121796","article-title":"Task Offloading and Resource Allocation for IoV Using 5G NR-V2X Communication","volume":"9","author":"Raza","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_11","first-page":"7234","article-title":"Monotonic value function factorisation for deep multi-agent reinforcement learning","volume":"21","author":"Rashid","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3547","DOI":"10.1109\/TITS.2020.3001682","article-title":"Efficient Power-Splitting and Resource Allocation for Cellular V2X Communications","volume":"22","author":"Jameel","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3545","DOI":"10.1109\/JIOT.2020.2973267","article-title":"Resource Allocation for D2D-Based V2X Communication with Imperfect CSI","volume":"7","author":"Li","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gao, J., Khandaker, M.R.A., Tariq, F., Wong, K.K., and Khan, R.T. (2019, January 22\u201325). Deep Neural Network Based Resource Allocation for V2X Communications. Proceedings of the 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall), Honolulu, HI, USA.","DOI":"10.1109\/VTCFall.2019.8891446"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2185","DOI":"10.1109\/TITS.2018.2865173","article-title":"A Novel Low-Latency V2V Resource Allocation Scheme Based on Cellular V2X Communications","volume":"20","author":"Abbas","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6380","DOI":"10.1109\/JIOT.2019.2962715","article-title":"Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications","volume":"7","author":"Zhang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gui, J., Lin, L., Deng, X., and Cai, L. (2024). Spectrum-Energy-Efficient Mode Selection and Resource Allocation for Heterogeneous V2X Networks: A Federated Multi-Agent Deep Reinforcement Learning Approach. IEEE\/ACM Trans. Netw.","DOI":"10.1109\/TNET.2024.3364161"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"8964","DOI":"10.1109\/TVT.2021.3098854","article-title":"Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications","volume":"70","author":"Yuan","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_19","first-page":"100532","article-title":"Dynamic mode selection and resource allocation approach for 5G-vehicle-to-everything (V2X) communication using asynchronous federated deep reinforcement learning method","volume":"38","author":"Rasheed","year":"2022","journal-title":"Veh. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, J., Zhao, J., and Sun, X. (2021, January 20\u201322). Deep Reinforcement Learning Based Wireless Resource Allocation for V2X Communications. Proceedings of the 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP), Changsha, China.","DOI":"10.1109\/WCSP52459.2021.9613367"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3215","DOI":"10.1007\/s10462-020-09938-y","article-title":"A survey on multi-agent deep reinforcement learning: From the perspective of challenges and applications","volume":"54","author":"Du","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"ref_22","unstructured":"Roderick, M., MacGlashan, J., and Tellex, S. (2017). Implementing the deep q-network. arXiv."},{"key":"ref_23","unstructured":"Luu, Q.T. (2024, February 19). Q-Learning vs. Deep Q-Learning. Available online: https:\/\/www.baeldung.com\/cs\/q-learning-vs-deep-q-learning-vs-deep-q-network."},{"key":"ref_24","first-page":"10199","article-title":"Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning","volume":"33","author":"Rashid","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_25","unstructured":"Yang, Y., Hao, J., Liao, B., Shao, K., Chen, G., Liu, W., and Tang, H. (2020). Qatten: A general framework for cooperative multiagent reinforcement learning. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3115","DOI":"10.1007\/s00371-021-02269-1","article-title":"Multi-agent reinforcement learning for character control","volume":"37","author":"Li","year":"2021","journal-title":"Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, X., Xiong, G., Lv, Y., Chen, Y., Song, B., and Wang, F.Y. (2021, January 19\u201322). A Collaborative Communication-Qmix Approach for Large-scale Networked Traffic Signal Control. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9564683"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/TNNLS.2021.3089493","article-title":"Smix (\u03bb): Enhancing centralized value functions for cooperative multiagent reinforcement learning","volume":"34","author":"Yao","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","unstructured":"(2024, January 24). 3GPP. 3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Study on LTE-based V2X Services; (Release 14). 3GPP TS 36.300 V14.0.0. Available online: https:\/\/www.etsi.org\/deliver\/etsi_ts\/136300_136399\/136361\/14.00.00_60\/ts_136361v140000p.pdf."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/5\/152\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:35:17Z","timestamp":1760106917000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/5\/152"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,28]]},"references-count":29,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["fi16050152"],"URL":"https:\/\/doi.org\/10.3390\/fi16050152","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,28]]}}}