{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T16:23:23Z","timestamp":1775838203913,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T00:00:00Z","timestamp":1730851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Mobile communications have experienced exponential growth both in connectivity and multimedia traffic in recent years. To support this tremendous growth, device-to-device (D2D) communications play a significant role in 5G and beyond 5G networks. However, enabling D2D communications in an underlay, heterogeneous cellular network poses two major challenges. First, interference management between D2D and cellular users directly affects a system\u2019s performance. Second, achieving an acceptable level of link quality for both D2D and cellular networks is necessary. An optimum resource allocation is required to mitigate the interference and improve a system\u2019s performance. In this paper, we provide a solution to interference management with an acceptable quality of services (QoS). To this end, we propose a machine learning-based resource allocation method to maximize throughput and achieve minimum QoS requirements for all active D2D pairs and cellular users. We first solve a resource optimization problem by allocating spectrum resources and controlling power transmission on demand. As resource optimization is an integer nonlinear programming problem, we address this problem by proposing a deep Q-network-based reinforcement learning algorithm (DRL) to optimize the resource allocation issue. The proposed DRL algorithm is trained with a decision-making policy to obtain the best solution in terms of spectrum efficiency, computational time, and throughput. The system performance is validated by simulation. The results show that the proposed method outperforms the existing ones.<\/jats:p>","DOI":"10.3390\/fi16110408","type":"journal-article","created":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T04:03:24Z","timestamp":1730865804000},"page":"408","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Machine Learning-Based Resource Allocation Algorithm to Mitigate Interference in D2D-Enabled Cellular Networks"],"prefix":"10.3390","volume":"16","author":[{"given":"Md","family":"Kamruzzaman","sequence":"first","affiliation":[{"name":"Computer Science and Software Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2770-8319","authenticated-orcid":false,"given":"Nurul I.","family":"Sarkar","sequence":"additional","affiliation":[{"name":"Computer Science and Software Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2103-8636","authenticated-orcid":false,"given":"Jairo","family":"Gutierrez","sequence":"additional","affiliation":[{"name":"Computer Science and Software Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kamruzzaman, M., Sarkar, N.I., and Gutierrez, J. (2022). A dynamic algorithm for interference management in D2D-enabled heterogeneous cellular networks: Modeling and analysis. Sensors, 22.","DOI":"10.3390\/s22031063"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s12083-020-00962-x","article-title":"A review on resource allocation techniques in D2D communication for 5G and B5G technology","volume":"14","author":"Jayakumar","year":"2021","journal-title":"Peer-to-Peer Netw. Appl."},{"key":"ref_3","first-page":"2022","article-title":"Cisco visual networking index: Global mobile data traffic forecast update, 2017\u20132022","volume":"2017","author":"Forecast","year":"2019","journal-title":"Update"},{"key":"ref_4","unstructured":"Ericsson (2022). Ericsson Mobility Report: Global 5G Growth Amid Macroeconomic Challenges, Ericsson."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1109\/MCOM.2014.6815897","article-title":"Device-to-device communication in 5G cellular networks: Challenges, solutions, and future directions","volume":"52","author":"Tehrani","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","unstructured":"ETSI (2020). 5G: Study on Scenarios and Requirements for Next Generation Access Technologies, 3GPP TR 38.913 Version 16.0.0 Release 16, ETSI."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kamruzzaman, M., Sarkar, N.I., Gutierrez, J., and Ray, S.K. (2019, January 9\u201311). A mode selection algorithm for mitigating interference in D2D enabled next-generation heterogeneous cellular networks. Proceedings of the 2019 International Conference on Information Networking (ICOIN), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICOIN.2019.8718182"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/MCOM.2014.6807945","article-title":"An overview of 3GPP device-to-device proximity services","volume":"52","author":"Lin","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3066","DOI":"10.1109\/JIOT.2020.3023111","article-title":"Deep multiagent reinforcement-learning-based resource allocation for internet of controllable things","volume":"8","author":"Gu","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cai, X., Zheng, J., and Zhang, Y. (2015, January 8\u201312). A graph-coloring based resource allocation algorithm for D2D communication in cellular networks. Proceedings of the 2015 IEEE International Conference on Communications (ICC), London, UK.","DOI":"10.1109\/ICC.2015.7249187"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kuruvatti, N.P., Hernandez, R., and Schotten, H.D. (2019, January 25\u201327). Interference Aware Power Management in D2D Underlay Cellular Networks. Proceedings of the 2019 IEEE AFRICON, Accra, Ghana.","DOI":"10.1109\/AFRICON46755.2019.9133851"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.pmcj.2018.10.005","article-title":"Interference management for D2D communications in heterogeneous cellular networks","volume":"51","author":"Xu","year":"2018","journal-title":"Pervasive Mob. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rezazadeh, F., Chergui, H., Christofi, L., and Verikoukis, C. (2021, January 14\u201323). Actor-critic-based learning for zero-touch joint resource and energy control in network slicing. Proceedings of the ICC 2021-IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500265"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Luo, Y., Shi, Z., Zhou, X., Liu, Q., and Yi, Q. (2014, January 19\u201321). Dynamic resource allocations based on Q-learning for D2D communication in cellular networks. Proceedings of the 2014 11th International Computer Conference on Wavelet Actiev Media Technology and Information Processing (ICCWAMTIP), Chengdu, China.","DOI":"10.1109\/ICCWAMTIP.2014.7073432"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zia, K., Javed, N., Sial, M.N., Ahmed, S., Pirzada, A.A., and Pervez, F. (2022). Distributed Multi-Agent RL-Based Autonomous Spectrum Allocation in D2D-Enabled Multi-Tier HetNets. Interference Mitigation in Device-to-Device Communications, John Wiley & Sons.","DOI":"10.1002\/9781119788829.ch6"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"AlQerm, I., and Shihada, B. (2016, January 22\u201327). A cooperative online learning scheme for resource allocation in 5G systems. Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICC.2016.7511617"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhu, L., Liu, C., Yuan, J., and Yu, G. (December, January 18). Machine learning-based resource optimization for d2d communication underlaying networks. Proceedings of the 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), Virtual.","DOI":"10.1109\/VTC2020-Fall49728.2020.9348830"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4163","DOI":"10.1109\/TWC.2018.2821151","article-title":"Resource allocation for device-to-device communications underlaying heterogeneous cellular networks using coalitional games","volume":"17","author":"Chen","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Saied, A., Okaf, A., and Qiu, D. (November, January 31). An efficient resource allocation for d2d communications underlaying in hetnets. Proceedings of the 2021 International Symposium on Networks, Computers and Communications (ISNCC), Dubai, United Arab Emirates.","DOI":"10.1109\/ISNCC52172.2021.9615658"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Fan, Z., Gu, X., Nie, S., and Chen, M. (2017, January 13\u201316). D2D power control based on supervised and unsupervised learning. Proceedings of the 2017 3rd IEEE International Conference on Computer and Communications (ICCC), Chengdu, China.","DOI":"10.1109\/CompComm.2017.8322607"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1109\/LCOMM.2020.3042490","article-title":"Resource allocation scheme for guarantee of QoS in D2D communications using deep neural network","volume":"25","author":"Lee","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1109\/TVT.2021.3130159","article-title":"Double deep q-network based distributed resource matching algorithm for d2d communication","volume":"71","author":"Yuan","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kai, C., Meng, X., Mei, L., and Huang, W. (2021, January 28\u201330). Deep reinforcement learning based user association and resource allocation for d2d-enabled wireless networks. Proceedings of the 2021 IEEE\/CIC International Conference on Communications in China (ICCC), Xiamen, China.","DOI":"10.1109\/ICCC52777.2021.9580261"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1109\/LCOMM.2021.3079920","article-title":"Deep reinforcement learning-based dynamic spectrum access for D2D communication underlay cellular networks","volume":"25","author":"Huang","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4426","DOI":"10.1109\/TMC.2021.3085206","article-title":"Make smart decisions faster: Deciding d2d resource allocation via stackelberg game guided multi-agent deep reinforcement learning","volume":"21","author":"Shi","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5235","DOI":"10.1109\/TWC.2021.3138733","article-title":"Deep learning-based resource allocation for device-to-device communication","volume":"21","author":"Lee","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"101262","DOI":"10.1016\/j.phycom.2020.101262","article-title":"Joint resource allocation and power control for D2D communication with deep reinforcement learning in MCC","volume":"45","author":"Wang","year":"2021","journal-title":"Phys. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1109\/TWC.2020.3032991","article-title":"Deep reinforcement learning for joint channel selection and power control in D2D networks","volume":"20","author":"Tan","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"140439","DOI":"10.1109\/ACCESS.2024.3467393","article-title":"Deep Reinforcement Learning-based Optimization Method for D2D Communication Energy Efficiency in Heterogeneous Cellular Networks","volume":"12","author":"Pan","year":"2024","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Guo, J., and Chen, J. (2023, January 15\u201317). Hybrid Action Space D2D Resource Allocation Algorithm Based on Multi-Agent Reinforcement Learning. Proceedings of the 2023 3rd International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), Wuhan, China.","DOI":"10.1109\/CEI60616.2023.10527865"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"140270","DOI":"10.1109\/ACCESS.2023.3341585","article-title":"DRL-Based Resource Allocation for NOMA-Enabled D2D Communications Underlay Cellular Networks","volume":"11","author":"Jeong","year":"2023","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"16426","DOI":"10.1109\/TVT.2023.3290258","article-title":"Cooperative Distributed Resource Allocation in Heterogeneous Networks with D2D Communication","volume":"72","author":"Cai","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9330","DOI":"10.1109\/TVT.2016.2519456","article-title":"On the performance of device-to-device communications with delay constraint","volume":"65","author":"Xu","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"137184","DOI":"10.1109\/ACCESS.2019.2942390","article-title":"Machine learning for 5G\/B5G mobile and wireless communications: Potential, limitations, and future directions","volume":"7","author":"Lee","year":"2019","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1007\/s13042-020-01178-4","article-title":"A survey of 5G network systems: Challenges and machine learning approaches","volume":"12","author":"Fourati","year":"2021","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"37689","DOI":"10.1109\/ACCESS.2022.3161511","article-title":"Machine learning based load balancing algorithms in future heterogeneous networks: A survey","volume":"10","author":"Gures","year":"2022","journal-title":"IEEE Access"},{"key":"ref_37","unstructured":"El Amine, A. (2019). Radio Resource Allocation in 5G Cellular Networks Powered by the Smart Grid and Renewable Energies. [Ph.D. Thesis, Ecole Nationale sup\u00e9rieure Mines-T\u00e9l\u00e9com Atlantique Bretagne Pays de la Loire]."},{"key":"ref_38","unstructured":"Cayamcela, M.E.M., and Lim, W. (2018, January 17\u201319). Artificial intelligence in 5G technology: A survey. Proceedings of the 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sun, H., Chen, X., Shi, Q., Hong, M., Fu, X., and Sidiropoulos, N.D. (2017, January 3\u20136). Learning to optimize: Training deep neural networks for wireless resource management. Proceedings of the 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Sapporo, Japan.","DOI":"10.1109\/SPAWC.2017.8227766"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Song, Y., Khandaker, M.R., Tariq, F., Wong, K.K., and Toding, A. (2021, January 25\u201328). Truly intelligent reflecting surface-aided secure communication using deep learning. Proceedings of the 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Virtual Event.","DOI":"10.1109\/VTC2021-Spring51267.2021.9448826"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5438","DOI":"10.1109\/TSP.2018.2866382","article-title":"Learning to optimize: Training deep neural networks for interference management","volume":"66","author":"Sun","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/MCOM.001.1900199","article-title":"Intelligent resource allocation in wireless communications systems","volume":"58","author":"Lee","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1109\/LCOMM.2018.2825444","article-title":"Deep power control: Transmit power control scheme based on convolutional neural network","volume":"22","author":"Lee","year":"2018","journal-title":"IEEE Commun. Lett."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1109\/TCDS.2017.2649564","article-title":"A basal ganglia network centric reinforcement learning model and its application in unmanned aerial vehicle","volume":"10","author":"Zeng","year":"2017","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Saied, A., Qiu, D., and Swessi, M. (2020, January 20\u201322). Resource management based on reinforcement learning for D2D communication in cellular networks. Proceedings of the 2020 International Symposium on Networks, Computers and Communications (ISNCC), Montreal, QC, Canada.","DOI":"10.1109\/ISNCC49221.2020.9297353"},{"key":"ref_46","unstructured":"Hausknecht, M., and Stone, P. (2015, January 12\u201314). Deep recurrent q-learning for partially observable mdps. Proceedings of the 2015 AAAI Fall Symposium Series, Arlington, VA, USA."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1007\/s11276-022-03176-6","article-title":"Multi-agent reinforcement learning based joint uplink\u2013downlink subcarrier assignment and power allocation for D2D underlay networks","volume":"29","author":"Kai","year":"2023","journal-title":"Wirel. Netw."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/11\/408\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:27:06Z","timestamp":1760113626000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/11\/408"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,6]]},"references-count":47,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["fi16110408"],"URL":"https:\/\/doi.org\/10.3390\/fi16110408","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,6]]}}}