{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T16:19:00Z","timestamp":1778084340435,"version":"3.51.4"},"reference-count":43,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T00:00:00Z","timestamp":1734652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science Committee of the Ministry of Science and Higher Education of the Republic of Kazakhstan","award":["AP19678552"],"award-info":[{"award-number":["AP19678552"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JSAN"],"abstract":"<jats:p>Wireless communication technologies (WSN) are pivotal for the successful deployment of the Internet of Things (IoT). Among them, long-range (LoRa) and long-range wide-area network (LoRaWAN) technologies have been widely adopted due to their ability to provide long-distance communication, low energy consumption (EC), and cost-effectiveness. One of the critical issues in the implementation of wireless networks is the selection of optimal transmission parameters to minimize EC while maximizing the packet delivery ratio (PDR). This study introduces a reinforcement learning (RL) algorithm, Double Deep Q-Network with Prioritized Experience Replay (DDQN-PER), designed to optimize network transmission parameter selection, particularly the spreading factor (SF) and transmission power (TP). This research explores a variety of network scenarios, characterized by different device numbers and simulation times. The proposed approach demonstrates the best performance, achieving a 17.2% increase in the packet delivery ratio compared to the traditional Adaptive Data Rate (ADR) algorithm. The proposed DDQN-PER algorithm showed PDR improvement in the range of 6.2\u20138.11% compared to other existing RL and machine-learning-based works.<\/jats:p>","DOI":"10.3390\/jsan13060089","type":"journal-article","created":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T04:45:49Z","timestamp":1735015549000},"page":"89","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Enhanced Reinforcement Learning Algorithm Based-Transmission Parameter Selection for Optimization of Energy Consumption and Packet Delivery Ratio in LoRa Wireless Networks"],"prefix":"10.3390","volume":"13","author":[{"given":"Batyrbek","family":"Zholamanov","sequence":"first","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Askhat","family":"Bolatbek","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3442-8550","authenticated-orcid":false,"given":"Ahmet","family":"Saymbetov","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Madiyar","family":"Nurgaliyev","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Evan","family":"Yershov","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Kymbat","family":"Kopbay","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Sayat","family":"Orynbassar","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3935-7213","authenticated-orcid":false,"given":"Gulbakhar","family":"Dosymbetova","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Ainur","family":"Kapparova","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Nurzhigit","family":"Kuttybay","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]},{"given":"Nursultan","family":"Koshkarbay","sequence":"additional","affiliation":[{"name":"Faculty of Physics and Technology, Al-Farabi Kazakh National University, 71 Al-Farabi, Almaty 050040, Kazakhstan"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mroue, M., Ramadan, A., Nasser, A., and Zaki, C. (2024, January 6\u20137). LPWAN Technologies in Smart Cities: A Comparative Analysis of LoRa, Sigfox, and LTE-M. Proceedings of the Eighth International Conference on Information System Design and Intelligent Applications, Dubai, United Arab Emirates.","DOI":"10.1007\/978-981-97-4895-2_18"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"145852","DOI":"10.1109\/ACCESS.2021.3123162","article-title":"Integrated Framework for Development, Emulation, and Testing of High-Level Converter Control Functions for Distributed Generation Sources","volume":"9","author":"Porobic","year":"2021","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e5966","DOI":"10.1002\/dac.5136","article-title":"ND-ADR. Nondestructive Adaptive Data Rate for LoRaWAN Internet of Things","volume":"35","author":"Lodhi","year":"2022","journal-title":"Int. J. Commun. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kufakunesu, R., Hancke, G., and Abu-Mahfouz, A. (2022). A Fuzzy-Logic Based Adaptive Data Rate Scheme for Energy-Efficient LoRaWAN Communication. J. Sens. Actuator Netw., 11.","DOI":"10.3390\/jsan11040065"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ksiazek, K., and Grochla, K. (July, January 28). Flexibility Analysis of Adaptive Data Rate Algorithm in LoRa Networks. Proceedings of the 2021 International Wireless Communications and Mobile Computing (IWCMC), Harbin City, China.","DOI":"10.1109\/IWCMC51323.2021.9498664"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10501","DOI":"10.1109\/TII.2023.3240696","article-title":"LoRa Performance in Industrial Environments: Analysis of Different ADR Algorithms","volume":"19","author":"Arismendy","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2545","DOI":"10.1109\/ACCESS.2023.3234188","article-title":"Delay Optimization in LoRaWAN by Employing Adaptive Scheduling Algorithm with Unsupervised Learning","volume":"11","author":"Ali","year":"2023","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Farhad, A., and Pyun, J.-Y. (2023). LoRaWAN Meets ML. A Survey on Enhancing Performance with Machine Learning. Sensors, 23.","DOI":"10.3390\/s23156851"},{"key":"ref_9","unstructured":"Jiang, Y., Wang, M., and Wang, X. (2023, January 6\u20138). A Efficient Adaptive Data Rate Algorithm in LoRaWAN Networks: K-ADR. Proceedings of the 24st Asia-Pacific Network Operations and Management Symposium (APNOMS), Sejong, Republic of Korea."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"11873","DOI":"10.1109\/JIOT.2020.3005881","article-title":"EARN: Enhanced ADR with coding rate adaptation in LoRaWAN","volume":"7","author":"Park","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"102388","DOI":"10.1016\/j.simpat.2021.102388","article-title":"Extending ADR mechanism for LoRa enabled mobile end-devices","volume":"113","author":"Moysiadis","year":"2021","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1841","DOI":"10.1109\/COMST.2023.3274934","article-title":"A survey on scalable LoRaWAN for massive IoT: Recent advances, potentials, and challenges","volume":"25","author":"Jouhari","year":"2023","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hakim, G., Braun, R., and Lipman, J. (2024). Adapted Diffusion for Energy-Efficient Routing in Wireless Sensor Networks. Electronics, 13.","DOI":"10.3390\/electronics13112072"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103858","DOI":"10.1016\/j.jnca.2024.103858","article-title":"AI-enhanced blockchain technology: A review of advancements and opportunities","volume":"225","author":"Ressi","year":"2024","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Perkovi\u0107, T., Duji\u0107 Rodi\u0107, L., \u0160abi\u0107, J., and \u0160oli\u0107, P. (2023). Machine Learning Approach towards LoRaWAN Indoor Localization. Electronics, 12.","DOI":"10.3390\/electronics12020457"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Nurgaliyev, M., Bolatbek, A., Zholamanov, B., Saymbetov, A., Kopbay, K., Yershov, E., Orynbassar, S., Dosymbetova, G., Kapparova, A., and Kuttybay, N. (2024). Machine Learning Based Localization of LoRa Mobile Wireless Nodes Using a Novel Sectorization Method. Future Internet, 16.","DOI":"10.3390\/fi16120450"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cuomo, F., Garlisi, D., Martino, A., and Martino, A. (2020). Predicting LoRaWAN Behavior: How Machine Learning Can Help. Computers, 9.","DOI":"10.3390\/computers9030060"},{"key":"ref_18","unstructured":"Carvalho Filho, M.N., and Campista, M.E.M. (December, January 30). Intelligent Configuration of PHY-Layer Parameters to Reduce Energy Consumption in LoRa. Proceedings of the IEEE Latin-American Conference on Communications (LATINCOM), Rio de Janeiro, Brazil."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100","DOI":"10.4018\/IJWNBT.2021070106","article-title":"A comparative study of machine learning models for spreading factor selection in LoRa networks","volume":"10","author":"Bouras","year":"2021","journal-title":"Int. J. Wirel. Netw. Broadband Technol. (IJWNBT)"},{"key":"ref_20","first-page":"4039","article-title":"Efficient collision control and auction-based resource allocation mechanism in dense LoRaWAN network via TCP using DRL technique","volume":"16","author":"Parween","year":"2024","journal-title":"Int. J. Inf. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"191495","DOI":"10.1109\/ACCESS.2020.3031974","article-title":"Unsupervised learning clustering and dynamic transmission scheduling for efficient dense LoRaWAN networks","volume":"8","author":"Alenezi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Asad Ullah, M., Iqbal, J., Hoeller, A., Souza, R.D., and Alves, H. (2019). K-Means Spreading Factor Allocation for Large-Scale LoRa Networks. Sensors, 19.","DOI":"10.3390\/s19214723"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"10092","DOI":"10.1109\/ACCESS.2023.3240308","article-title":"Intelligent resource allocation in LoRaWAN using machine learning techniques","volume":"11","author":"Minhaj","year":"2023","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, M., Mokdad, L., Charmois, C., Ben-Othman, J., and Fourneau, J.M. (2022, January 16\u201320). An mdp model-based initial strategy prediction method for lorawan. Proceedings of the ICC 2022-IEEE International Conference on Communications, Seoul, Republic of Korea.","DOI":"10.1109\/ICC45855.2022.9838364"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Teymuri, B., Serati, R., Anagnostopoulos, N.A., and Rasti, M. (2023). LP-MAB: Improving the Energy Efficiency of LoRaWAN Using a Reinforcement-Learning-Based Adaptive Configuration Algorithm. Sensors, 23.","DOI":"10.3390\/s23042363"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Fedullo, T., Morato, A., Tramarin, F., Ferrari, P., and Sisinni, E. (2022, January 7\u20139). Smart measurement systems exploiting adaptive LoRaWAN under power consumption constraints: A RL approach. Proceedings of the 2022 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4.0&IoT), Trento, Italy.","DOI":"10.1109\/MetroInd4.0IoT54413.2022.9831487"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"11640","DOI":"10.1109\/TII.2023.3248074","article-title":"AI-ERA: Artificial intelligence-empowered resource allocation for LoRa-enabled IoT applications","volume":"19","author":"Farhad","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_28","unstructured":"Sornin, N., Luis, M., Eirich, T., Kramp, T., and Hersent, O. (2015). LoRaWAN 1.0 Specification, LoRa Alliance, Inc."},{"key":"ref_29","unstructured":"Seller, O., and Sornin, N. (2016). Low Power Long Range Transmitter. (20140219329), U.S. Patent."},{"key":"ref_30","unstructured":"(2018). LoRaWAN 1.0.3 Regional Parameters, LoRa Alliance."},{"key":"ref_31","unstructured":"Libelium Comunicaciones Distribuidas S.L. (2015). Waspmote LoRa 868 MHz 915 MHz SX1272 Networking Guide, Libelium Comunicaciones Distribuidas S.L.. V4(2)."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Babaki, J., Rasti, M., and Taskou, S.K. (2020, January 4\u20136). Improved configuration of transmission variables for LoRaWAN in high-noise channels. Proceedings of the 2020 28th Iranian Conference on Electrical Engineering (ICEE), Tabriz, Iran.","DOI":"10.1109\/ICEE50131.2020.9260713"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Peruzzo, A., and Vangelista, L. (2018, January 25\u201328). A power efficient adaptive data rate algorithm for LoRaWAN networks. Proceedings of the 2018 21st International Symposium on Wireless Personal Multimedia Communications (WPMC), Chiang Rai, Thailand.","DOI":"10.1109\/WPMC.2018.8713092"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Slabicki, M., Premsankar, G., and Di Francesco, M. (2018, January 23\u201327). Adaptive configuration of LoRa networks for dense IoT deployments. Proceedings of the NOMS 2018-2018 IEEE\/IFIP Network Operations and Management Symposium, Taipei, Taiwan.","DOI":"10.1109\/NOMS.2018.8406255"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_36","unstructured":"Fan, J., Wang, Z., Xie, Y., and Yang, Z. (2020). A theoretical analysis of deep Q-learning. arXiv."},{"key":"ref_37","first-page":"1","article-title":"Deep reinforcement learning with double q-learning","volume":"30","author":"Guez","year":"2016","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_38","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized experience replay. arXiv, arXiv:1511.05952."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"101112","DOI":"10.1016\/j.iot.2024.101112","article-title":"Intelligent service placement algorithm based on DDQN and prioritized experience replay in IoT-Fog computing environment","volume":"25","author":"Sharma","year":"2024","journal-title":"Internet Things"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Carvalho, R., Al-Tam, F., and Correia, N. (2021, January 27\u201328). Q-learning adr agent for lorawan optimization. Proceedings of the 2021 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), Bandung, Indonesia.","DOI":"10.1109\/IAICT52856.2021.9532518"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Navas, R.E., Dandachi, G., Hadjadj-Aoul, Y., and Maill\u00e9, P. (2023, January 12\u201315). Energy-Aware Spreading Factor Selection in LoRaWAN Using Delayed-Feedback Bandits. Proceedings of the 2023 IFIP Networking Conference (IFIP Networking), Barcelona, Spain.","DOI":"10.23919\/IFIPNetworking57963.2023.10186444"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Farhad, A., Kim, D.H., Yoon, J.S., and Pyun, J.Y. (2022, January 6\u20139). Deep learning-based channel adaptive resource allocation in LoRaWAN. Proceedings of the 2022 International Conference on Electronics, Information, and Communication (ICEIC), Jeju, Republic of Korea.","DOI":"10.1109\/ICEIC54506.2022.9748580"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kufakunesu, R., Hancke, G.P., and Abu-Mahfouz, A.M. (2024). Collision Avoidance Adaptive Data Rate Algorithm for LoRaWAN. Future Internet, 16.","DOI":"10.3390\/fi16100380"}],"container-title":["Journal of Sensor and Actuator Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2224-2708\/13\/6\/89\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:56:52Z","timestamp":1760115412000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2224-2708\/13\/6\/89"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,20]]},"references-count":43,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["jsan13060089"],"URL":"https:\/\/doi.org\/10.3390\/jsan13060089","relation":{},"ISSN":["2224-2708"],"issn-type":[{"value":"2224-2708","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,20]]}}}