{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,11]],"date-time":"2026-10-11T15:43:50Z","timestamp":1791733430584,"version":"4.3.4"},"reference-count":30,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T00:00:00Z","timestamp":1782432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["RS-2021-NR060120"],"award-info":[{"award-number":["RS-2021-NR060120"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2120251015517"],"award-info":[{"award-number":["2120251015517"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["RS-2022-NR069161"],"award-info":[{"award-number":["RS-2022-NR069161"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"award":["RS-2021-NR060120"],"award-info":[{"award-number":["RS-2021-NR060120"]}],"id":[{"id":"https:\/\/ror.org\/013aysd81","id-type":"ROR","asserted-by":"publisher"}]},{"award":["2120251015517"],"award-info":[{"award-number":["2120251015517"]}],"id":[{"id":"https:\/\/ror.org\/013aysd81","id-type":"ROR","asserted-by":"publisher"}]},{"award":["RS-2022-NR069161"],"award-info":[{"award-number":["RS-2022-NR069161"]}],"id":[{"id":"https:\/\/ror.org\/013aysd81","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>6G promises ultra-low latency, high data throughput, and seamless global connectivity. However, providing uninterrupted connectivity in remote and underserved regions remains a critical challenge for Terrestrial Networks (TNs), where the cost of deploying infrastructure is difficult to justify against sparse user density. Standardized under 3GPP Release 17, Non-Terrestrial Networks (NTNs) have emerged as a viable solution to close this digital divide. Among NTN platforms, High-Altitude Platform Stations (HAPS) occupy a strategic middle ground, as they deliver lower propagation delays than Low-Earth Orbit (LEO) satellites while achieving far broader coverage than TN-based Base Stations (BS). Despite these advantages, battery-powered Internet of Things (IoT) devices communicating via HAPS face a fundamental energy efficiency (EE) challenge: transmit power must be carefully managed to maximize data throughput while preserving battery life and minimizing packet queuing delays. To address this, we propose a Q-learning-based Reinforcement Learning (RL) framework. The RL agent observes the instantaneous battery level and queue state of the IoT device, and dynamically selects optimal power levels from a discrete action space across successive time slots. Unlike traditional heuristic algorithms, such as Round Robin (RR), Max Single-to-Noise Ratio (Max-SNR), and fixed-power allocation, which rely on static rules or greedy channel-based decisions, the proposed Q-learning agent learns adaptive, long-term optimal policies through direct interaction with the environment, without requiring explicit mathematical modeling of the channel or traffic dynamics. Extensive simulations demonstrate that the proposed framework achieves up to 40% higher average EE compared to all benchmark schemes, maintains consistently lower power consumption, and exhibits superior statistical reliability as evidenced by a right-shifted Cumulative Distribution Function (CDF) of EE. These results demonstrate Q-learning as a promising candidate for scalable, energy-aware power control of next-generation HAPS-assisted IoT deployments in 6G NTN ecosystems.<\/jats:p>","DOI":"10.3390\/s26134057","type":"journal-article","created":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T08:43:57Z","timestamp":1782463437000},"page":"4057","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["An Intelligent Energy-Aware Framework for 6G-Enabled Non-Terrestrial IoT via Reinforcement Learning"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2133-5286","authenticated-orcid":false,"given":"Ali","family":"Nauman","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, College of Digital Convergence, Yeungnam University, Gyeongsan 38541, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8454-6980","authenticated-orcid":false,"given":"Sung Won","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, College of Digital Convergence, Yeungnam University, Gyeongsan 38541, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jamshed, M.A., Ayaz, F., Kaushik, A., Fischione, C., and Ur-Rehman, M. (2024). Green uav-enabled internet-of-things network with ai-assisted noma for disaster management. Proceedings of the 2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), IEEE.","DOI":"10.1109\/PIMRC59610.2024.10817434"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1561\/1300000072","article-title":"A tutorial on non-terrestrial networks: Towards global and ubiquitous 6G connectivity","volume":"14","author":"Jamshed","year":"2025","journal-title":"Found. Trends Netw."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Haq, B., Jamshed, M.A., and Nauman, A. (2024). Integrated Terrestrial and Non-Terrestrial Network: An Overview. Integrated Terrestrial and Non-Terrestrial Networks, Springer.","DOI":"10.1007\/978-3-031-63705-6"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MCOM.2016.7470933","article-title":"Wireless communications with unmanned aerial vehicles: Opportunities and challenges","volume":"54","author":"Zeng","year":"2016","journal-title":"IEEE Commun. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/OJCOMS.2026.3655070","article-title":"RIS-Enabled Joint Communications and Sensing in 6G NTNs: Opportunities and Challenges","volume":"7","author":"Khan","year":"2026","journal-title":"IEEE Open J. Commun. Soc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1109\/COMST.2021.3066905","article-title":"A vision and framework for the high altitude platform station (HAPS) networks of the future","volume":"23","author":"Kurt","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"157995","DOI":"10.1109\/ACCESS.2020.3019820","article-title":"System design of gigabit HAPS mobile communications","volume":"8","author":"Shibata","year":"2020","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5482","DOI":"10.1109\/JSEN.2022.3148128","article-title":"Challenges, applications, and future of wireless sensors in Internet of Things: A review","volume":"22","author":"Jamshed","year":"2022","journal-title":"IEEE Sens. J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nauman, A., Jamshed, M.A., Ahmad, Y., Ali, R., Zikria, Y.B., and Kim, S.W. (2019). An intelligent deterministic D2D communication in narrow-band Internet of Things. Proceedings of the 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), IEEE.","DOI":"10.1109\/IWCMC.2019.8766786"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Li, Z. (2024, January 6\u20138). Research on dynamic allocation of wireless resources optimized by reinforcement learning. Proceedings of the 2024 International Symposium on Artificial Intelligence for Education, Xi\u2019an, China.","DOI":"10.1145\/3700297.3700397"},{"key":"ref_11","unstructured":"Sinha, S. (2025). Number of Connected IoT Devices Growing 14% to 21.1 Billion, IoT Analytics."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jamshed, M.A. (2026). Artificial Intelligence for Integrated Terrestrial and Non-Terrestrial Networks, Springer. Synthesis Lectures on Communications.","DOI":"10.1007\/978-3-032-06512-4"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1109\/IOTM.001.2400204","article-title":"Artificial Intelligence, Ambient Backscatter Communication and Non-Terrestrial Networks: A 6G Commixture","volume":"8","author":"Jamshed","year":"2025","journal-title":"IEEE Internet Things Mag."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Plastras, S., Tsoumatidis, D., Skoutas, D.N., Rouskas, A., Kormentzas, G., and Skianis, C. (2024). Non-terrestrial networks for energy-efficient connectivity of remote iot devices in the 6g era: A survey. Sensors, 24.","DOI":"10.3390\/s24041227"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Worka, C.E., Khan, F.A., Ahmed, Q.Z., Sureephong, P., and Alade, T. (2024). Reconfigurable intelligent surface (RIS)-assisted non-terrestrial network (NTN)-based 6G communications: A contemporary survey. Sensors, 24.","DOI":"10.3390\/s24216958"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yang, D., Wu, J., and He, Y. (2024). Optimizing the agricultural internet of things (IoT) with edge computing and low-altitude platform stations. Sensors, 24.","DOI":"10.3390\/s24217094"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Takabatake, W., Shibata, Y., and Hoshino, K. (2023). Neural-network-based dynamic area optimization algorithm for high-altitude platform station. Proceedings of the 2023 IEEE 98th Vehicular Technology Conference (VTC2023-Fall), IEEE.","DOI":"10.1109\/VTC2023-Fall60731.2023.10333657"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"190304","DOI":"10.1007\/s11432-024-4511-4","article-title":"Game-theoretic clustering and scalable beamforming for multi-RIS-assisted cohesive satellite anti-jamming systems","volume":"68","author":"Cao","year":"2025","journal-title":"Sci. China Inf. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10846","DOI":"10.1038\/s41598-025-93251-7","article-title":"Synthesize multiple V\/H directional beams for high altitude platform station based on deep-learning algorithm","volume":"15","author":"Mahmoud","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1109\/MCOM.001.1900461","article-title":"Federated learning for wireless communications: Motivation, opportunities, and challenges","volume":"58","author":"Niknam","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2595","DOI":"10.1109\/COMST.2018.2846401","article-title":"Deep learning for intelligent wireless networks: A comprehensive survey","volume":"20","author":"Mao","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Al Janaby, A., Al-Rizzo, H., and Qassim, Y. (2026). Enhancing Spectral Efficiency of 6G Downlink Beamforming via Cooperative Multi-Agent Deep Reinforcement Learning. Sensors, 26.","DOI":"10.3390\/s26030950"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.comcom.2021.05.007","article-title":"Reinforcement learning-enabled intelligent device-to-device (I-D2D) communication in narrowband Internet of Things (NB-IoT)","volume":"176","author":"Nauman","year":"2021","journal-title":"Comput. Commun."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2555","DOI":"10.1109\/LCOMM.2020.3013125","article-title":"Lyapunov Drift-Plus-Penalty Optimization for Queues With Finite Capacity","volume":"24","author":"Bracciale","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jameel, F., Jamshed, M.A., Chang, Z., J\u00e4ntti, R., and Pervaiz, H. (2020). Low latency ambient backscatter communications with deep Q-learning for beyond 5G applications. Proceedings of the 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), IEEE.","DOI":"10.1109\/VTC2020-Spring48590.2020.9129364"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jamshed, M.A., Khan, W.U., Pervaiz, H., Imran, M.A., and Ur-Rehman, M. (2022). Emission-aware resource optimization framework for backscatter-enabled uplink noma networks. Proceedings of the 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring), IEEE.","DOI":"10.1109\/VTC2022-Spring54318.2022.9860409"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"11869","DOI":"10.1038\/s41598-023-31596-7","article-title":"Configuring reconfigurable intelligent surfaces using a practical codebook approach","volume":"13","author":"Hassouna","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Meng, R., Shah, A.A., Jamshed, M.A., and Pezaros, D. (2024). Federated learning-based intrusion detection framework for internet of things and edge computing backed critical infrastructure. Proceedings of the 2024 IEEE International Conference on Communications Workshops (ICC Workshops), IEEE.","DOI":"10.1109\/ICCWorkshops59551.2024.10615814"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bilal, A., Mohsin, M.A., Umer, M., Bangash, M.A.K., and Jamshed, M.A. (2025). Meta-thinking in llms via multi-agent reinforcement learning: A survey. arXiv.","DOI":"10.21203\/rs.3.rs-8994957\/v1"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/13\/4057\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T08:52:02Z","timestamp":1782463922000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/13\/4057"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,26]]},"references-count":30,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["s26134057"],"URL":"https:\/\/doi.org\/10.3390\/s26134057","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,26]]}}}