{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T09:00:46Z","timestamp":1773910846488,"version":"3.50.1"},"reference-count":14,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T00:00:00Z","timestamp":1647648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Thousands of energy-aware sensors have been placed for monitoring in a variety of scenarios, such as manufacturing, control systems, disaster management, flood control and so on, requiring time-critical energy-efficient solutions to extend their lifetime. This paper proposes reinforcement learning (RL) based dynamic data streams for time-critical IoT systems in energy-aware IoT devices. The designed solution employs the Q-Learning algorithm. The proposed mechanism has the potential to adjust the data transport rate based on the amount of renewable energy resources that are available, to ensure collecting reliable data while also taking into account the sensor battery lifetime. The solution was evaluated using historical data for solar radiation levels, which shows that the proposed solution can increase the amount of transmitted data up to 23%, ensuring the continuous operation of the device.<\/jats:p>","DOI":"10.3390\/s22062375","type":"journal-article","created":{"date-parts":[[2022,3,20]],"date-time":"2022-03-20T21:37:17Z","timestamp":1647812237000},"page":"2375","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Dynamic Data Streams for Time-Critical IoT Systems in Energy-Aware IoT Devices Using Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1581-6016","authenticated-orcid":false,"given":"Fawzy","family":"Habeeb","sequence":"first","affiliation":[{"name":"School of Computing, Newcastle University, Newcastle upon Tyne NE1 7RU, UK"},{"name":"Alkamil College of Computer Science, University of Jeddah, Jeddah 21959, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomasz","family":"Szydlo","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, AGH University of Science and Technology, 30-059 Krakow, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lukasz","family":"Kowalski","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, AGH University of Science and Technology, 30-059 Krakow, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3344-2847","authenticated-orcid":false,"given":"Ayman","family":"Noor","sequence":"additional","affiliation":[{"name":"College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dhaval","family":"Thakker","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Bradford, Bradford BD7 1DP, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Graham","family":"Morgan","sequence":"additional","affiliation":[{"name":"School of Computing, Newcastle University, Newcastle upon Tyne NE1 7RU, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rajiv","family":"Ranjan","sequence":"additional","affiliation":[{"name":"School of Computing, Newcastle University, Newcastle upon Tyne NE1 7RU, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1016\/j.future.2015.09.021","article-title":"Integration of Cloud computing and Internet of Things: A survey","volume":"56","author":"Botta","year":"2016","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MITP.2017.20","article-title":"Urban Risk Analytics in the Cloud","volume":"19","author":"Ranjan","year":"2017","journal-title":"IT Prof."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MIC.2009.51","article-title":"Agents and Service-Oriented Computing for Autonomic Computing: A Research Agenda","volume":"13","author":"Brazier","year":"2009","journal-title":"IEEE Internet Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1016\/j.iot.2019.01.001","article-title":"Osmotic computing as a distributed multi-agent system: The Body Area Network scenario","volume":"5","author":"Carnevale","year":"2019","journal-title":"Internet Things"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1722","DOI":"10.1109\/COMST.2020.2988367","article-title":"Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges","volume":"22","author":"Lei","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2439","DOI":"10.1109\/JIOT.2018.2869986","article-title":"Using Correlated Information to Extend Device Lifetime","volume":"6","author":"Hribar","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_7","unstructured":"Schneider, J., Wong, W.K., Moore, A., and Riedmiller, M. (1999, January 27\u201330). Distributed Value Functions. Proceedings of the Sixteenth International Conference on Machine Learning, San Francisco, CA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1109\/TWC.2013.030413.121120","article-title":"A Learning Theoretic Approach to Energy Harvesting Communication System Optimization","volume":"12","author":"Blasco","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2009","DOI":"10.1109\/JIOT.2018.2872440","article-title":"Reinforcement Learning-Based Multiaccess Control and Battery Prediction with Energy Harvesting in IoT Systems","volume":"6","author":"Chu","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Huang, T., Lin, W., Li, Y., Wang, X., Wu, Q., Li, R., Hsu, C.H., and Zomaya, A.Y. (2021). Adaptive Processor Frequency Adjustment for Mobile Edge Computing with Intermittent Energy Supply. arXiv.","DOI":"10.1109\/JIOT.2021.3119866"},{"key":"ref_11","unstructured":"Laidi, R., Djenouri, D., and Balasingham, I. (2021). On Predicting Sensor Readings With Sequence Modeling and Reinforcement Learning for Energy-Efficient IoT Applications. IEEE Trans. Syst. Man Cybern. Syst., 1\u201312."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.15439\/2014F252","article-title":"Power aware MOM for telemetry-oriented applications using GPRS-enabled embedded devices-levee monitoring use case","volume":"Volume 2","author":"Ganzha","year":"2014","journal-title":"Proceedings of the 2014 Federated Conference on Computer Science and Information Systems"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.future.2016.08.007","article-title":"Holistic approach to management of IT infrastructure for environmental monitoring and decision support systems with urgent computing capabilities","volume":"79","author":"Balis","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101956","DOI":"10.1016\/j.sysarc.2020.101956","article-title":"IoTSim-Osmosis: A framework for modeling and simulating IoT applications over an edge-cloud continuum","volume":"116","author":"Alwasel","year":"2021","journal-title":"J. Syst. Archit."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2375\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:39:27Z","timestamp":1760135967000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,19]]},"references-count":14,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22062375"],"URL":"https:\/\/doi.org\/10.3390\/s22062375","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,19]]}}}