{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T06:09:58Z","timestamp":1776838198921,"version":"3.51.2"},"reference-count":34,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T00:00:00Z","timestamp":1767830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Smart environments play a key role in improving user comfort, energy efficiency, and sustainability through intelligent automation. Nevertheless, real-world deployments still face major challenges, including network instability, delayed responsiveness, inconsistent AI decisions, and limited adaptability under dynamic conditions. Many existing approaches lack advanced context-awareness, effective multi-agent coordination, and scalable learning, leading to high computational cost and reduced reliability. To address these limitations, this paper proposes MACxRL, a lightweight Multi-Agent Context-Aware Reinforcement Learning framework for autonomous smart-environment control. The system adopts a three-tier architecture consisting of real-time context acquisition, lightweight prediction, and centralized RL-based decision learning. Local agents act quickly at the edge using rule-based reasoning, while a shared CxRL engine refines actions for global coordination, combining fast responsiveness with continuous adaptive learning. Experiments show that MACxRL reduces energy consumption by 45\u201360%, converges faster, and achieves more stable performance than standard and deep RL baselines. Future work will explore self-adaptive reward tuning and extend deployment to multi-room environments toward practical real-world realization.<\/jats:p>","DOI":"10.3390\/fi18010040","type":"journal-article","created":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T08:09:06Z","timestamp":1767946146000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Adaptive and Sustainable Smart Environments Using Predictive Reasoning and Context-Aware Reinforcement Learning"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-5177-5306","authenticated-orcid":false,"given":"Abderrahim","family":"Lakehal","sequence":"first","affiliation":[{"name":"Networks and Distributed Systems Laboratory, Computer Science Department, Faculty of Sciences, University Ferhat Abbas S\u00e9tif-1, S\u00e9tif P.O. Box 19000, Algeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-6277-4695","authenticated-orcid":false,"given":"Boubakeur","family":"Annane","sequence":"additional","affiliation":[{"name":"Networks and Distributed Systems Laboratory, Computer Science Department, Faculty of Sciences, University Ferhat Abbas S\u00e9tif-1, S\u00e9tif P.O. Box 19000, Algeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8348-1679","authenticated-orcid":false,"given":"Adel","family":"Alti","sequence":"additional","affiliation":[{"name":"Networks and Distributed Systems Laboratory, Computer Science Department, Faculty of Sciences, University Ferhat Abbas S\u00e9tif-1, S\u00e9tif P.O. Box 19000, Algeria"},{"name":"Department of Management Information Systems, College of Business and Economics, Qassim University, Buraydah 51452, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2227-3283","authenticated-orcid":false,"given":"Philippe","family":"Roose","sequence":"additional","affiliation":[{"name":"LIUPPA-T2I\/IUT of Bayonne E2S UPPA, University of Pau, 64600 Anglet, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soliman","family":"Aljarboa","sequence":"additional","affiliation":[{"name":"Department of Management Information Systems, College of Business and Economics, Qassim University, Buraydah 51452, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/SURV.2013.042313.00197","article-title":"Context aware computing for the internet of things: A survey","volume":"16","author":"Perera","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1504\/IJAHUC.2007.014070","article-title":"A survey on context-aware systems","volume":"2","author":"Baldauf","year":"2007","journal-title":"Int. J. Ad Hoc Ubiquitous Comput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1017\/S0269888907001208","article-title":"Ontology-based models in pervasive computing systems","volume":"22","author":"Ye","year":"2007","journal-title":"Knowl. Eng. Rev."},{"key":"ref_4","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_5","unstructured":"Wang, X., and Sandholm, T. (2003). Reinforcement learning to play an optimal Nash equilibrium in team Markov games. Adv. Neural Inf. Process. Syst., 15, Available online: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2002\/file\/f8e59f4b2fe7c5705bf878bbd494ccdf-Paper.pdf."},{"key":"ref_6","first-page":"16","article-title":"A reinforcement learning-based approach for smart home energy management","volume":"132","author":"Boukerche","year":"2019","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_7","unstructured":"Tesauro, G., Jong, N.K., Das, R., and Bennani, M.N. (2006, January 12\u201316). A hybrid reinforcement learning approach to autonomic resource allocation. Proceedings of the IEEE International Conference on Autonomic Computing, Dublin, Ireland."},{"key":"ref_8","unstructured":"Du, H., Thudumu, S., Vasa, R., and Mouzakis, K. (2024). A survey on context-aware multi-agent systems: Techniques, challenges and future directions. arXiv."},{"key":"ref_9","unstructured":"Chen, B., Xu, M., Liu, Z., Li, L., and Zhao, D. (2020). Delay-aware multi-agent reinforcement learning for cooperative and competitive environments. arXiv."},{"key":"ref_10","unstructured":"Chen, D., and Zhang, Q. (2023, January 23\u201329). Context-aware Bayesian network actor-critic methods for cooperative multi-agent reinforcement learning. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA."},{"key":"ref_11","unstructured":"Li, X., and Zhang, J. (2023). Context-aware communication for multi-agent reinforcement learning. arXiv."},{"key":"ref_12","unstructured":"Iqbal, S., and Sha, F. (2019, January 9\u201315). Actor-attention-critic for multi-agent reinforcement learning. Proceedings of the International Conference on machine learning, Long Beach, CA, USA."},{"key":"ref_13","unstructured":"Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P.H., Kohli, P., and Whiteson, S. (2017, January 6\u201311). Stabilizing experience replay for deep multi-agent reinforcement learning. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Park, B., and Choi, J. (2024). Message Action Adapter Framework in Multi-Agent Reinforcement Learning. Appl. Sci., 14.","DOI":"10.3390\/app142110079"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.pmcj.2018.06.011","article-title":"Arcades: A deep model for adaptive decision making in voice-controlled smart-home","volume":"49","author":"Brenon","year":"2018","journal-title":"Pervasive Mob. Comput."},{"key":"ref_16","unstructured":"Hosseinloo, A.H., and Dahleh, M.A. (2020, January 23\u201325). Event-Triggered Reinforcement Learning; An Application to Buildings\u2019 Micro-Climate Control. Proceedings of the AAAI Spring Symposium: MLPS, Stanford, CA, USA."},{"key":"ref_17","unstructured":"Kalivarathan, S., Mohamed, M.A.R., Ravikumar, A., and Harini, S. (2025). Intelligence of Things: A Spatial Context-Aware Control System for Smart Devices. arXiv."},{"key":"ref_18","unstructured":"Chen, B., Liu, Z., Zhu, J., Xu, M., Ding, W., Li, L., and Zhao, D. (June, January 30). Context-aware safe reinforcement learning for non-stationary environments. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"25","DOI":"10.3991\/ijoe.v20i06.46545","article-title":"Context-Aware IoT System Development Approach Based on Meta-Modeling and Reinforcement Learning: A Smart Home Case Study","volume":"20","author":"Hallou","year":"2024","journal-title":"Int. J. Online Biomed. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"15501329221082030","DOI":"10.1177\/15501329221082030","article-title":"Intelligent control system of smart home for context awareness","volume":"18","author":"Li","year":"2022","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Weerasinghe, S., Zaslavsky, A., Loke, S.W., Hassani, A., Medvedev, A., and Abken, A. (2023). Adaptive context caching for IoT-based applications: A reinforcement learning approach. Sensors, 23.","DOI":"10.3390\/s23104767"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.future.2019.11.028","article-title":"CaaSSET: A framework for model-driven development of context as a service","volume":"105","author":"Moradi","year":"2020","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Delnevo, G., Ghini, V., Fiumana, E., and Mirri, S. (2024). A Support Tool for Emergency Management in Smart Campuses: Reference Architecture and Enhanced Web User Interfaces. Sensors, 24.","DOI":"10.3390\/s24185887"},{"key":"ref_24","unstructured":"Polo-Rodr\u00edguez, A., Fiorini, L., Rovini, E., Cavallo, F., and Medina-Quero, J. (2025). Enhancing smart environments with context-aware chatbots using large language models. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s43621-025-00954-y","article-title":"Cognition and context-aware decision-making systems for a sustainable planet: A survey on recent advancements, applications and open challenges","volume":"6","author":"Violos","year":"2025","journal-title":"Discov. Sustain."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2008644","DOI":"10.1080\/08839514.2021.2008644","article-title":"Contexts and context-awareness revisited from an intelligent environments perspective","volume":"36","author":"Augusto","year":"2022","journal-title":"Appl. Artif. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4679","DOI":"10.1007\/s12652-018-0846-8","article-title":"One app to rule them all: Collaborative injection of situations in an adaptable context-aware application","volume":"10","author":"Karchoud","year":"2019","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.buildenv.2013.11.016","article-title":"Theory and applications of HVAC control systems\u2013A review of model predictive control (MPC)","volume":"72","author":"Afram","year":"2014","journal-title":"Build. Environ."},{"key":"ref_29","first-page":"114487","article-title":"Development and evaluation of rule-based control strategies for smart building energy management","volume":"264","author":"Park","year":"2020","journal-title":"Appl. Energy"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.3390\/app14041421","article-title":"Deep Q-learning-based smart scheduling of EVS for demand response in smart grids","volume":"14","author":"Chifu","year":"2024","journal-title":"Appl. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Coraci, D., Brandi, S., Piscitelli, M.S., and Capozzoli, A. (2021). Online implementation of a soft actor-critic agent to enhance indoor temperature control and energy efficiency in buildings. Energies, 14.","DOI":"10.3390\/en14040997"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Anan, M., Kanaan, K., Benhaddou, D., Nasser, N., Qolomany, B., Talei, H., and Sawalmeh, A. (2024). Occupant-aware energy consumption prediction in smart buildings using a LSTM model and time series data. Energies, 17.","DOI":"10.3390\/en17246451"},{"key":"ref_33","first-page":"47","article-title":"SecNetworkCloudSim: An extensible simulation tool for secure distributed mobile applications","volume":"12","author":"Annane","year":"2020","journal-title":"Int. J. Commun. Netw. Inf. Secur."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"6025","DOI":"10.1109\/TCSI.2025.3534194","article-title":"Adaptive cooperative fault-tolerant control for output-constrained nonlinear multi-agent systems under stochastic FDI attacks","volume":"72","author":"Liu","year":"2025","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/1\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T08:14:50Z","timestamp":1767946490000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/1\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,8]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["fi18010040"],"URL":"https:\/\/doi.org\/10.3390\/fi18010040","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,8]]}}}