{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T12:55:50Z","timestamp":1777380950539,"version":"3.51.4"},"reference-count":61,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AIS"],"published-print":{"date-parts":[[2024,3,14]]},"abstract":"<jats:p>Over the last fifty years, societies across the world have experienced multiple periods of energy insufficiency with the most recent one being the 2022 global energy crisis. In addition, the electric power industry has been experiencing a steady increase in electricity consumption since the second industrial revolution because of the widespread usage of electrical appliances and devices. Newer devices are equipped with sensors and actuators, they can collect a large amount of data that could help in power management. However, current energy management approaches are mostly applied to limited types of devices in specific domains and are difficult to implement in other scenarios. They fail when it comes to their level of autonomy, flexibility, and genericity. To address these shortcomings, we present, in this paper, an automated energy management approach for connected environments based on generating power estimation models, representing a formal description of energy-related knowledge, and using reinforcement learning (RL) techniques to accomplish energy-efficient actions. The architecture of this approach is based on three main components: power estimation models, knowledge base, and intelligence module. Furthermore, we develop algorithms that exploit knowledge from both the power estimator and the ontology, to generate the corresponding RL agent and environment. We also present different reward functions based on user preferences and power consumption. We illustrate our proposal in the smart home domain. An implementation of the approach is developed and two validation experiments are conducted. Both case studies are deployed in the context of smart homes: (a) a living room with a variety of devices and (b) a smart home with a heating system. The obtained results show that our approach performs well given the low convergence period, the high level of user preferences satisfaction, and the significant decrease in energy consumption.<\/jats:p>","DOI":"10.3233\/ais-220482","type":"journal-article","created":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T11:22:39Z","timestamp":1700220159000},"page":"23-42","source":"Crossref","is-referenced-by-count":5,"title":["An automated energy management framework for smart homes"],"prefix":"10.1177","volume":"16","author":[{"given":"Houssam","family":"Kanso","sequence":"first","affiliation":[{"name":"Universite de Pau et des Pays de l\u2019Adour, E2S UPPA, LIUPPA, Anglet, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adel","family":"Noureddine","sequence":"additional","affiliation":[{"name":"Universite de Pau et des Pays de l\u2019Adour, E2S UPPA, LIUPPA, Anglet, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ernesto","family":"Exposito","sequence":"additional","affiliation":[{"name":"Universite de Pau et des Pays de l\u2019Adour, E2S UPPA, LIUPPA, Anglet, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/AIS-220482_ref1","unstructured":"Y.\u00a0Agarwal, S.\u00a0Savage and R.\u00a0Gupta, SleepServer: A software-only approach for reducing the energy consumption of PCs within enterprise environments, in: Proceedings of the 2010 USENIX Annual Technical Conference, Boston, MA, USA, 2010, pp.\u00a0285\u2013299. ISBN 9781931971751."},{"issue":"1","key":"10.3233\/AIS-220482_ref2","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1109\/TETC.2017.2680322","article-title":"Human behavior aware energy management in residential cyber-physical systems","volume":"8","author":"Aksanli","year":"2016","journal-title":"IEEE Transactions on Emerging Topics in Computing"},{"key":"10.3233\/AIS-220482_ref3","doi-asserted-by":"publisher","first-page":"1246","DOI":"10.1016\/j.apenergy.2018.09.188","article-title":"Automating occupant-building interaction via smart zoning of thermostatic loads: A switched self-tuning approach","volume":"231","author":"Baldi","year":"2018","journal-title":"Applied Energy, Elsevier"},{"key":"10.3233\/AIS-220482_ref4","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-39940-9_1073"},{"key":"10.3233\/AIS-220482_ref5","doi-asserted-by":"publisher","DOI":"10.1109\/ICUT.2009.5405699"},{"key":"10.3233\/AIS-220482_ref6","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.websem.2012.05.003","article-title":"The SSN ontology of the W3C semantic sensor network incubator group","volume":"17","author":"Compton","year":"2012","journal-title":"Journal of Web Semantics"},{"issue":"8","key":"10.3233\/AIS-220482_ref7","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1016\/0305-0548(91)90013-H","article-title":"Estia: A real-time consumer control scheme for space conditioning usage under spot electricity pricing","volume":"18","author":"Constantopoulos","year":"1991","journal-title":"Computers & Operations Research"},{"key":"10.3233\/AIS-220482_ref8","isbn-type":"print","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1145\/3294109.3295634","volume-title":"CairnFORM, in: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction TEI 2019","author":"Daniel","year":"2019","ISBN":"https:\/\/id.crossref.org\/isbn\/9781450361965"},{"issue":"1\u20132","key":"10.3233\/AIS-220482_ref9","doi-asserted-by":"publisher","first-page":"1000","DOI":"10.1080\/00207543.2017.1394596","article-title":"Data cleansing for energy-saving: A case of cyber-physical machine tools health monitoring system","volume":"56","author":"Deng","year":"2018","journal-title":"International Journal of Production Research"},{"key":"10.3233\/AIS-220482_ref10","doi-asserted-by":"publisher","DOI":"10.1109\/GLOCOM.2016.7841899"},{"issue":"4","key":"10.3233\/AIS-220482_ref11","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1175\/BAMS-D-12-00170.1","article-title":"U.S. climate reference network after one decade of operations: Status and assessment","volume":"94","author":"Diamond","year":"2013","journal-title":"Bulletin of the American Meteorological Society"},{"key":"10.3233\/AIS-220482_ref13","doi-asserted-by":"publisher","DOI":"10.1109\/ASPDAC.2014.6742873"},{"key":"10.3233\/AIS-220482_ref14","doi-asserted-by":"publisher","DOI":"10.1109\/GLOCOM.2013.6831558"},{"key":"10.3233\/AIS-220482_ref15","doi-asserted-by":"publisher","DOI":"10.1016\/j.apenergy.2019.113804"},{"issue":"3","key":"10.3233\/AIS-220482_ref16","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1109\/LES.2017.2741419","article-title":"The IoT energy challenge: A software perspective","volume":"10","author":"Georgiou","year":"2018","journal-title":"IEEE Embedded Systems Letters"},{"issue":"1","key":"10.3233\/AIS-220482_ref17","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1109\/TCE.2011.5735485","article-title":"More efficient home energy management system based on ZigBee communication and infrared remote controls","volume":"57","author":"Han","year":"2011","journal-title":"IEEE Transactions on Consumer Electronics"},{"issue":"2","key":"10.3233\/AIS-220482_ref18","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1007\/s10664-013-9276-6","article-title":"Green mining: A methodology of relating software change and configuration to power consumption","volume":"20","author":"Hindle","year":"2015","journal-title":"Empirical Software Engineering"},{"issue":"8","key":"10.3233\/AIS-220482_ref19","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.adhoc.2018.08.004","article-title":"Context-aware energy-efficient applications for cyber-physical systems","volume":"82","author":"Horcas","year":"2019","journal-title":"Ad Hoc Networks"},{"key":"10.3233\/AIS-220482_ref21","doi-asserted-by":"crossref","unstructured":"J.A.E.\u00a0Isasa, P.G.\u00a0Larsen and F.O.\u00a0Hansen, Energy-aware model-driven development of a wearable healthcare device, in: Software Engineering in Health Care, M.\u00a0Huhn and L.\u00a0Williams, eds, Springer International Publishing, Cham, 2017, pp.\u00a044\u201363. ISBN 978-3-319-63194-3.","DOI":"10.1007\/978-3-319-63194-3_4"},{"key":"10.3233\/AIS-220482_ref22","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.180695"},{"key":"10.3233\/AIS-220482_ref23","doi-asserted-by":"publisher","first-page":"2210","DOI":"10.1016\/j.suscom.2022.100837","article-title":"Automated power modeling of computing devices: Implementation and use case for Raspberry PIs","volume":"37","author":"Kanso","year":"2023","journal-title":"Sustainable Computing: Informatics and Systems"},{"key":"10.3233\/AIS-220482_ref24","doi-asserted-by":"publisher","DOI":"10.1109\/ICUFN.2017.7993747"},{"key":"10.3233\/AIS-220482_ref25","doi-asserted-by":"publisher","DOI":"10.3390\/en11082010"},{"key":"10.3233\/AIS-220482_ref26","doi-asserted-by":"publisher","DOI":"10.1109\/CCWC.2018.8301740"},{"key":"10.3233\/AIS-220482_ref27","doi-asserted-by":"publisher","first-page":"810","DOI":"10.1016\/j.apenergy.2018.12.065","article-title":"A deep learning and gamification approach to improving human-building interaction and energy efficiency in smart infrastructure","volume":"237","author":"Konstantakopoulos","year":"2019","journal-title":"Applied Energy"},{"key":"10.3233\/AIS-220482_ref28","doi-asserted-by":"publisher","DOI":"10.1109\/CONTROLO.2018.8514291"},{"issue":"2","key":"10.3233\/AIS-220482_ref29","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1109\/TETC.2013.2274042","article-title":"Scheduling co-design for reliability and energy in cyber-physical systems","volume":"1","author":"Lin","year":"2013","journal-title":"IEEE Transactions on Emerging Topics in Computing"},{"key":"10.3233\/AIS-220482_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAD.2015.7372628"},{"key":"10.3233\/AIS-220482_ref31","doi-asserted-by":"publisher","DOI":"10.1016\/j.egyai.2020.100043"},{"issue":"3","key":"10.3233\/AIS-220482_ref32","doi-asserted-by":"publisher","first-page":"572","DOI":"10.17775\/CSEEJPES.2019.02890","article-title":"Optimization strategy based on deep reinforcement learning for home energy management","volume":"6","author":"Liu","year":"2020","journal-title":"CSEE Journal of Power and Energy Systems"},{"issue":"2","key":"10.3233\/AIS-220482_ref33","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1109\/TIE.2018.2818666","article-title":"Smart residential load simulator for energy management in smart grids","volume":"66","author":"L\u00f3pez","year":"2019","journal-title":"IEEE Transactions on Industrial Electronics"},{"issue":"4","key":"10.3233\/AIS-220482_ref34","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1109\/THMS.2018.2844119","article-title":"IoT-enabled adaptive context-aware and playful cyber-physical system for everyday energy savings","volume":"48","author":"Lu","year":"2018","journal-title":"IEEE Transactions on Human-Machine Systems"},{"issue":"6","key":"10.3233\/AIS-220482_ref35","doi-asserted-by":"publisher","first-page":"6629","DOI":"10.1109\/TSG.2019.2909266","article-title":"Demand response for home energy management using reinforcement learning and artificial neural network","volume":"10","author":"Lu","year":"2019","journal-title":"IEEE Transactions on Smart Grid"},{"key":"10.3233\/AIS-220482_ref36","doi-asserted-by":"publisher","first-page":"300","DOI":"10.1016\/j.compeleceng.2019.07.019","article-title":"A review of reinforcement learning for autonomous building energy management","volume":"78","author":"Mason","year":"2019","journal-title":"Computers and Electrical Engineering"},{"issue":"4","key":"10.3233\/AIS-220482_ref38","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1145\/2757001.2757003","article-title":"The prot\u00e9g\u00e9 project: A look back and a look forward","volume":"1","author":"Musen","year":"2015","journal-title":"AI Matters"},{"key":"10.3233\/AIS-220482_ref39","unstructured":"N.\u00a0Noy and D.\u00a0Mcguinness, Ontology development 101: A guide to creating your first ontology, Knowledge Systems Laboratory 32 (2001)."},{"key":"10.3233\/AIS-220482_ref40","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.enbuild.2014.01.036","article-title":"Saving energy in an office environment: A serious game intervention","volume":"74","author":"Orland","year":"2014","journal-title":"Energy and Buildings"},{"issue":"1","key":"10.3233\/AIS-220482_ref41","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1007\/s40684-019-00084-7","article-title":"Cyber physical energy system for saving energy of the dyeing process with industrial Internet of things and manufacturing big data","volume":"7","author":"Park","year":"2020","journal-title":"International Journal of Precision Engineering and Manufacturing-Green Technology"},{"issue":"1","key":"10.3233\/AIS-220482_ref42","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1109\/JPROC.2011.2161244","article-title":"A cyber-physical systems approach to data center modeling and control for energy efficiency","volume":"100","author":"Parolini","year":"2012","journal-title":"Proceedings of the IEEE"},{"key":"10.3233\/AIS-220482_ref43","doi-asserted-by":"publisher","DOI":"10.1145\/1795194.1795218"},{"key":"10.3233\/AIS-220482_ref44","doi-asserted-by":"publisher","DOI":"10.1109\/ICCICT.2012.6398115"},{"issue":"2","key":"10.3233\/AIS-220482_ref45","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1007\/s11280-019-00684-y","article-title":"A QoS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systems","volume":"23","author":"Qi","year":"2020","journal-title":"World Wide Web"},{"key":"10.3233\/AIS-220482_ref46","doi-asserted-by":"publisher","DOI":"10.1109\/DEST.2010.5610636"},{"issue":"3","key":"10.3233\/AIS-220482_ref47","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1109\/JSYST.2010.2059212","article-title":"Efficient utilization of renewable energy sources by gridable vehicles in cyber-physical energy systems","volume":"4","author":"Saber","year":"2010","journal-title":"IEEE Systems Journal"},{"key":"10.3233\/AIS-220482_ref48","doi-asserted-by":"publisher","DOI":"10.1109\/ICC.2014.6883919"},{"issue":"1","key":"10.3233\/AIS-220482_ref49","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.future.2013.03.001","article-title":"WSN in cyber physical systems: Enhanced energy management routing approach using software agents","volume":"31","author":"Shakshuki","year":"2014","journal-title":"Future Generation Computer Systems"},{"key":"10.3233\/AIS-220482_ref50","doi-asserted-by":"publisher","first-page":"24498","DOI":"10.1109\/ACCESS.2018.2831917","article-title":"Review on home energy management system considering demand responses, smart technologies, and intelligent controllers","volume":"6","author":"Shareef","year":"2018","journal-title":"IEEE Access"},{"issue":"4","key":"10.3233\/AIS-220482_ref51","doi-asserted-by":"publisher","first-page":"773","DOI":"10.1109\/TSMCA.2012.2224337","article-title":"Sensing-driven energy purchasing in smart grid cyber-physical system","volume":"43","author":"Tham","year":"2013","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"issue":"3","key":"10.3233\/AIS-220482_ref52","doi-asserted-by":"publisher","first-page":"1418","DOI":"10.1109\/TSG.2012.2190114","article-title":"Towards a unified operational value index of energy storage in smart grid environment","volume":"3","author":"Thatte","year":"2012","journal-title":"IEEE Transactions on Smart Grid"},{"issue":"8","key":"10.3233\/AIS-220482_ref54","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1093\/comjnl\/bxt043","article-title":"Cyber-physical systems for optimal energy management scheme of autonomous electric vehicle","volume":"56","author":"Wan","year":"2013","journal-title":"Computer Journal"},{"key":"10.3233\/AIS-220482_ref55","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.apenergy.2018.11.079","article-title":"Linking energy-cyber-physical systems with occupancy prediction and interpretation through WiFi probe-based ensemble classification","volume":"236","author":"Wang","year":"2019","journal-title":"Applied Energy"},{"key":"10.3233\/AIS-220482_ref56","doi-asserted-by":"publisher","DOI":"10.1109\/ISGT-Asia.2011.6167084"},{"issue":"4","key":"10.3233\/AIS-220482_ref57","doi-asserted-by":"publisher","first-page":"3201","DOI":"10.1109\/TSG.2020.2971427","article-title":"A multi-agent reinforcement learning-based data-driven method for home energy management","volume":"11","author":"Xu","year":"2020","journal-title":"IEEE Transactions on Smart Grid"},{"issue":"15","key":"10.3233\/AIS-220482_ref58","doi-asserted-by":"publisher","first-page":"12046","DOI":"10.1109\/JIOT.2021.3078462","article-title":"A review of deep reinforcement learning for smart building energy management","volume":"8","author":"Yu","year":"2021","journal-title":"IEEE Internet of Things Journal"},{"key":"10.3233\/AIS-220482_ref59","first-page":"1","article-title":"Deep reinforcement learning for smart home energy management","volume":"7","author":"Yu","year":"2019","journal-title":"IEEE Internet of Things Journal"},{"issue":"4","key":"10.3233\/AIS-220482_ref60","doi-asserted-by":"publisher","first-page":"2751","DOI":"10.1109\/JIOT.2019.2957289","article-title":"Deep reinforcement learning for smart home energy management","volume":"7","author":"Yu","year":"2019","journal-title":"IEEE Internet of Things Journal"},{"issue":"3","key":"10.3233\/AIS-220482_ref61","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1109\/MS.2014.27","article-title":"The impact of user choice on energy consumption","volume":"31","author":"Zhang","year":"2014","journal-title":"IEEE Software"},{"issue":"4","key":"10.3233\/AIS-220482_ref62","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.1109\/TSG.2016.2552169","article-title":"An optimal and learning-based demand response and home energy management system","volume":"7","author":"Zhang","year":"2016","journal-title":"IEEE Transactions on Smart Grid"},{"key":"10.3233\/AIS-220482_ref63","doi-asserted-by":"publisher","DOI":"10.1109\/INDUSCON.2010.5739891"},{"key":"10.3233\/AIS-220482_ref64","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.rser.2016.03.047","article-title":"Smart home energy management systems: Concept, configurations, and scheduling strategies","volume":"61","author":"Zhou","year":"2016","journal-title":"Renewable and Sustainable Energy Reviews"},{"issue":"1","key":"10.3233\/AIS-220482_ref65","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/TSMC.2019.2896323","article-title":"Secure and efficient vehicle-to-grid energy trading in cyber physical systems: Integration of blockchain and edge computing","volume":"50","author":"Zhou","year":"2020","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"}],"container-title":["Journal of Ambient Intelligence and Smart Environments"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/AIS-220482","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T09:19:25Z","timestamp":1777367965000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/AIS-220482"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,14]]},"references-count":61,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/ais-220482","relation":{},"ISSN":["1876-1372","1876-1364"],"issn-type":[{"value":"1876-1372","type":"electronic"},{"value":"1876-1364","type":"print"}],"subject":[],"published":{"date-parts":[[2024,3,14]]}}}