{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T14:12:45Z","timestamp":1790691165387,"version":"4.1.0"},"reference-count":48,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,6,18]],"date-time":"2020-06-18T00:00:00Z","timestamp":1592438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)","award":["2019R1F1A1042721"],"award-info":[{"award-number":["2019R1F1A1042721"]}]},{"name":"BK21 Plus project (SW Human Resource Development Program for Supporting Smart Life) funded by the Ministry of Education, School of Computer Science and Engineering, Kyungpook National University, Korea","award":["21A20131600005"],"award-info":[{"award-number":["21A20131600005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Maintaining a fair use of energy consumption in smart homes with many household appliances requires sophisticated algorithms working together in real time. Similarly, choosing a proper schedule for appliances operation can be used to reduce inappropriate energy consumption. However, scheduling appliances always depend on the behavior of a smart home user. Thus, modeling human interaction with appliances is needed to design an efficient scheduling algorithm with real-time support. In this regard, we propose a scheduling algorithm based on human appliances interaction in smart homes using reinforcement learning (RL). The proposed scheduling algorithm divides the entire day into various states. In each state, the agents attached to household appliances perform various actions to obtain the highest reward. To adjust the discomfort which arises due to performing inappropriate action, the household appliances are categorized into three groups i.e., (1) adoptable, (2) un-adoptable, (3) manageable. Finally, the proposed system is tested for the energy consumption and discomfort level of the home user against our previous scheduling algorithm based on least slack time phenomenon. The proposed scheme outperforms the Least Slack Time (LST) based scheduling in context of energy consumption and discomfort level of the home user.<\/jats:p>","DOI":"10.3390\/s20123450","type":"journal-article","created":{"date-parts":[[2020,6,18]],"date-time":"2020-06-18T11:00:56Z","timestamp":1592478056000},"page":"3450","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["A Multi-Objective Approach for Optimal Energy Management in Smart Home Using the Reinforcement Learning"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5862-4498","authenticated-orcid":false,"given":"Muhammad","family":"Diyan","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bhagya Nathali","family":"Silva","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kijun","family":"Han","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"24498","DOI":"10.1109\/ACCESS.2018.2831917","article-title":"Review on home energy management system considering demand responses, smart technologies, and intelligent controllers","volume":"6","author":"Shareef","year":"2018","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/JSYST.2013.2291943","article-title":"A new intelligent neuro-fuzzy paradigm for energy-efficient homes","volume":"8","author":"Shahgoshtasbi","year":"2014","journal-title":"IEEE Syst. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1109\/TSG.2017.2751465","article-title":"Appliance Identification Algorithm for a Non-Intrusive Home Energy Monitor Using Cogent Confabulation","volume":"10","author":"Park","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Silva, B.N., Lee, K., Yoon, Y., Han, J., Cao, Z.B., and Han, K. (2019). Cost-and comfort-aware aggregated modified least slack time\u2013based domestic power scheduling for residential communities. Trans. Emerg. Telecommun. Technol., e3834.","DOI":"10.1002\/ett.3834"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5103","DOI":"10.1109\/TSG.2018.2875727","article-title":"Energy optimization of HVAC systems in commercial buildings considering indoor air quality management","volume":"10","author":"Yu","year":"2018","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Silva, B.N., Khan, M., Jung, C., Seo, J., Muhammad, D., Han, J., Yoon, Y., and Han, K. (2018). Urban planning and smart city decision management empowered by real-time data processing using big data analytics. Sensors, 18.","DOI":"10.3390\/s18092994"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Diyan, M., Nathali Silva, B., Han, J., Cao, Z.B., and Han, K. (2020). Intelligent Internet of Things gateway supporting heterogeneous energy data management and processing. Trans. Emerg. Telecommun. Technol., e3919.","DOI":"10.1002\/ett.3919"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ahmed, M.S., Mohamed, A., Shareef, H., Homod, R.Z., and Ali, J.A. (2016, January 14\u201316). Artificial neural network based controller for home energy management considering demand response events. Proceedings of the 2016 International Conference on Advances in Electrical, Electronic and Systems Engineering (ICAEES), Putrajaya, Malaysia.","DOI":"10.1109\/ICAEES.2016.7888097"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1109\/TSG.2015.2506152","article-title":"Including Smart Loads for Optimal Demand Response in Integrated Energy Management Systems for Isolated Microgrids","volume":"8","author":"Solanki","year":"2017","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1790","DOI":"10.1109\/TSG.2016.2552169","article-title":"An Optimal and Learning-Based Demand Response and Home Energy Management System","volume":"7","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1109\/ACCESS.2016.2539369","article-title":"Real-Time Energy Management for Cloud Data Centers in Smart Microgrids","volume":"4","author":"Yu","year":"2016","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/TII.2016.2585122","article-title":"Assessment of Demand-Response-Driven Load Pattern Elasticity Using a Combined Approach for Smart Households","volume":"12","author":"Paterakis","year":"2016","journal-title":"IEEE Trans. Ind. Informatics"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1109\/TSG.2017.2775209","article-title":"Online energy management for a sustainable smart home with an HVAC load and random occupancy","volume":"10","author":"Yu","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/JIOT.2017.2765359","article-title":"Distributed Real-Time HVAC Control for Cost-Efficient Commercial Buildings under Smart Grid Environment","volume":"5","author":"Yu","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Viard, K., Fanti, M.P., Faraut, G., and Lesage, J.-J. (2020). Human Activity Discovery and Recognition Using Probabilistic Finite-State Automata. IEEE Trans. Autom. Sci. Eng., 1\u201312.","DOI":"10.1109\/TASE.2020.2989226"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1016\/j.enpol.2013.06.018","article-title":"Actual and theoretical gas consumption in Dutch dwellings: What causes the differences?","volume":"61","author":"Majcen","year":"2013","journal-title":"Energy Policy"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1109\/TII.2017.2753408","article-title":"Enabling cooperative behavior for building demand response based on extended joint action learning","volume":"14","author":"Hurtado","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kazmi, H., Amayri, M., and Mehmood, F. (2017, January 21\u201323). Smart home futures: Algorithmic challenges and opportunities. Proceedings of the 2017 14th International Symposium on Pervasive Systems, Algorithms and Networks & 2017 11th International Conference on Frontier of Computer Science and Technology & 2017 Third International Symposium of Creative Computing (ISPAN-FCST-ISCC), Exeter, UK.","DOI":"10.1109\/ISPAN-FCST-ISCC.2017.60"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"59141","DOI":"10.1109\/ACCESS.2018.2872589","article-title":"Distributed energy optimization for HVAC systems in university campus buildings","volume":"6","author":"Xie","year":"2018","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1109\/TIE.2016.2599479","article-title":"Real-time demand bidding for energy management in discrete manufacturing facilities","volume":"64","author":"Li","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/j.energy.2013.11.049","article-title":"User-expected price-based demand response algorithm for a home-to-grid system","volume":"64","author":"Li","year":"2014","journal-title":"Energy"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6629","DOI":"10.1109\/TSG.2019.2909266","article-title":"Demand Response for Home Energy Management Using Reinforcement Learning and Artificial Neural Network","volume":"10","author":"Lu","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1016\/j.ijepes.2015.11.099","article-title":"Residential power scheduling for demand response in smart grid","volume":"78","author":"Ma","year":"2016","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.apenergy.2017.06.010","article-title":"Incentive-based demand response considering hierarchical electricity market: A Stackelberg game approach","volume":"203","author":"Yu","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1612","DOI":"10.1016\/j.buildenv.2010.01.009","article-title":"ANN-based thermal control models for residential buildings","volume":"45","author":"Moon","year":"2010","journal-title":"Build. Environ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1109\/TSG.2010.2089069","article-title":"Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid","volume":"1","author":"Wong","year":"2010","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2166","DOI":"10.1109\/TSG.2012.2201182","article-title":"An algorithm for intelligent home energy management and demand response analysis","volume":"3","author":"Pipattanasomporn","year":"2012","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2531","DOI":"10.1016\/j.eswa.2009.08.019","article-title":"Power load forecasting using support vector machine and ant colony optimization","volume":"37","author":"Niu","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.enbuild.2015.11.017","article-title":"ANN-GA smart appliance scheduling for optimised energy management in the domestic sector","volume":"111","author":"Yuce","year":"2016","journal-title":"Energy Build."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1109\/TSG.2014.2349352","article-title":"Optimal smart home energy management considering energy saving and a comfortable lifestyle","volume":"6","author":"Monsef","year":"2015","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ahmed, M., Mohamed, A., Homod, R., and Shareef, H. (2016). Hybrid LSA-ANN Based Home Energy Management Scheduling Controller for Residential Demand Response Strategy. Energies, 9.","DOI":"10.3390\/en9090716"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1109\/TCE.2014.6851994","article-title":"Smart home energy management system including renewable energy based on ZigBee and PLC","volume":"60","author":"Han","year":"2014","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1258","DOI":"10.1109\/JSAC.2013.130709","article-title":"Heterogeneous delay tolerant task scheduling and energy management in the smart grid with renewable energy","volume":"31","author":"Chen","year":"2013","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_34","first-page":"74","article-title":"Conserving energy through neural prediction of sensed data","volume":"6","author":"Aram","year":"2015","journal-title":"JoWUA"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2081","DOI":"10.1109\/TCE.2009.5373772","article-title":"Intelligent power management device with middleware based living pattern learning for power reduction","volume":"55","author":"Lee","year":"2009","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1109\/JSEN.2015.2483745","article-title":"Accurate Wireless Sensor Localization Technique Based on Hybrid PSO-ANN Algorithm for Indoor and Outdoor Track Cycling","volume":"16","author":"Gharghan","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1650","DOI":"10.1109\/TSG.2015.2432571","article-title":"Queuing-Based Energy Consumption Management for Heterogeneous Residential Demands in Smart Grid","volume":"7","author":"Liu","year":"2016","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2751","DOI":"10.1109\/JIOT.2019.2957289","article-title":"Deep Reinforcement Learning for Smart Home Energy Management","volume":"7","author":"Yu","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_39","first-page":"1","article-title":"Real-Time Residential Demand Response","volume":"3053","author":"Li","year":"2020","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2149","DOI":"10.1109\/TSG.2016.2517211","article-title":"Residential demand response of thermostatically controlled loads using batch reinforcement learning","volume":"8","author":"Ruelens","year":"2017","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3698","DOI":"10.1109\/TSG.2018.2834219","article-title":"On-Line Building Energy Optimization Using Deep Reinforcement Learning","volume":"10","author":"Mocanu","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.conengprac.2017.11.010","article-title":"A simulation and control model for building energy management","volume":"72","author":"Fanti","year":"2018","journal-title":"Control Eng. Pract."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1109\/TSG.2013.2268664","article-title":"Load profiles of selected major household appliances and their demand response opportunities","volume":"5","author":"Pipattanasomporn","year":"2014","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1090\/S0002-9904-1954-09848-8","article-title":"The theory of dynamic programming","volume":"60","author":"Bellman","year":"1954","journal-title":"Bull. Amer. Math. Soc."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lee, S., and Choi, D.H. (2019). Reinforcement learning-based energy management of smart home with rooftop solar photovoltaic system, energy storage system, and home appliances. Sensors, 19.","DOI":"10.3390\/s19183937"},{"key":"ref_46","first-page":"879","article-title":"A real-time demand-response algorithm for smart grids: A stackelberg game approach","volume":"7","author":"Yu","year":"2016","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TPWRS.2012.2207134","article-title":"A game-theoretic approach for optimal time-of-use electricity pricing","volume":"28","author":"Yang","year":"2013","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Silva, B.N., Khan, M., and Han, K. (2018). Load balancing integrated least slack time-based appliance scheduling for smart home energy management. Sensors, 18.","DOI":"10.3390\/s18030685"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/12\/3450\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:40:31Z","timestamp":1760175631000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/12\/3450"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,18]]},"references-count":48,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["s20123450"],"URL":"https:\/\/doi.org\/10.3390\/s20123450","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,18]]}}}