{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T03:23:47Z","timestamp":1773977027887,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T00:00:00Z","timestamp":1684281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the massive, worldwide, smart metering roll-out, both energy suppliers and users are starting to tap into the potential of higher resolution energy readings for accurate billing, improved demand response, improved tariffs better tuned to users and the grid, and empowering end-users to know how much their individual appliances contribute to their electricity bills via nonintrusive load monitoring (NILM). A number of NILM approaches, based on machine learning (ML), have been proposed over the years, focusing on improving the NILM model performance. However, the trustworthiness of the NILM model itself has hardly been addressed. It is important to explain the underlying model and its reasoning to understand why the model underperforms in order to satisfy user curiosity and to enable model improvement. This can be done by leveraging naturally interpretable or explainable models as well as explainability tools. This paper adopts a naturally interpretable decision tree (DT)-based approach for a NILM multiclass classifier. Furthermore, this paper leverages explainability tools to determine local and global feature importance, and design a methodology that informs feature selection for each appliance class, which can determine how well a trained model will predict an appliance on any unseen test data, minimising testing time on target datasets. We explain how one or more appliances can negatively impact classification of other appliances and predict appliance and model performance of the REFIT-data trained models on unseen data of the same house and on unseen houses on the UK-DALE dataset. Experimental results confirm that models trained with the explainability-informed local feature importance can improve toaster classification performance from 65% to 80%. Additionally, instead of one five-classifier approach incorporating all five appliances, a three-classifier approach comprising a kettle, microwave, and dishwasher and a two-classifier comprising a toaster and washing machine improves classification performance for the dishwasher from 72% to 94% and the washing machine from 56% to 80%.<\/jats:p>","DOI":"10.3390\/s23104845","type":"journal-article","created":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T07:35:50Z","timestamp":1684395350000},"page":"4845","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Explainability-Informed Feature Selection and Performance Prediction for Nonintrusive Load Monitoring"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8591-9830","authenticated-orcid":false,"given":"Rachel Stephen","family":"Mollel","sequence":"first","affiliation":[{"name":"Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8112-1976","authenticated-orcid":false,"given":"Lina","family":"Stankovic","sequence":"additional","affiliation":[{"name":"Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1075-2420","authenticated-orcid":false,"given":"Vladimir","family":"Stankovic","sequence":"additional","affiliation":[{"name":"Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UK"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"111951","DOI":"10.1016\/j.enbuild.2022.111951","article-title":"NILM Applications: Literature review of learning approaches, recent developments and challenges","volume":"261","author":"Angelis","year":"2022","journal-title":"Energy Build."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Huber, P., Calatroni, A., Rumsch, A., and Paice, A. (2021). Review on Deep Neural Networks Applied to Low-Frequency NILM. Energies, 14.","DOI":"10.20944\/preprints202104.0421.v1"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kaselimi, M., Protopapadakis, E., Voulodimos, A., Doulamis, N., and Doulamis, A. (2022). Towards Trustworthy Energy Disaggregation: A Review of Challenges, Methods, and Perspectives for Non-Intrusive Load Monitoring. Sensors, 22.","DOI":"10.3390\/s22155872"},{"key":"ref_4","unstructured":"European Commission (2019). Directorate-General for Communications Networks, Content and Technology, Ethics Guidelines for Trustworthy AI, European Commission. Publications Office."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Murray, D., Stankovic, L., and Stankovic, V. (2021, January 17\u201318). Transparent AI: Explainability of deep learning based load disaggregation. Proceedings of the 1st ACM SIGEnergy Workshop of Fair, Accountable, Transparent and Ethical AI for Smart Environments and Energy Systems (FATEsys \u201921), Coimbra, Portugal.","DOI":"10.1145\/3486611.3492410"},{"key":"ref_6","unstructured":"Molnar, C. (2023, March 29). Interpretable Machine Learning, A Guide for Making Black Box Models Explainable. Available online: https:\/\/christophm.github.io\/interpretable-ml-book\/."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mollel, S.R., Stankovic, L., and Stankovic, V. (2022, January 11). Using Explainability Tools to Inform NILM Algorithm Performance: A Decision Tree Approach. Proceedings of the 6th International Workshop on Non-Intrusive Load Monitoring (NILM \u201922), Hybrid Event, Boston, MA, USA.","DOI":"10.1145\/3563357.3566148"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1080\/10618600.2014.907095","article-title":"Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation","volume":"24","author":"Goldstein","year":"2015","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201c Why should i trust you?\u201d Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA. Available online: https:\/\/arxiv.org\/abs\/1602.04938.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"103502","DOI":"10.1016\/j.artint.2021.103502","article-title":"Explaining individual predictions when features are dependent: More accurate approximations to Shapley values","volume":"298","author":"Aas","year":"2021","journal-title":"Artif. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1784","DOI":"10.1109\/ACCESS.2016.2557460","article-title":"On a Training-Less Solution for Non-Intrusive Appliance Load Monitoring Using Graph Signal Processing","volume":"4","author":"Zhao","year":"2016","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liao, J., Elafoudi, G., Stankovic, L., and Stankovic, V. (2014, January 3\u20136). Non-Intrusive Appliance Load Monitoring using Low-Resolution Smart Meter Data. Proceedings of the 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm), Venice, Italy.","DOI":"10.1109\/SmartGridComm.2014.7007702"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Vavouris, A., Garside, B., Stankovic, L., and Stankovic, V. (2022). Low-Frequency Non-Intrusive Load Monitoring of Electric Vehicles in Houses with Solar Generation: Generalisability and Transferability. Energies, 15.","DOI":"10.3390\/en15062200"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"6472","DOI":"10.1109\/TSG.2019.2906012","article-title":"A generic optimisation-based approach for improving non-intrusive load monitoring","volume":"10","author":"He","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.enbuild.2017.06.042","article-title":"Comprehensive feature selection for appliance classification in NILM","volume":"151","author":"Sadeghianpourhamami","year":"2017","journal-title":"Energy Build."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108673","DOI":"10.1016\/j.epsr.2022.108673","article-title":"Electric energy disaggregation via non-intrusive load monitoring: A state-of-the-art systematic review","volume":"213","author":"Dash","year":"2022","journal-title":"Electr. Power Syst. Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e1265","DOI":"10.1002\/widm.1265","article-title":"Performance evaluation in non-intrusive load monitoring: Datasets, metrics, and tools\u2014A review","volume":"8","author":"Pereira","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"884","DOI":"10.3934\/energy.2016.1.1","article-title":"Low-complexity energy disaggregation using appliance load modelling","volume":"4","author":"Altrabalsi","year":"2016","journal-title":"Aims Energy"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Suthaharan, S. (2016). Machine Learning Models and Algorithms for Big Data Classification. Integrated Series in Information Systems, Springer.","DOI":"10.1007\/978-1-4899-7641-3"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Khazaei, M., Stankovic, L., and Stankovic, V. (2020, January 18). Evaluation of low-complexity supervised and unsupervised NILM methods and pre-processing for detection of multistate white goods. Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring (NILM\u2019 20), Virtual Event.","DOI":"10.1145\/3427771.3427850"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Machlev, R., Malka, A., Perl, M., Levron, Y., and Belikov, J. (2022, January 17\u201321). Explaining the Decisions of Deep Learning Models for Load Disaggregation (NILM) Based on XAI. Proceedings of the 2022 IEEE Power & Energy Society General Meeting (PESGM), Denver, CO, USA.","DOI":"10.1109\/PESGM48719.2022.9917049"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"101647","DOI":"10.1016\/j.compenvurbsys.2021.101647","article-title":"Peeking inside the black-box: Explainable machine learning applied to household transportation energy consumption","volume":"88","author":"Amiri","year":"2021","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_23","unstructured":"Tukey, J.W. (1977). Exploratory Data Analysis, Addison-Wesley."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hu, K. (2020). Become Competent within One Day in Generating Boxplots and Violin Plots for a Novice without Prior R Experience. Methods Protoc., 3.","DOI":"10.3390\/mps3040064"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"127800","DOI":"10.1016\/j.physleta.2021.127800","article-title":"Bandgap prediction of metal halide perovskites using regression machine learning models","volume":"422","author":"Vakharia","year":"2022","journal-title":"Phys. Lett. A"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"160122","DOI":"10.1038\/sdata.2016.122","article-title":"An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study","volume":"4","author":"Murray","year":"2017","journal-title":"Sci. Data"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"150007","DOI":"10.1038\/sdata.2015.7","article-title":"The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes","volume":"2","author":"Kelly","year":"2015","journal-title":"Sci. Data"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1109\/TSG.2015.2494592","article-title":"Exploiting HMM Sparsity to Perform Online Real-Time Nonintrusive Load Monitoring","volume":"7","author":"Makonin","year":"2016","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sykiotis, S., Athanasoulias, S., Kaselimi, M., Doulamis, A., Doulamis, N., Stankovic, L., and Stankovic, V. (2023). Performance-aware NILM model optimization for edge deployment. IEEE Trans. Green Commun. Netw., 1\u201326.","DOI":"10.1109\/TGCN.2023.3244278"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1007\/s42979-021-00748-9","article-title":"Efficient hardware implementation of decision tree training accelerator","volume":"2","author":"Choudhury","year":"2021","journal-title":"SN Comput. Sci."},{"key":"ref_31","unstructured":"(2023). MATLAB Version: 9.14.0.2206163 (R2023a), The MathWorks Inc."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Stankovic, V., and Stankovic, L. (2020). Building a graph signal processing model using dynamic time warping for load disaggregation. Sensors, 20.","DOI":"10.3390\/s20226628"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4845\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:37:07Z","timestamp":1760125027000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4845"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,17]]},"references-count":32,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23104845"],"URL":"https:\/\/doi.org\/10.3390\/s23104845","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,17]]}}}