{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T19:52:00Z","timestamp":1777405920052,"version":"3.51.4"},"reference-count":54,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T00:00:00Z","timestamp":1714694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Outlier detection plays a critical role in building operation optimization and data quality maintenance. However, existing methods often struggle with the complexity and variability of building energy data, leading to poorly generalized and explainable results. To address the gap, this study introduces a novel Vision-based Outlier Detection (VOD) approach, leveraging computer vision models to spot outliers in the building energy records. The models are trained to identify outliers by analyzing the load shapes in 2D time series plots derived from the energy data. The VOD approach is tested on four years of workday time-series electricity consumption data from 290 commercial buildings in the United States. Two distinct models are developed for different usage purposes, namely a classification model for broad-level outlier detection and an object detection model for the demands of precise pinpointing of outliers. The classification model is also interpreted via Grad-CAM to enhance its usage reliability. The classification model achieves an F1 score of 0.88, and the object detection model achieves an Average Precision (AP) of 0.84. VOD is a very efficient path to identifying energy consumption outliers in building operations, paving the way for the enhancement of building energy data quality, operation efficiency, and energy savings.<\/jats:p>","DOI":"10.3390\/make6020045","type":"journal-article","created":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T05:52:02Z","timestamp":1714715522000},"page":"965-986","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["VOD: Vision-Based Building Energy Data Outlier Detection"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4494-0523","authenticated-orcid":false,"given":"Jinzhao","family":"Tian","sequence":"first","affiliation":[{"name":"School of Architecture, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3808-7549","authenticated-orcid":false,"given":"Tianya","family":"Zhao","sequence":"additional","affiliation":[{"name":"Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, FL 33199, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0036-7028","authenticated-orcid":false,"given":"Zhuorui","family":"Li","sequence":"additional","affiliation":[{"name":"School of Engineering, The University of Kansas, Lawrence, KS 66045, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2123-1679","authenticated-orcid":false,"given":"Tian","family":"Li","sequence":"additional","affiliation":[{"name":"School of Architecture, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-5125-9110","authenticated-orcid":false,"given":"Haipei","family":"Bie","sequence":"additional","affiliation":[{"name":"School of Architecture, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8292-9072","authenticated-orcid":false,"given":"Vivian","family":"Loftness","sequence":"additional","affiliation":[{"name":"School of Architecture, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,3]]},"reference":[{"key":"ref_1","unstructured":"EIA (2022). Buildings Sectorial Overview, Technical Report CC BY 4.0; U.S. Energy Information Administration (EIA)."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100320","DOI":"10.1016\/j.dibe.2024.100320","article-title":"Generalized building energy and carbon emissions benchmarking with post-prediction analysis","volume":"17","author":"Li","year":"2024","journal-title":"Dev. Built Environ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: A survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gaddam, A., Wilkin, T., Angelova, M., and Gaddam, J. (2020). Detecting sensor faults, anomalies and outliers in the internet of things: A survey on the challenges and solutions. Electronics, 9.","DOI":"10.3390\/electronics9030511"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.neucom.2019.01.025","article-title":"Sensor fault detection and diagnosis in the presence of outliers","volume":"349","author":"Xu","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3639","DOI":"10.1109\/TIE.2009.2027926","article-title":"Classification of energy consumption in buildings with outlier detection","volume":"57","author":"Li","year":"2009","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Martin Nascimento, G.F., Wurtz, F., Kuo-Peng, P., Delinchant, B., and Jhoe Batistela, N. (2021). Outlier Detection in Buildings\u2019 Power Consumption Data Using Forecast Error. Energies, 14.","DOI":"10.3390\/en14248325"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Larson, S., Mahendran, A., Lee, A., Kummerfeld, J.K., Hill, P., Laurenzano, M.A., Hauswald, J., Tang, L., and Mars, J. (2019). Outlier detection for improved data quality and diversity in dialog systems. arXiv.","DOI":"10.18653\/v1\/N19-1051"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1504\/IJSNET.2010.033116","article-title":"Ensuring high sensor data quality through use of online outlier detection techniques","volume":"7","author":"Zhang","year":"2010","journal-title":"Int. J. Sens. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"116601","DOI":"10.1016\/j.apenergy.2021.116601","article-title":"Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives","volume":"287","author":"Himeur","year":"2021","journal-title":"Appl. Energy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"105775","DOI":"10.1016\/j.engappai.2022.105775","article-title":"An innovative deep anomaly detection of building energy consumption using energy time-series images","volume":"119","author":"Copiaco","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_12","first-page":"226","article-title":"A density-based algorithm for discovering clusters in large spatial databases with noise","volume":"96","author":"Ester","year":"1996","journal-title":"kdd"},{"key":"ref_13","first-page":"33","article-title":"A New Clustering Method to Identify Outliers and Diurnal Schedules from Building Energy Interval Data","volume":"121","author":"Jalori","year":"2015","journal-title":"ASHRAE Trans."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"113727","DOI":"10.1016\/j.apenergy.2019.113727","article-title":"Recognition and classification of typical load profiles in buildings with non-intrusive learning approach","volume":"255","author":"Piscitelli","year":"2019","journal-title":"Appl. Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"111888","DOI":"10.1016\/j.enbuild.2022.111888","article-title":"A data mining research on office building energy pattern based on time-series energy consumption data","volume":"259","author":"Liu","year":"2022","journal-title":"Energy Build."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"122716","DOI":"10.1016\/j.apenergy.2024.122716","article-title":"MEBA: AI-powered precise building monthly energy benchmarking approach","volume":"359","author":"Li","year":"2024","journal-title":"Appl. Energy"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Price, P. (2010). Methods for Analyzing Electric Load Shape and Its Variability, Technical Report; Lawrence Berkeley National Lab. (LBNL).","DOI":"10.2172\/985909"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1016\/j.apenergy.2017.07.108","article-title":"Electric load shape benchmarking for small-and medium-sized commercial buildings","volume":"204","author":"Luo","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"110601","DOI":"10.1016\/j.enbuild.2020.110601","article-title":"A data mining-based framework for the identification of daily electricity usage patterns and anomaly detection in building electricity consumption data","volume":"231","author":"Liu","year":"2021","journal-title":"Energy Build."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1016\/j.apenergy.2017.12.005","article-title":"Analytical investigation of autoencoder-based methods for unsupervised anomaly detection in building energy data","volume":"211","author":"Fan","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.1109\/TII.2017.2785963","article-title":"Wide and deep convolutional neural networks for electricity-theft detection to secure smart grids","volume":"14","author":"Zheng","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ins.2020.02.069","article-title":"TSI: Time series to imaging based model for detecting anomalous energy consumption in smart buildings","volume":"523","author":"Fahim","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.ins.2019.05.042","article-title":"Combining unsupervised and supervised learning in credit card fraud detection","volume":"557","author":"Carcillo","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bawono, A.H., and Bachtiar, F.A. (2019, January 28\u201330). Outlier Detection with Supervised Learning Method. Proceedings of the 2019 International Conference on Sustainable Information Engineering and Technology (SIET), Lombok, Indonesia.","DOI":"10.1109\/SIET48054.2019.8986101"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1007\/s10994-015-5507-y","article-title":"A decomposition of the outlier detection problem into a set of supervised learning problems","volume":"100","author":"Paulheim","year":"2015","journal-title":"Mach. Learn."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Aggarwal, C.C., and Aggarwal, C.C. (2017). Outlier Analysis, Springer.","DOI":"10.1007\/978-3-319-47578-3"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"108539","DOI":"10.1016\/j.jobe.2024.108539","article-title":"Seasonal threshold to reduce false positives for prediction-based outlier detection in building energy data","volume":"84","author":"Takahashi","year":"2024","journal-title":"J. Build. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.enbuild.2017.02.058","article-title":"An ensemble learning framework for anomaly detection in building energy consumption","volume":"144","author":"Araya","year":"2017","journal-title":"Energy Build."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shoemaker, L., and Hall, L.O. (2011, January 15\u201317). Anomaly detection using ensembles. Proceedings of the Multiple Classifier Systems: 10th International Workshop, MCS 2011, Naples, Italy.","DOI":"10.1007\/978-3-642-21557-5_3"},{"key":"ref_30","first-page":"1","article-title":"Fault Detection and Diagnosis in Building Heat Source Systems Using Machine Learning (Part 2) Preprocessing of Fault Data for Improvement in Diagnosis Performance and Application to BEMS Data","volume":"261","author":"Miyata","year":"2018","journal-title":"Trans. SHASE Jpn."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.ijrefrig.2020.02.035","article-title":"Abnormal energy consumption detection for GSHP system based on ensemble deep learning and statistical modeling method","volume":"114","author":"Xu","year":"2020","journal-title":"Int. J. Refrig."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.apenergy.2015.08.126","article-title":"Comparison of integrated clustering methods for accurate and stable prediction of building energy consumption data","volume":"160","author":"Hsu","year":"2015","journal-title":"Appl. Energy"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"100123","DOI":"10.1016\/j.adapen.2023.100123","article-title":"Interpretable machine learning for building energy management: A state-of-the-art review","volume":"9","author":"Chen","year":"2023","journal-title":"Adv. Appl. Energy"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MCG.2018.042731661","article-title":"Visual analytics for explainable deep learning","volume":"38","author":"Choo","year":"2018","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"111174","DOI":"10.1016\/j.rser.2021.111174","article-title":"Grey-box modeling and application for building energy simulations\u2014A critical review","volume":"146","author":"Li","year":"2021","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"114683","DOI":"10.1016\/j.apenergy.2020.114683","article-title":"Building thermal load prediction through shallow machine learning and deep learning","volume":"263","author":"Wang","year":"2020","journal-title":"Appl. Energy"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"100169","DOI":"10.1016\/j.egyai.2022.100169","article-title":"Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities","volume":"9","author":"Machlev","year":"2022","journal-title":"Energy AI"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"110591","DOI":"10.1016\/j.rser.2020.110591","article-title":"A deep learning framework for building energy consumption forecast","volume":"137","author":"Somu","year":"2021","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, C., Ding, Z., Zhao, D., Yi, J., and Zhang, G. (2017). Building energy consumption prediction: An extreme deep learning approach. Energies, 10.","DOI":"10.3390\/en10101525"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1016\/j.apenergy.2017.03.064","article-title":"A short-term building cooling load prediction method using deep learning algorithms","volume":"195","author":"Fan","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Hawkins, D.M. (1980). Identification of Outliers, Springer.","DOI":"10.1007\/978-94-015-3994-4"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"6481","DOI":"10.1109\/JIOT.2019.2958185","article-title":"Anomaly detection for IoT time-series data: A survey","volume":"7","author":"Cook","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A survey of convolutional neural networks: Analysis, applications, and prospects","volume":"33","author":"Li","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Shafiq, M., and Gu, Z. (2022). Deep residual learning for image recognition: A survey. Appl. Sci., 12.","DOI":"10.3390\/app12188972"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_47","unstructured":"Jocher, G., Stoken, A., Borovec, J., Changyu, L., Hogan, A., Diaconu, L., Poznanski, J., Yu, L., Rai, P., and Ferriday, R. (2020). ultralytics\/yolov5: v3. 0. Zenodo."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wang, Z., Jin, L., Wang, S., and Xu, H. (2022). Apple stem\/calyx real-time recognition using YOLO-v5 algorithm for fruit automatic loading system. Postharvest Biol. Technol., 185.","DOI":"10.1016\/j.postharvbio.2021.111808"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zhu, X., Lyu, S., Wang, X., and Zhao, Q. (2021, January 11\u201317). TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00312"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Zhou, F., Zhao, H., and Nie, Z. (2021, January 22\u201324). Safety helmet detection based on YOLOv5. Proceedings of the 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), Shenyang, China.","DOI":"10.1109\/ICPECA51329.2021.9362711"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18\u201322). Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00913"},{"key":"ref_52","unstructured":"(2024, March 30). Overview of Model Structure about YOLOv5. Available online: https:\/\/github.com\/ultralytics\/yolov5\/issues\/280."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft coco: Common objects in context. Proceedings of the Computer Vision\u2013ECCV 2014: 13th European Conference, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/45\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:39:07Z","timestamp":1760107147000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,3]]},"references-count":54,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["make6020045"],"URL":"https:\/\/doi.org\/10.3390\/make6020045","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,3]]}}}