{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:44:18Z","timestamp":1773931458473,"version":"3.50.1"},"reference-count":28,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T00:00:00Z","timestamp":1690848000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Funds through the Portuguese Funding Agency, FCT\u2013Foundation for Science and Technology Portugal","award":["LA\/P\/0063\/2020"],"award-info":[{"award-number":["LA\/P\/0063\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Forecasting energy consumption models allow for improvements in building performance and reduce energy consumption. Energy efficiency has become a pressing concern in recent years due to the increasing energy demand and concerns over climate change. This paper addresses the energy consumption forecast as a crucial ingredient in the technology to optimize building system operations and identifies energy efficiency upgrades. The work proposes a modified multi-head transformer model focused on multi-variable time series through a learnable weighting feature attention matrix to combine all input variables and forecast building energy consumption properly. The proposed multivariate transformer-based model is compared with two other recurrent neural network models, showing a robust performance while exhibiting a lower mean absolute percentage error. Overall, this paper highlights the superior performance of the modified transformer-based model for the energy consumption forecast in a multivariate step, allowing it to be incorporated in future forecasting tasks, allowing for the tracing of future energy consumption scenarios according to the current building usage, playing a significant role in creating a more sustainable and energy-efficient building usage.<\/jats:p>","DOI":"10.3390\/s23156840","type":"journal-article","created":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T09:32:35Z","timestamp":1690882355000},"page":"6840","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Transformers for Energy Forecast"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4948-550X","authenticated-orcid":false,"given":"Hugo S.","family":"Oliveira","sequence":"first","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science\u2014INESC TEC, University of Porto, 4200-465 Porto, Portugal"},{"name":"Faculty of Sciences (FCUP), University of Porto, 4169-007 Porto, Portugal"}]},{"given":"Helder P.","family":"Oliveira","sequence":"additional","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science\u2014INESC TEC, University of Porto, 4200-465 Porto, Portugal"},{"name":"Faculty of Sciences (FCUP), University of Porto, 4169-007 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/S1062-9769(02)00137-0","article-title":"Energy security: Is the wolf at the door?","volume":"42","author":"Bielecki","year":"2002","journal-title":"Q. Rev. Econ. Financ."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Madni, A.M., Madni, C.C., and Lucero, S.D. (2019). Leveraging digital twin technology in model-based systems engineering. Systems, 7.","DOI":"10.3390\/systems7010007"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"124040","DOI":"10.1016\/j.energy.2022.124040","article-title":"Digital twins model and its updating method for heating, ventilation and air conditioning system using broad learning system algorithm","volume":"251","author":"Chen","year":"2022","journal-title":"Energy"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"110208","DOI":"10.1016\/j.rser.2020.110208","article-title":"Recent advances on industrial data-driven energy savings: Digital twins and infrastructures","volume":"135","author":"Teng","year":"2021","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"194730","DOI":"10.1155\/2013\/194730","article-title":"Forecasting electrical energy consumption of equipment maintenance using neural network and particle swarm optimization","volume":"2013","author":"Jiang","year":"2013","journal-title":"Math. Probl. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"104382","DOI":"10.1016\/j.scs.2022.104382","article-title":"Data-driven prediction of energy consumption of district cooling systems (DCS) based on the weather forecast data","volume":"90","author":"Zhao","year":"2023","journal-title":"Sustain. Cities Soc."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Khahro, S.H., Kumar, D., Siddiqui, F.H., Ali, T.H., Raza, M.S., and Khoso, A.R. (2021). Optimizing energy use, cost and carbon emission through building information modelling and a sustainability approach: A case-study of a hospital building. Sustainability, 13.","DOI":"10.3390\/su13073675"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.isprsjprs.2012.04.004","article-title":"CityGML\u2013Interoperable semantic 3D city models","volume":"71","year":"2012","journal-title":"ISPRS J. Photogramm. Remote. Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Runge, J., and Zmeureanu, R. (2021). A review of deep learning techniques for forecasting energy use in buildings. Energies, 14.","DOI":"10.3390\/en14030608"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4062","DOI":"10.3390\/s22114062","article-title":"Predicting Energy Consumption Using LSTM, Multi-Layer GRU and Drop-GRU Neural Networks","volume":"22","author":"Mahjoub","year":"2022","journal-title":"Sensors"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"110591","DOI":"10.1016\/j.rser.2020.110591","article-title":"A deep learning framework for building energy consumption forecast","volume":"137","author":"Somu","year":"2021","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"109689","DOI":"10.1016\/j.buildenv.2022.109689","article-title":"Occupancy prediction using deep learning approaches across multiple space types: A minimum sensing strategy","volume":"226","author":"Tekler","year":"2022","journal-title":"Build. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1401","DOI":"10.1109\/TITS.2020.2970229","article-title":"Commercial vehicle activity prediction with imbalanced class distribution using a hybrid sampling and gradient boosting approach","volume":"22","author":"Low","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Berriel, R.F., Lopes, A.T., Rodrigues, A., Varejao, F.M., and Oliveira-Santos, T. (2017, January 14\u201319). Monthly energy consumption forecast: A deep learning approach. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, Anchorage, AL, USA.","DOI":"10.1109\/IJCNN.2017.7966398"},{"key":"ref_15","first-page":"22419","article-title":"Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting","volume":"34","author":"Wu","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104445","DOI":"10.1016\/j.autcon.2022.104445","article-title":"Application of deep learning in facility management and maintenance for heating, ventilation, and air conditioning","volume":"141","author":"Sanzana","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_19","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_21","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2019, January 9\u201312). The performance of LSTM and BiLSTM in forecasting time series. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), IEEE, Los Angeles, CA, USA.","DOI":"10.1109\/BigData47090.2019.9005997"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"104495","DOI":"10.1016\/j.rinp.2021.104495","article-title":"Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods","volume":"27","author":"Ayoobi","year":"2021","journal-title":"Results Phys."},{"key":"ref_24","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"902","DOI":"10.1016\/j.rser.2017.02.085","article-title":"A review on time series forecasting techniques for building energy consumption","volume":"74","author":"Deb","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2127","DOI":"10.1007\/s12273-022-0925-9","article-title":"ROBOD, room-level occupancy and building operation dataset","volume":"Volume 15","author":"Tekler","year":"2022","journal-title":"Building Simulation"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1038\/s41597-020-00712-x","article-title":"The Building Data Genome Project 2, energy meter data from the ASHRAE Great Energy Predictor III competition","volume":"7","author":"Miller","year":"2020","journal-title":"Sci. Data"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1038\/s41597-022-01475-3","article-title":"A global building occupant behavior database","volume":"9","author":"Dong","year":"2022","journal-title":"Sci. Data"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6840\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:23:48Z","timestamp":1760127828000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6840"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,1]]},"references-count":28,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23156840"],"URL":"https:\/\/doi.org\/10.3390\/s23156840","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,1]]}}}